libgccjit questions

libgccjit questions

[Daurnimator]

Hi,

I'm looking into using libgccjit as an ahead-of-time compiler to write
a backend for zig (https://ziglang.org/)
I've got a few questions I hope you can answer:

What became of cross-compilation? I saw it mentioned as a possible
roadmap item in
https://gcc.gnu.org/legacy-ml/jit/2015-q3/msg00086.html)

How can I support inline assembly?

Would it be reasonable to add an API that allows extracting sections
without writing out to a file and reading it back in?
Alteratively, would it be possible to get a pointer and length to e.g.
an ELF in memory?

Should distros be packaging libgccjit?
I noticed that e.g. ArchLinux is not. Possibly because it is
documented as "experimental"; possibly because it requires building
gcc with --enable-host-shared ?

What is the license situation with libgccjit; is it GPL? LGPL? what
does this mean for code that uses libgccjit?

Thanks,
Daurnimator

Re: libgccjit questions

[Andrea Corallo]

Daurnimator <quae@daurnimator.com> writes:

> Hi,

Hi!

> I'm looking into using libgccjit as an ahead-of-time compiler to write
> a backend for zig (https://ziglang.org/)
> I've got a few questions I hope you can answer:
>
> What became of cross-compilation? I saw it mentioned as a possible
> roadmap item in
> https://gcc.gnu.org/legacy-ml/jit/2015-q3/msg00086.html)

AFAIK cross compilation is not supported and nothing is going on in that
sense.  I believe this is in line with the idea of libgccjit being
mainly intended for making jitters.

> How can I support inline assembly?

I believe you cannot use inline assembly as we have no specific support
for that and asm and __asm__ are not builtin functions so they can't be
used with gcc_jit_context_get_builtin_function.

> Would it be reasonable to add an API that allows extracting sections
> without writing out to a file and reading it back in?

The elf file is always produced because the assembler is invoked as a
child process, so as of today this cannot happen in memory.

> Alteratively, would it be possible to get a pointer and length to e.g.
> an ELF in memory?
>
> Should distros be packaging libgccjit?
> I noticed that e.g. ArchLinux is not. Possibly because it is
> documented as "experimental"; possibly because it requires building
> gcc with --enable-host-shared ?

Most distros are already packing libgccjit, even if the quality of the
package may vary.

> What is the license situation with libgccjit; is it GPL? LGPL? what
> does this mean for code that uses libgccjit?

AFAIU libgccjit is part of GCC and this is released under the GNU
General Public License.

> Thanks,
> Daurnimator

Bests

  Andrea
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

Re: libgccjit questions

[David Malcolm]

On Wed, 2020-09-09 at 10:18 +0200, Andrea Corallo wrote:
> Daurnimator <quae@daurnimator.com> writes:
> 
> > Hi,
> 
> Hi!

Hi!

> > I'm looking into using libgccjit as an ahead-of-time compiler to
> > write
> > a backend for zig (https://ziglang.org/)
> > I've got a few questions I hope you can answer:
> > 
> > What became of cross-compilation? I saw it mentioned as a possible
> > roadmap item in
> > https://gcc.gnu.org/legacy-ml/jit/2015-q3/msg00086.html)
> 
> AFAIK cross compilation is not supported and nothing is going on in
> that
> sense.  I believe this is in line with the idea of libgccjit being
> mainly intended for making jitters.

I think it would be possible to do ahead-of-time cross-compilation with
libgccjit, but I can think of some issues.

When I last looked at it (years ago) I got bogged down in the question
of "which target are you compiling for?" with thoughts about being able
to support multiple targets, with different backends as DSOs loaded by
libgccjit.  But that approach would involve a massive refactoring, and
most of my cycles these days are on the static analyzer feature I added
in GCC 10.

Rethinking this, a less ambitious approach would be to simply support
configuring libgccjit for a different target with the usual --target=
param at configure time, and make it the client code's responsibility
to load the correct libgccjit.so (and there could only be one linked
into any given process).

In theory that would work, but given the non-standard way that
libgccjit integrates with the gcc driver code I'd anticipate issues
with converting the assembler code to .o and executables (you'd need
binutils configured for the relevant target also, and libgccjit would
need to be able to find it).

(I'm not personally very familiar with building cross toolchains; help
with that would be appreciated if we pursue this idea)

> > How can I support inline assembly?
> 
> I believe you cannot use inline assembly as we have no specific
> support
> for that and asm and __asm__ are not builtin functions so they can't
> be
> used with gcc_jit_context_get_builtin_function.

I've written a mostly complete patch for inline assembly here:
  https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87291
which I hope to get into the tree for GCC 11.

> > Would it be reasonable to add an API that allows extracting
> > sections
> > without writing out to a file and reading it back in?
> 
> The elf file is always produced because the assembler is invoked as a
> child process, so as of today this cannot happen in memory.
> 
> > Alteratively, would it be possible to get a pointer and length to
> > e.g.
> > an ELF in memory?
> > 
> > Should distros be packaging libgccjit?
> > I noticed that e.g. ArchLinux is not. Possibly because it is
> > documented as "experimental"; possibly because it requires building
> > gcc with --enable-host-shared ?

Given that we've had > 5 years of ABI-compatible releases I removed the
"alpha" warnings from https://gcc.gnu.org/wiki/JIT back in May.

I would like distros to package it.

Some notes on packaging are at:
https://gcc.gnu.org/onlinedocs/jit/internals/index.html#packaging-notes


> Most distros are already packing libgccjit, even if the quality of
> the
> package may vary.
> 
> > What is the license situation with libgccjit; is it GPL? LGPL? what
> > does this mean for code that uses libgccjit?
> 
> AFAIU libgccjit is part of GCC and this is released under the GNU
> General Public License.

Thanks Andrea for your responses.

I hope mine are also helpful.

Dave

Re: libgccjit questions

[Daurnimator]

On Fri, 11 Sep 2020 at 08:50, David Malcolm <dmalcolm@redhat.com> wrote:
> When I last looked at it (years ago) I got bogged down in the question
> of "which target are you compiling for?" with thoughts about being able
> to support multiple targets, with different backends as DSOs loaded by
> libgccjit.  But that approach would involve a massive refactoring

This sounds reasonable to me....

> Rethinking this, a less ambitious approach would be to simply support
> configuring libgccjit for a different target with the usual --target=
> param at configure time, and make it the client code's responsibility
> to load the correct libgccjit.so (and there could only be one linked
> into any given process).

At first glance, this seems like it could be on the route to the previous idea.
Conceivably you could have a sort of proxy-shared object that loads
the target-specific libgccjit.
I'm imagining that when you create a context you'd specify a target
architecture: if the target-specific-libgccjit isn't installed then
the context creation would fail.

> In theory that would work, but given the non-standard way that
> libgccjit integrates with the gcc driver code I'd anticipate issues
> with converting the assembler code to .o and executables (you'd need
> binutils configured for the relevant target also, and libgccjit would
> need to be able to find it).

So for inline assembly, does libgccjit call out to binutils' gas? Or
is it used somehow as a library?
Are other pieces of binutils actually needed?

> (I'm not personally very familiar with building cross toolchains; help
> with that would be appreciated if we pursue this idea)

We currently link against libllvm for this and we can target any
architecture we wish with the one binary/library. I'm hoping we can
one day use GCC in a similar way.

> > > How can I support inline assembly?
> >
> > I believe you cannot use inline assembly as we have no specific
> > support
> > for that and asm and __asm__ are not builtin functions so they can't
> > be
> > used with gcc_jit_context_get_builtin_function.
>
> I've written a mostly complete patch for inline assembly here:
>   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87291
> which I hope to get into the tree for GCC 11.

Looks great! Will be happy to give this a try once merged/released.

> > > Would it be reasonable to add an API that allows extracting
> > > sections
> > > without writing out to a file and reading it back in?
> >
> > The elf file is always produced because the assembler is invoked as a
> > child process, so as of today this cannot happen in memory.
> >
> > >
> > > Should distros be packaging libgccjit?
> > > I noticed that e.g. ArchLinux is not. Possibly because it is
> > > documented as "experimental"; possibly because it requires building
> > > gcc with --enable-host-shared ?
>
> Given that we've had > 5 years of ABI-compatible releases I removed the
> "alpha" warnings from https://gcc.gnu.org/wiki/JIT back in May.
>
> I would like distros to package it.
>
> Some notes on packaging are at:
> https://gcc.gnu.org/onlinedocs/jit/internals/index.html#packaging-notes

What would the penalty be of always compiling with
--enable-host-shared ? The docs note "slight performance hit": but is
it actually significant?

> > > What is the license situation with libgccjit; is it GPL? LGPL? what
> > > does this mean for code that uses libgccjit?
> >
> > AFAIU libgccjit is part of GCC and this is released under the GNU
> > General Public License.

Is my understanding correct then that a zig compiler built with
libgccjit support would then become GPL? (and otherwise it would be
MIT licensed)