Re: Rust front-end

[Philip Herron <philip.herron@embecosm.com>]

Hi Richard

Thanks for your detailed response, I took some time to figure this out
myself to give a decent response. It seems like we should keep the end
of the year as our goal to aim for but in the meantime see if we can
split patches which affect GCC over the next month or so. We have no
changes to the GCC gimple/generic IRs. Usually, when I hit something
inside the GCC middle-end, the front-end is doing something wrong,
which has helped keep me on a good path. Other than that we have
changes to:

1. Grabbing target info using the TARGET_HOOKS interfaces in gcc/config.
2. Tweaks to selftest which was already merged last year by Arthur Cohen
3. We have some minor issues with lang.opt in the latest merge from
upstream which Thomas Schwinge is working on, but I believe we can
work around this if we want
4. Our compiler driver needs some cleanup which we can do in the short
term to get it reviewed
5. We need to make some build changes to incorporate libcore being
built by gccrs which is still WIP.

As for the compile link cycle, we mostly reuse the model from the GO
front-end. In Rust the "crate" is a compilation unit, which means if
you have a project with multiple files, you point gccrs at a single
src/source.rs, the main entry point for a library (usually lib.rs).
Keywords such as "mod foo", trigger the expansion of a relative path
of foo.rs like a C++ included inside a namespace. All source files are
then included inside this single compilation unit. When the
compilation is successful, we reuse code from the Go front-end to put
custom front-end metadata into a section ".rust_export". At this
point, all source files are compiled into a single object file, which
can be compiled into an archive or shared library as required. To link
against this, it again follows similar to Go front-end, whereby the
source.rs has a declaration such as "extern crate foo"; the search
code will look for foo.o or libfoo.a (I haven't tested against shared
libraries yet) and grab the metadata out of it and parse it in the
front-end for all the necessary information such as types, public
functions and generics, etc., so we can compile any imports correctly
and emit the correct mangled symbols for linking.

Given we are still working on this I think we can try to line up all
the other GCC relating pieces for review over the summer, do we send
this as usual to gcc-patches?

Again thanks to everyone for helping me navigate this and answering my
questions.

--Phil

On Tue, 28 Jun 2022 at 08:30, Richard Biener <richard.guenther@gmail.com> wrote:
>
> On Mon, Jun 27, 2022 at 4:52 PM Philip Herron
> <philip.herron@embecosm.com> wrote:
> >
> > Hi everyone,
> >
> > Since November 2020, I've worked full-time on the Rust front-end for
> > GCC, thanks to Open Source Security, Inc and Embecosm. As a result, I
> > am writing to this mailing list to seek feedback from the collective
> > experience here early to plan a path for upstreaming the front-end
> > into GCC.
> >
> > 1. What is the actual process of merging a prominent feature like this upstream
> >   - How do we review this?
> >   - Do we create a "mega-commit" patch
> >   - How long should we expect this review process to take
> >   - Is there anything we can do to make this easier?
>
> Usually a new frontend is first proposed for merge and generally approved
> by the steering committee (which should also sort out legal issues).
>
> For the actual review process it's best to consult previous frontend
> merges - the most recent merged frontend was the D frontend and the
> modula2 frontend is in the process of being reviewed.
>
> To be able to focus on the possibly controversical pieces separating
> out changes to the generic GCC code base (such as driver or
> even middle-end) should be separated out.
>
> It would also be helpful to provide an overview of how a rust
> compile + link cycle works through the pieces in GCC (see the
> modula-2 case where that involved creating stub C++ code,
> compiling and linking that and how this is now done much
> more straight-forward).
>
> > 2. What sort of quality does the GCC community expect?
> >   - I think it is essential that we can compile valid test cases from
> > a testsuite and real projects before merging.
> >   - It seems reasonable that our error handling may not be 100% but be
> > expected to improve over time
> >   - Upon merging, can features like Rust be marked as experimental
>
> Rust can be marked as experimental, sure.  It would be not enabled
> to be built by default (and you can have a whitelist of supported targets).
> The most important part would be that the build works when enabled
> and that most of the existing testsuite passes so it can be used to
> regression test middle-end changes.
>
> If it is not useful at all for (basic) real-world usage then it might be not
> ready yet.
>
> > 3. How do GCC releases work?
> >   - If you miss a window can we still merge code into the front-end?
> >   - Can we merge without a borrow checker and backport this in the future?
>
> The rust frontend will not be part of the release critical pieces of the
> compiler (which includes the C and C++ frontends plus the set of
> primary and secondary targets) so it is up to the maintainers to decide
> what to merge and when.  Release managers will generally ignore
> issues in Rust.
>
> > 4. What about the possibility of merging sooner rather than later,
> > which would help the project gain interest through the increased
> > visibility of it as part of the GCC family.
> >   - Does this still allow for development churn, or will it cause friction?
>
> The parts where GCC and Rust overlap still need to be reviewed and
> _some_ usability for users should be provided.
>
> > 5. Does anyone have prior experience or advice they could give us?
>
> I suppose Ian (for the Go frontend) or Iain (for the D frontend) can give
> you hints.
>
> > For some context, my current project plan brings us to November 2022
> > where we (unexpected events permitting) should be able to support
> > valid Rust code targeting Rustc version ~1.40 and reuse libcore,
> > liballoc and libstd. This date does not account for the borrow checker
> > feature and the proc macro crate, which we have a plan to implement,
> > but this will be a further six-month project.
> >
> > Regarding patch management, we currently do our development on GitHub:
> > https://github.com/Rust-GCC/gccrs; this means we can integrate our
> > issue tracking with the official Rust project by linking back to the
> > official Rust project's RFC issues, for example. The downside is that
> > when someone uses our compiler and hits an ICE, they will be directed
> > to the GCC Bugzilla, which is correct but can lead to a mismatch in
> > issue tracking. Nevertheless, I think it's essential to have the
> > GitHub link here to integrate with the broader Rust community. I
> > believe we can triage Rust issues on the Bugzilla and raise associated
> > ones on Github to manage this.
> >
> > From my perspective as the lead on this front-end, we are currently
> > under heavy development, so this means a fair amount of code churn
> > still, and I don't see this changing until we can successfully compile
> > the libcore crate later this year. Although I would love to see us
> > merged into GCC 13, I want to make sure this project is a success for
> > everyone, and this might mean pushing back to the next release window
> > to make sure this is manageable to produce a quality front-end to sit
> > alongside the others.
>
> If you want to target GCC 13 for experimental Rust support I suggest to
> get review on the overall design (where it touches GCC) and the changes
> necessary to driver and build changes.  The core frontend itself will usually
> only get review on the parts that interface to the middle-end (thus
> GENERIC code generation and language hooks).  Dropping in the frontend
> during Stage3 (thus until the end of the year) should be possible, especially
> if the driver and build changes have been reviewed already.
>
> Richard.
>
> > I wish to thank you those in the GCC developer community, who have
> > inspired me and helped me navigate my journey to this point in time.
> >
> > - Thomas Schwinge
> > - Mark Wielaard
> > - Tom Tromey
> > - Ian Lance Taylor
> > - David Edelsohn
> > - David Malcolm
> > - Martin Jambor
> >
> > Thanks
> >
> > –Phil