Re: [Request for Comments] Using Rust libraries in the Rust frontend

[connor horman <chorman64@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 6650 bytes --]

Note that it may not make sense to include a source copy of rustc, as that
will itself require an (earlier) stage of rustc to build. mrustc can offer
a bootstrap for 1.54, but depending on the versions required, you may need
upwards of 10 additional rustc sources.

On Thu, 25 Jan 2024 at 10:04, Arthur Cohen <arthur.cohen@embecosm.com>
wrote:

> Hi Richard,
>
> On 1/23/24 08:23, Richard Biener wrote:
> > On Mon, Jan 22, 2024 at 7:51 PM Arthur Cohen <arthur.cohen@embecosm.com>
> wrote:
> >>
> >> Hi everyone,
> >>
> >> In order to increase the development speed of Rust features, we are
> >> seeking feedback on reusing some Rust components directly within our
> >> front-end. As mentioned in other conferences, the most important
> >> component we would like to integrate into the front-end is the polonius
> >> borrow-checker. Another component we've become interested in is the
> >> `rustc_format_parser` library, responsible for handling parsing and
> >> handling of format arguments as described in the documentation for
> >> `std::fmt` [1].
> >>
> >> However, since these libraries are written in Rust, GCC in itself is not
> >> yet able to compile them. They all depend on the Rust standard library,
> >> which we cannot yet compile and link properly. This obviously raises a
> >> question - how to actually compile, integrate and distribute these
> >> components?
> >>
> >> We do have the option to rewrite them from scratch, but we feel that
> >> spending time on these existing, correct, battle-tested and easily
> >> integrable components instead of focusing on other aspects of the
> >> language would be a mistake. Spending this time instead on Rust features
> >> that we are missing for compiling these components would, in our
> >> opinion, yield more results, as it would also help in compiling other
> >> Rust programs.
> >>
> >> We could either distribute these components as compiled libraries, or
> >> look at integrating the official Rust compiler to our build system as a
> >> temporary measure. I am aware that this would mean restricting the Rust
> >> GCC front-end to platforms where the official Rust compiler is also
> >> available, which is less than ideal.
> >
> > But that's only for the host part - you can still cross compile to
> another
> > target and possibly, once the GCC frontend can compile these libraries
> > itself, also use them to bootstrap a hosted version on that target -
> > speaking of ..
> >
> >> However, this would only be
> >> temporary - as soon as the Rust front-end is able to compile these
> >> components, we would simply reuse them and compile them with gccrs as
> >> part of our bootstrapping process.
> >
> > .. gccrs would then need to be able to build itself without those
> modules,
> > at least during stage1 so that the stage1 compiler can then be used to
> > build them.  Or you go like m2 and build a "mini-rust" that's just
> capable
> > of building the modules.
>
> Right, that makes a lot of sense. We should definitely be able to build
> the format string parser without a format string parser, as it does not
> use format strings for error handling or anything. And even if it did,
> it would be pretty easy to remove that and do the formatting by hand.
>
> Similarly, the borrow checker is not "needed" for compilation and we do
> plan on building stage1 without it, while making it mandatory for
> stage2/3 builds.
>
> > I think re-using parts already available is very sensible at this
> point.  Note
> > that while we might temporarily require a host rust compiler to boostrap
> > gccrs I'd rather not have the build system download something from the
> > internet - so at least the sources of those dependences need to be in the
> > GCC repository, possibly in a new toplevel directory.
>
> Okay, that makes a lot of sense. I was thinking of adding a basic check
> for the Rust compiler to be present when compiling these components -
> and error out if that isn't the case. Are you suggesting we embed a full
> copy of rustc in GCC and build it from source when compiling the Rust
> frontend? Or am I misunderstanding?
>
> >> The documentation for `std::fmt` [1] describes all of the features
> >> available in Rust format strings. It also contains a grammar for the
> >> format-string parser, which we would need to re-implement on top of
> >> supporting all the formatting features. As a prototype, I wrote an FFI
> >> interface to the `rustc_format_parser`  library and integrated it to our
> >> macro expansion system, which took me less than a couple hours. In less
> >> than an afternoon, we had bindings for all of the exported types and
> >> functions in the library and had access to a compliant and performant
> >> Rust format string parser. But re-implementing a correct
> >> `format_args!()` parser - with the same performance as the Rust one, and
> >> the same amount of features - would probably take days, if not weeks.
> >> And we are talking about one of the simplest components we aim to reuse.
> >> Something like a fully-compliant trait solver for the Rust programming
> >> language would take months if not years to implement properly from
> scratch.
> >>
> >> I would like to stress once again that relying on distributing compiled
> >> libraries or using `rustc` in our build system would be temporary, and
> >> that these measures would be removed as soon as gccrs is able to compile
> >> these components from source.
> >>
> >> I am looking for comments on this decision as well as the best practices
> >> we could adopt. Have there been any similar decisions for other
> >> self-hosted front-ends? Any previous email threads/commits that you
> >> could point me to would be greatly appreciated.
> >
> > Not in this very same way but the D frontend is a shim around the
> official
> > DMD frontend and the Go frontend imports the golang standard library
> > (but the frontend itself is written in C++ and doesn't use any part of
> it).
> >
> > The C++ frontend uses part of the C++ standard library (but it can of
> > course build that itself - but it requires a host C++ compiler with
> library).
>
> Thanks for the pointers. I was wondering if this is something that the
> Ada frontend had faced at the beginning, but I've been told it does not
> have a lot of dependencies anyway so this might not be helpful.
>
> >
> > Richard.
> >
> >> Thanks,
> >>
> >> Arthur
> >>
> >> [1]: https://doc.rust-lang.org/std/fmt/
>
> Thanks a lot for taking the time, I really appreciate it.
>
> Best,
>
> Arthur
>