From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yb1-xb2d.google.com (mail-yb1-xb2d.google.com [IPv6:2607:f8b0:4864:20::b2d]) by sourceware.org (Postfix) with ESMTPS id 183953858D20 for ; Sun, 11 Jun 2023 10:24:57 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 183953858D20 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-yb1-xb2d.google.com with SMTP id 3f1490d57ef6-bc5a3075e92so1176578276.0 for ; Sun, 11 Jun 2023 03:24:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686479096; x=1689071096; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=koC6250jYd9BY3Nd99dke8ztnXf5mevchoMQ3SXUquU=; b=Fl46rRABrNS/UiCZO9k7bC8F8s63haYavAAqxhMEejiGXCijaPmDMbqqnTVez6WdtA ARYzPTaiOqD6PAbHxnrFlAa6uayxWsK/LAKGZR+5h00cIL5SkekzQPXSjnFB4IdWy75p UOA0l0HN60wUO6wj4pne19szgW76TmeGgaMl+wOhVKgfJkI+2Zx6V+SQXNA1MTMySMam EC2pYQMxwu9SPhl6G38k8DuXxszX/V/3+YxrIWRhhs8PwEiNqz+yO7n9wMPIcE0nrpoL r7j5f+8XkcLl6osD+HMiNzd4+Om5G04gV7T9tedq3rn2TEyz7YWLKRrHpiU2cm/4uwk5 WYUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686479096; x=1689071096; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=koC6250jYd9BY3Nd99dke8ztnXf5mevchoMQ3SXUquU=; b=MuUGzZliODQfdruWxJDhYtoeFplcRppGaKfa18GLcnpwGdp4k2Y7u4p8CVhv/EWfOt oGVbrKJj8XaHzsm/VP6VCvQA3gTtoVuZZbYiIs91giHOn/egW8VaDbNK+dUDfMdIHf1m vmC1W9eMp8+8gcDVoAtVDnNAFVsjyvpRb6BftNWs2+rFMOootn/86aAGSd0MaLsG/I/P q7vzBm5I46oI3z5A5tkhR59OHxYfopsf1TiSPebGLXBRXEHqlwLh20JZnpVhdvBj/Kek X753dmvc7jsCtE2xzvK8ArBWXzyzMKBY2jqEFGWsc9HURatvOugTpsCN4nOM9rBMnWjd kbOw== X-Gm-Message-State: AC+VfDzdOk1D/vaOL370j3ucNw5tekWc9pu5U3t/OVKt4wHq5mfTnNXn eddE8McuazyR7UD+Q43Oa6lADD6vfHDN0JVmgsk= X-Google-Smtp-Source: ACHHUZ6km1qLyIHk5gth6F7IGE68pskZ+XYVhUqJnRpj8UYxIQZlyXi3ShRj3ADYL3WOsz3bHPJ9XX05ZxMGovZ95y4= X-Received: by 2002:a81:7cd5:0:b0:56c:f374:711e with SMTP id x204-20020a817cd5000000b0056cf374711emr3608238ywc.47.1686479096176; Sun, 11 Jun 2023 03:24:56 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Joshua Saxby Date: Sun, 11 Jun 2023 11:24:44 +0100 Message-ID: Subject: Re: Digging a bit deeper into JIT's compile internals To: David Malcolm Cc: jit@gcc.gnu.org Content-Type: multipart/alternative; boundary="00000000000072e4c405fdd803ce" X-Spam-Status: No, score=0.5 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,HTML_MESSAGE,KAM_SHORT,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: --00000000000072e4c405fdd803ce Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi David, Thanks very much for that helpful information. I think I now know what I need to do to achieve my goal. Essentially I will be changing the structure of the code slightly so that .s file from the initial compilation gets re=C3=BCsed when compiling to both file and memory. It'll probably take me quite a while to modify this, but if I am successful I will post again on mailing list with an update. Best Regards, J.S. On Sat, 10 Jun 2023, 22:47 David Malcolm, wrote: > On Sat, 2023-06-10 at 17:26 +0100, Joshua Saxby via Jit wrote: > > Hi Joshua > > > After following files jit-recording.cc and jit-playback.cc, I think > > I've > > found out where I need to patch JIT to do what I want it to do. > > Indeed. In case you haven't seen it yet, see: > > https://gcc.gnu.org/onlinedocs/jit/internals/index.html#overview-of-code-= structure > > You'll see there (and in jit-playback.cc) that the > gcc_jit_compile[_to_file] functions are basically running the > equivalent of cc1 in-process to generate a .s file in a tempdir, and > then calling out to subprocesses to run the assembler and linker as > needed. FWIW I've experimented in the past with libraries for the > assembler and linker (keeping it all in-process), and got some > speedups, but it needed nontrivial patches to binutils to turn as and > ld into shared libraries. > > > Looks like both compiling to file and memory ultimately rely upon > > playback::context::compile() to do the bulk of their work and then > > override > > a postprocess() method to do additional handling pertaining to their > > specific task. > > > > If I can find a way to cache or store the intermediate result > > generated by > > compile(), I should then be able to restructure this to have the > > follow-up > > tasks required by both different forms of compilation to be done > > starting > > with this intermediate result. > > > > Anyone see a problem with my approach? I'm hoping it will be possible > > to > > re=C3=BCse compilation results produced by compile() in this way without > > it > > causing any conflicts... > > I have a higher-level question, which is why do you want to compile to > both/reuse results? > > I believe you ought to be able to compile a context multiple times > (provided no errors occur), so if you need both you can currently write > code like this: > > gcc_jit_context *ctxt =3D populate_ctxt (); > gcc_jit_context_compile_to_file (ctxt, > GCC_JIT_OUTPUT_KIND_ASSEMBLER, > "foo.s"); > gcc_jit_result *result =3D gcc_jit_context_compile (ctxt); > > albeit with the drawback that it's duplicating work. Is it the > duplicated work that you're trying to avoid? > > Do you have ideas about what the API you'd want would look like? > > [...snip...] > > Hope this is helpful > Dave > > > > > > > ---------- Forwarded message ---------- > > > From: Joshua Saxby > > > To: jit@gcc.gnu.org > > > Cc: > > > Bcc: > > > Date: Sat, 10 Jun 2023 00:04:43 +0100 > > > Subject: Modifying the jit compiler API? > > > Currently the JIT has two functions allowing you to compile a > > > context > > > either to memory or to file. > > > > > > But what if you want to compile to both? There doesn't seem to be > > > any way > > > to do this except by calling both functions separately which I > > > believe will > > > effectively be two separate compilations... > > > > > > Presumably, it should be possible to modify this part of the API to > > > compile > > > to some form of intermediate representation of the work that is > > > common to > > > both kinds of compilation, and then turn that into code in memory > > > and in > > > file, respectively. > > > > > > Anyone got any pointers for me on where in the code would be the > > > best place > > > for me to modify to dupport this? I did take a look in the > > > implementation > > > code of JIT sone weeks ago and remember seeing lots of complicated > > > stuff > > > regarding recordings and replays that looked relevant... > > > > > > A slighty simpler alteration I am also interested in making is > > > allowing > > > compile to file to compile to a buffer in memory as well as a file > > > (I > > > suppose it could be termed "compile to binary"). Maybe I should > > > start with > > > that... > > > > > > > > > > --00000000000072e4c405fdd803ce--