From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTPS id CADA23858D28 for ; Sat, 10 Jun 2023 21:47:52 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org CADA23858D28 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686433672; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uPoiOjIuQFUhVRBaDqI/mIBUPKb1khpXZ2ECUR1X7B4=; b=jDJhaP/EW0wIr/iJmIvHr+Klz5lgkMByOtS/E7X9FGe9nhH+8Q+JFR+zC5iAoUl+tT8JJ1 c4634DUxGEkHRFmVXa3bkfuHKCt1sNYT81N3MEZyH1h57zTNrPjYbyTN/ychXtgKzsuRjE RJwVmMY94ldBC8FWU+OhzF42Ujcm5JY= Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-423-Q311EiqGMmG9RgMUZp91iw-1; Sat, 10 Jun 2023 17:47:51 -0400 X-MC-Unique: Q311EiqGMmG9RgMUZp91iw-1 Received: by mail-qv1-f72.google.com with SMTP id 6a1803df08f44-5eee6742285so37794836d6.2 for ; Sat, 10 Jun 2023 14:47:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686433670; x=1689025670; h=mime-version:user-agent:content-transfer-encoding:references :in-reply-to:date:to:from:subject:message-id:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=uPoiOjIuQFUhVRBaDqI/mIBUPKb1khpXZ2ECUR1X7B4=; b=duadtErLwWu+nXKAmt87PsFn0VpPtk/noI7aXn/vH2RS2OR4iF13q7GpbvY0sHubmA afyAcPItpXypiicm/ZyNO2i/1v6308Nw37Ym5ByhHLxzKFkqC8AaJVU0poZ/lKqrZr8m 9WC4KrG1MUMMFTI+8EosjM6HDTr9kPwKgUg/ijQmnxq0FkrUp5wkvxZufQ43pI1exEo1 K7nYlalEuecSYrbin8OSWrF0ptF+Qx5omyBv8yfsNWAAkszfXVBGsYW0oFt66TlfXeuo vyAJBLbmoAVKB/HaJoLO/NvS7fEKhkbtsKVdGbKF/94upSyuZ3HG8XMnmiUin06JOyRy rvtg== X-Gm-Message-State: AC+VfDwIz1u+Igx9stgFcHpA41/bBKSh8IqM1xE8sUphAJxVOQwtc4/o LEd95iK4UHFVBWEb77hel9IM7/7okpnRqBw38xyWQIHnAKcJQ8dYdYzYjBZQdLW9B3q57vlWD2o wOHJCE6nG7hne2UA= X-Received: by 2002:ad4:4ee9:0:b0:626:1893:6266 with SMTP id dv9-20020ad44ee9000000b0062618936266mr7161254qvb.9.1686433670371; Sat, 10 Jun 2023 14:47:50 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6TuBpYWMd/BtuE+3VgmaEcerq+niDYEvfEsjKZ5SqQxW4hkiGAt/QZk58xMZpe6KhUiFjPJw== X-Received: by 2002:ad4:4ee9:0:b0:626:1893:6266 with SMTP id dv9-20020ad44ee9000000b0062618936266mr7161245qvb.9.1686433670111; Sat, 10 Jun 2023 14:47:50 -0700 (PDT) Received: from t14s.localdomain (c-76-28-97-5.hsd1.ma.comcast.net. [76.28.97.5]) by smtp.gmail.com with ESMTPSA id kr30-20020a0562142b9e00b006286334f999sm2061094qvb.78.2023.06.10.14.47.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 10 Jun 2023 14:47:49 -0700 (PDT) Message-ID: Subject: Re: Digging a bit deeper into JIT's compile internals From: David Malcolm To: Joshua Saxby , jit@gcc.gnu.org Date: Sat, 10 Jun 2023 17:47:48 -0400 In-Reply-To: References: User-Agent: Evolution 3.44.4 (3.44.4-2.fc36) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-5.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,KAM_SHORT,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_NONE,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Sat, 2023-06-10 at 17:26 +0100, Joshua Saxby via Jit wrote: Hi Joshua > After following files jit-recording.cc and jit-playback.cc, I think > I've > found out where I need to patch JIT to do what I want it to do. Indeed. In case you haven't seen it yet, see: https://gcc.gnu.org/onlinedocs/jit/internals/index.html#overview-of-code-st= ructure You'll see there (and in jit-playback.cc) that the gcc_jit_compile[_to_file] functions are basically running the equivalent of cc1 in-process to generate a .s file in a tempdir, and then calling out to subprocesses to run the assembler and linker as needed. FWIW I've experimented in the past with libraries for the assembler and linker (keeping it all in-process), and got some speedups, but it needed nontrivial patches to binutils to turn as and ld into shared libraries. > Looks like both compiling to file and memory ultimately rely upon > playback::context::compile() to do the bulk of their work and then > override > a postprocess() method to do additional handling pertaining to their > specific task. >=20 > If I can find a way to cache or store the intermediate result > generated by > compile(), I should then be able to restructure this to have the > follow-up > tasks required by both different forms of compilation to be done > starting > with this intermediate result. >=20 > Anyone see a problem with my approach? I'm hoping it will be possible > to > re=C3=BCse compilation results produced by compile() in this way without > it > causing any conflicts... I have a higher-level question, which is why do you want to compile to both/reuse results? I believe you ought to be able to compile a context multiple times (provided no errors occur), so if you need both you can currently write code like this: gcc_jit_context *ctxt =3D populate_ctxt (); gcc_jit_context_compile_to_file (ctxt, GCC_JIT_OUTPUT_KIND_ASSEMBLER, "foo.s"); gcc_jit_result *result =3D gcc_jit_context_compile (ctxt); albeit with the drawback that it's duplicating work. Is it the duplicated work that you're trying to avoid? Do you have ideas about what the API you'd want would look like? [...snip...] Hope this is helpful Dave > >=20 > > ---------- Forwarded message ---------- > > From: Joshua Saxby > > To: jit@gcc.gnu.org > > Cc: > > Bcc: > > Date: Sat, 10 Jun 2023 00:04:43 +0100 > > Subject: Modifying the jit compiler API? > > Currently the JIT has two functions allowing you to compile a > > context > > either to memory or to file. > >=20 > > But what if you want to compile to both? There doesn't seem to be > > any way > > to do this except by calling both functions separately which I > > believe will > > effectively be two separate compilations... > >=20 > > Presumably, it should be possible to modify this part of the API to > > compile > > to some form of intermediate representation of the work that is > > common to > > both kinds of compilation, and then turn that into code in memory > > and in > > file, respectively. > >=20 > > Anyone got any pointers for me on where in the code would be the > > best place > > for me to modify to dupport this? I did take a look in the > > implementation > > code of JIT sone weeks ago and remember seeing lots of complicated > > stuff > > regarding recordings and replays that looked relevant... > >=20 > > A slighty simpler alteration I am also interested in making is > > allowing > > compile to file to compile to a buffer in memory as well as a file > > (I > > suppose it could be termed "compile to binary"). Maybe I should > > start with > > that... > >=20 > >=20 >=20