From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com [IPv6:2a00:1450:4864:20::12e]) by sourceware.org (Postfix) with ESMTPS id 85A003858D28 for ; Tue, 2 Apr 2024 22:31:25 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 85A003858D28 Authentication-Results: sourceware.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=google.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 85A003858D28 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2a00:1450:4864:20::12e ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1712097088; cv=none; b=r8nsqpOy4lRNb5p+27d1yDhq0o84U6Xofi3rHwiQzOQmY0eeje2Ny5j16u9UqCXL1XHyCbIR6SaRWzm0wGkzzmDE+jbcoNbFKtV2vxa5y/PX75F6uTmfJw27k9sjoTMPhdHsldRfegmOWq5By5bcZBu80G2c/5nbD+cQgu/Y2CU= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1712097088; c=relaxed/simple; bh=IJl8cMdv5vocPj3GANNrFIpt8ecDIAfFPoeAEVApn+E=; h=DKIM-Signature:MIME-Version:From:Date:Message-ID:Subject:To; b=eHOSBVLCenf5VKbayREKex0yPYZAaYncwBFqSWaNvnU4/uLqz/ctcuisymztuK85AfqTrP8n40CVMHXhXVMp28YfMVg77SQZYjus72qWAHgNSjyPNRt3dc/CE8ojPrGdHWioJOXp/Il6PLWV4+CHRqTZzM7Nvm+gs0gLa1StIME= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-lf1-x12e.google.com with SMTP id 2adb3069b0e04-515a81928a1so9475412e87.3 for ; Tue, 02 Apr 2024 15:31:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1712097084; x=1712701884; darn=gcc.gnu.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=4xkXfX1LTlT5jtm838iP0CbqEzf4SJSYik76+c7e3xA=; b=MmIs+ciqjr4WZ5H+j8i1chMeOWIn7VM5KBRcZ/FwIOtoBGN7KMqAcJut1foZ6IeTQk ghrKxl5w34FUG26FOpssTw//yanh6l5sDLRDxCmt3pYLrsDEZgQYmbYATeaWwTvTE3hC VfgeTWq1eyaBL082q7so/E2WPexkGZy1YiMhk/+wYrQSqaPjQ8DsnBXcJWge3KOwZaao +4ifa4bHfWyprSTv3FBrDg1A0ddslQeWUpZ4JtNkYpvFZh0kK/ToPnjhaYcLBdVuXRGJ dqCYjTvqjsh2caVM71xM5PLZBfyAcs74mlRcXz++MiFaszggS5kRvBm2NNC32Izok7Nh pzKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712097084; x=1712701884; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=4xkXfX1LTlT5jtm838iP0CbqEzf4SJSYik76+c7e3xA=; b=o9FHKNGzfCGvDXi5hYAy69fbuKOghAPza20/hg9LTqBO/XD4M4bv0qpu0WOh8xBa6N qnK+FAFafYxDFvxiwP4Su45cop2OkgrO6h4FXPtqhfi9vqyuaVd5FkGVB8JWihkR8yO9 xoAvaH3de84DQVxLvAgykO6HQT+glrJNtEuikF3V1DeIijhJOFHZv6svpD7Y1z1/RNOD t4PAOIgz/D3j0PRJwhMNPMCB8LQxrZM57nAC+q+9V/P9uEJ0exDaNBF/bZwNJF0eQIUx 9dg/2SDUrQ3F2vx3Ophyai04ivnZh61NpVRfr0RgXLxTus969ErEmE2tP/uRPkoHyx2I RHMQ== X-Forwarded-Encrypted: i=1; AJvYcCU1UnGo+y+jwedwOcTjhvzIBFJPGye7DOI7AzV4tmFzW576meI/5/tnREAiz9/9DApXoPEvVsgM9rVs2DSQuUQ= X-Gm-Message-State: AOJu0YwZgxckxN5QsieC1Zxf1eb6/vSMyIJmYoTwnLw5EJaiu+AGj/1U RFvWQhDauX27BEzAp/HtBDW+AZQ5jVjGURx7TtulXYVqqydqG1nIF4ya57NxYJ5ry/dd6FvAvTE 404tTX07EB1DMhm67P26GW18Gn8ngU26Kt2Ju X-Google-Smtp-Source: AGHT+IFlx6K0C7eAaVG9r79uk9O72QTsGDq116Xbm7At/lMuW1Q3lPH4Yzyz4R8A9idhwomGGhtKRhkeWk8uYhCPvbs= X-Received: by 2002:a2e:9b82:0:b0:2d4:6a34:97bf with SMTP id z2-20020a2e9b82000000b002d46a3497bfmr2476403lji.49.1712097083723; Tue, 02 Apr 2024 15:31:23 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Snehasish Kumar Date: Tue, 2 Apr 2024 15:30:46 -0700 Message-ID: Subject: Re: AutoFDO tools for GCC To: Jason Merrill Cc: Andi Kleen , Richard Biener , Eugene Rozenfeld , "gcc@gcc.gnu.org" , Jan Hubicka Content-Type: multipart/alternative; boundary="0000000000007ff2a9061524aaa9" X-Spam-Status: No, score=-16.4 required=5.0 tests=BAYES_00,DKIMWL_WL_MED,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,ENV_AND_HDR_SPF_MATCH,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP,USER_IN_DEF_DKIM_WL,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: --0000000000007ff2a9061524aaa9 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thanks for initiating this discussion Eugene. For a little bit more context on the motivation -- Meta has developed a new type of AutoFDO which is committed upstream in LLVM and we want to unify our tooling with this approach. > I do wonder how much common code there is > between the LLVM and the GCC tooling though and whether it makes sense > to keep it common (and working with both frontends)? The key components are the perf.data reader, profile construction data structures and the profile writer. The reader parses perf.data as protobuf [1] and suffers from a few drawbacks (as Andi pointed out). The intermediate data structure which represents the profile is shared (~1500LOC in [2]). Finally, LLVM and GCC have their own bespoke profile writers [3]. So given the drawbacks of reader and limited sharing I think it would be best to fork these tools into the GCC repo. Having perf generate the profiles is an interesting idea and in addition to addressing the issues Andi raised, would also simplify replicated symbolization logic. In fact, the new implementation in LLVM parses perf script output [4] to generate AutoFDO profiles. Finally, if AutoFDO is adopted by the kernel, a dependence on another repository is undesirable. > I think what makes sense to have from the code based are > profile_diff/merge etc. which are needed for scalable collection. > Or perhaps it would be best if gcov just gained those functionalities. Yes, this should be straight-forward. >> In tree would need convincing Google to assign the copyright. > > Would it? Looks like it's under a free license (apache 2), not > everything in the tree is copyright FSF or GPL3. I can ask around more on my end if I get clarification on this. Thanks, Snehasish [1] https://github.com/google/perf_data_converter [2] https://github.com/google/autofdo/blob/master/symbol_map.cc [3] https://github.com/google/autofdo/blob/master/profile_writer.cc [4] https://github.com/llvm/llvm-project/blob/main/llvm/tools/llvm-profgen/Perf= Reader.cpp On Wed, Mar 27, 2024 at 1:49=E2=80=AFPM Jason Merrill wr= ote: > On Tue, Mar 26, 2024 at 6:41=E2=80=AFPM Andi Kleen via Gcc > wrote: > > > > On Tue, Mar 26, 2024 at 08:45:22AM +0100, Richard Biener wrote: > > > On Mon, Mar 25, 2024 at 9:54=E2=80=AFPM Eugene Rozenfeld via Gcc > > > wrote: > > > > > > > > Hello, > > > > > > > > I've been the AutoFDO maintainer for the last 1.5 years. I've > resurrected autoprofiledbootstrap build and made a number of other > fixes/improvements (e.g., discriminator support). > > > > > > > > The tools for AutoFDO (create_gcov, etc.) currently live in > https://github.com/google/AutoFDO repo and GCC AutoFDO documentation > points users to that repo. That repo also has tools for LLVM AutoFDO. > > > > https://github.com/google/AutoFDO has several submodules: > https://github.com/google/autofdo/blob/master/.gitmodules > > > > > > > > I got a message from Snehasish (cc'd) that google intends to migrate > the tools for LLVM to the LLVM repo and wants to archive > https://github.com/google/AutoFDO. That will be a problem for AutoFDO in > GCC. The idea to find a different home for GCC AutoFDO tools was discussed > before on this alias but this becomes more urgent now. One idea was to > build these tools from GCC repo and another was to produce gcov from perf > tool directly. Andi (cc'd) had some early unfinished prototype for latte= r. > > > > > > > > Please let me know if you have thoughts on how we should proceed. > > > > > > I think it makes sense for GCC specific parts to live in the GCC > > > repository alongside gcov tools. I do wonder how much common code > > > there is > > > between the LLVM and the GCC tooling though and whether it makes sense > > > to keep it common (and working with both frontends)? The > > > pragmatic solution would have been to fork the repo on github to a > > > place not within the google group ... > > > > In tree would need convincing Google to assign the copyright. > > Would it? Looks like it's under a free license (apache 2), not > everything in the tree is copyright FSF or GPL3. > > Jason > > --0000000000007ff2a9061524aaa9--