From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ciao.gmane.io (ciao.gmane.io [116.202.254.214]) by sourceware.org (Postfix) with ESMTPS id 77CB53858D33 for ; Thu, 19 Oct 2023 12:47:58 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 77CB53858D33 Authentication-Results: sourceware.org; dmarc=fail (p=quarantine dis=none) header.from=westcontrol.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=m.gmane-mx.org ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 77CB53858D33 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=116.202.254.214 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1697719688; cv=none; b=cdM/uN2hqSFIr+FZhlymTpEQPaJJl/nLHVOwCNOw3te9FpUiSRjuRybgwEoCqIL0qmot1f0Q3QUWj+EJQgyDrw4Zq7wJtRneRYZGhpcvdRk3IcYsOGw/6TLrZoNFpVeKptJg/KrbtGPj46ZC7Pcw8b6grP3jNJez5iwTtuZxo+E= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1697719688; c=relaxed/simple; bh=zu0aI1mbgbhe4pccWk75tZ7BdlwtdKytsfaRa3g24e8=; h=To:From:Subject:Date:Message-ID:Mime-Version; b=kImjp3HZgiJ8BaAKuAygSaGO8rahZt+BJWw6qpkVNqRLaFde7FLTO82Lf6DTiorZG2prVkZ1AW8vwlKBLC3WxuUlJVFoKp/2a1FsOhIT4m3kdoWQK6J9hTjkE4Np0x9iddx9USbC61UbiGkXfhdHDV0755nIh0XELaALMDCONF0= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from list by ciao.gmane.io with local (Exim 4.92) (envelope-from ) id 1qtSRR-0006sM-1D for gcc-help@gcc.gnu.org; Thu, 19 Oct 2023 14:47:57 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: gcc-help@gcc.gnu.org From: David Brown Subject: Re: Compilation of lengthy C++ Files Date: Thu, 19 Oct 2023 14:47:49 +0200 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 Content-Language: en-GB In-Reply-To: X-Spam-Status: No, score=-3.5 required=5.0 tests=BAYES_00,HEADER_FROM_DIFFERENT_DOMAINS,KAM_DMARC_STATUS,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Message-ID: <20231019124749.y3Pdf1VLuJZJZvw9AF_uUDJItA7z2epB4HgerRpgHxs@z> On 18/10/2023 18:04, Kai Song via Gcc-help wrote: > Dear GCC Developers, > > I am unsuccessfully using g++ 12.0.4 to compile lengthy c++ codes. Those > codes are automatically generated from my own code-generator tools that > depend on parameters p. > Typical applications are: > - Taylor series of order p inserted into consistency conditions of > numerical schemes, to determine optimal method parameters (think of, e.g., > Runge-Kutta methods) > - recursive automatic code transformation (think of adjoints of adjoints of > adjoints...) of recursion level p > - Hilbert curves or other space-filling curves to generate code that > simulates cache utilization in a Monte-Carlo context > > I verify that for small p the codes compile and execute to the expected > result. However, there is always a threshold for p so that the generated > cpp file is so long that the compiler will just terminate after ~10min > without monitor output but return the value +1. > My cpp files range from 600k LOC up to 1Bio LOC. Often, the file comprises > of one single c++ template class member function definition that relies on > a few thousand lines of template-classes. > > I would like to know: > 1) Am I doing something wrong in that GCC should be able to compile lengthy > codes? > 2) Is it known that GCC is unable to compile lengthy codes? > 3) Is it acknowledged that a compiler's ability to compile large files is > relevant? > 4) Are possible roots known for this inability and are these deliberate? > I am curious to know why you are generating code like this. I can see how some code generators for mathematical code could easily produce large amounts of code, but it is rarely ideal for real-world uses. Such flattened code can reduce overheads and improve optimisation opportunities (like inlining, constant folding, function cloning, etc.) for small cases, but then they get impractical for compiling while the costs for cache misses outweigh the overhead cost for the loops or recursion needed for general solutions. Any compiler is going to be targeted and tuned towards "normal" or "typical" code. That means primarily hand-written code, or smaller generated code. I know that some systems generate very large functions or large files, but those are primarily C code, and the code is often very simple and "flat". (Applications here include compilers that use C as a intermediary target language, and simulators of various kinds.) It typically makes sense to disable certain optimisation passes here, and a number of passes scale badly (quadratic or perhaps worse) with function size. However, if you are generating huge templates in C++, you are going a big step beyond that - templates are, in a sense, code generators themselves that run at compile time as an interpreted meta-language. I don't expect that there has been a deliberate decision to limit GCC's handling of larger files, but I can't imagine that huge templates are a major focus for the compiler development. And I would expect enormous memory use and compile times even when it does work.