public inbox for gcc-help@gcc.gnu.org
 help / color / mirror / Atom feed
From: "relliott@umn.edu" <relliott@umn.edu>
To: Jonathan Wakely <jwakely.gcc@gmail.com>
Cc: gcc-help <gcc-help@gcc.gnu.org>
Subject: Re: High memory usage compiling large ‘xxd -i’ output
Date: Sat, 11 Apr 2020 16:31:05 -0500	[thread overview]
Message-ID: <3F1B5DA6-7224-4431-8070-123D18B69B3C@umn.edu> (raw)
In-Reply-To: <CAH6eHdQz1h=Pv_k43P=bS2BrtzJBfF2HkeonFs_w34jh0mxdYg@mail.gmail.com>

Hello,

Thanks for your reply.  I guess I don’t understand why this requires so much memory all at once.  Is it not possible to write to the object file as the initializer is parsed?  I feel like there is some complexity about the parsing process/requirements I’m not appreciating.  An example or explanation would probably help me a lot.

Thanks,

Ryan 

> On Apr 11, 2020, at 4:22 PM, Jonathan Wakely <jwakely.gcc@gmail.com> wrote:
> 
> On Sat, 11 Apr 2020 at 21:04, relliott--- via Gcc-help
> <gcc-help@gcc.gnu.org> wrote:
>> 
>> Hello,
>> 
>> I’m seeing gcc memory usage exceed 15GB when compiling a source file generated from the command:
>> 
>> % xxd -i file.txt
>> 
>> where ‘file.txt’ is a 170MB file.
>> 
>> Is there a way to avoid this high memory usage?  Can anyone explain why it occurs?
> 
> Because the compiler is parsing an array initializer with millions of elements.

  reply	other threads:[~2020-04-11 21:31 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-04-11 20:03 relliott
2020-04-11 21:22 ` Jonathan Wakely
2020-04-11 21:31   ` relliott [this message]
2020-04-12  2:07     ` Xi Ruoyao
2020-04-12 22:01       ` relliott

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=3F1B5DA6-7224-4431-8070-123D18B69B3C@umn.edu \
    --to=relliott@umn.edu \
    --cc=gcc-help@gcc.gnu.org \
    --cc=jwakely.gcc@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).