* [PATCH] Fix compiling large files
[not found] ` <56E88179.6010805@redhat.com>
@ 2016-03-15 22:31 ` Richard Henderson
2016-03-15 23:02 ` Jakub Jelinek
` (3 more replies)
0 siblings, 4 replies; 6+ messages in thread
From: Richard Henderson @ 2016-03-15 22:31 UTC (permalink / raw)
To: DJ Delorie; +Cc: gcc-patches
On 03/10/2016 08:20 PM, DJ Delorie wrote:
> I'm moving on to Plan C but I put a copy of the file on
> .../dj/foo.c.gz (195Mb) if anyone wants to find out
> why there's a 16Gb limit compiling it...
With just the following, we successfully compile your file.
It takes about 25 minutes and memory use tops out around 40GB.
Which still seems insane for a 1.6GB input file consisting
primarily of data for a static array, but that's a
different problem.
At this point we usually have a PR to go with all stage4
changes. But a meaningful PR is difficult to create, since
the attachment would be too large. Perhaps a generator could
be created, but since it wouldn't go in the testsuite it seems
like a waste of time.
Thoughts?
r~
* line-map.c (new_linemap): Make alloc_size a size_t.
diff --git a/libcpp/line-map.c b/libcpp/line-map.c
index 1fb634a..80d4e6b 100644
--- a/libcpp/line-map.c
+++ b/libcpp/line-map.c
@@ -376,7 +376,7 @@ new_linemap (struct line_maps *set,
if (LINEMAPS_USED (set, macro_map_p) == LINEMAPS_ALLOCATED (set, macro_map_p))
{
/* We ran out of allocated line maps. Let's allocate more. */
- unsigned alloc_size;
+ size_t alloc_size;
/* Cast away extern "C" from the type of xrealloc. */
line_map_realloc reallocator = (set->reallocator
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] Fix compiling large files
2016-03-15 22:31 ` [PATCH] Fix compiling large files Richard Henderson
@ 2016-03-15 23:02 ` Jakub Jelinek
2016-03-16 0:07 ` DJ Delorie
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Jakub Jelinek @ 2016-03-15 23:02 UTC (permalink / raw)
To: Richard Henderson; +Cc: DJ Delorie, gcc-patches
On Tue, Mar 15, 2016 at 03:31:44PM -0700, Richard Henderson wrote:
> On 03/10/2016 08:20 PM, DJ Delorie wrote:
> >I'm moving on to Plan C but I put a copy of the file on
> >.../dj/foo.c.gz (195Mb) if anyone wants to find out
> >why there's a 16Gb limit compiling it...
>
> With just the following, we successfully compile your file.
>
> It takes about 25 minutes and memory use tops out around 40GB.
> Which still seems insane for a 1.6GB input file consisting
> primarily of data for a static array, but that's a
> different problem.
>
> At this point we usually have a PR to go with all stage4
> changes. But a meaningful PR is difficult to create, since
> the attachment would be too large. Perhaps a generator could
> be created, but since it wouldn't go in the testsuite it seems
> like a waste of time.
>
> Thoughts?
>
>
> r~
>
>
> * line-map.c (new_linemap): Make alloc_size a size_t.
Ok for stage4.
> diff --git a/libcpp/line-map.c b/libcpp/line-map.c
> index 1fb634a..80d4e6b 100644
> --- a/libcpp/line-map.c
> +++ b/libcpp/line-map.c
> @@ -376,7 +376,7 @@ new_linemap (struct line_maps *set,
> if (LINEMAPS_USED (set, macro_map_p) == LINEMAPS_ALLOCATED (set, macro_map_p))
> {
> /* We ran out of allocated line maps. Let's allocate more. */
> - unsigned alloc_size;
> + size_t alloc_size;
>
> /* Cast away extern "C" from the type of xrealloc. */
> line_map_realloc reallocator = (set->reallocator
Jakub
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] Fix compiling large files
2016-03-15 22:31 ` [PATCH] Fix compiling large files Richard Henderson
2016-03-15 23:02 ` Jakub Jelinek
@ 2016-03-16 0:07 ` DJ Delorie
2016-03-16 16:29 ` Jeff Law
2016-07-19 11:19 ` [BACKPORT 4.9/5] " Martin Liška
3 siblings, 0 replies; 6+ messages in thread
From: DJ Delorie @ 2016-03-16 0:07 UTC (permalink / raw)
To: Richard Henderson; +Cc: gcc-patches
> At this point we usually have a PR to go with all stage4
> changes. But a meaningful PR is difficult to create, since
> the attachment would be too large. Perhaps a generator could
> be created, but since it wouldn't go in the testsuite it seems
> like a waste of time.
>
> Thoughts?
CPP macros grow exponentially, we could do it there, if we can get it
to preserve (inject?) line breaks. But I wouldn't want to be the poor
developer on a 4Gb 32-bit system trying to run it...
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] Fix compiling large files
2016-03-15 22:31 ` [PATCH] Fix compiling large files Richard Henderson
2016-03-15 23:02 ` Jakub Jelinek
2016-03-16 0:07 ` DJ Delorie
@ 2016-03-16 16:29 ` Jeff Law
2016-07-19 11:19 ` [BACKPORT 4.9/5] " Martin Liška
3 siblings, 0 replies; 6+ messages in thread
From: Jeff Law @ 2016-03-16 16:29 UTC (permalink / raw)
To: Richard Henderson, DJ Delorie; +Cc: gcc-patches
On 03/15/2016 04:31 PM, Richard Henderson wrote:
> On 03/10/2016 08:20 PM, DJ Delorie wrote:
>> I'm moving on to Plan C but I put a copy of the file on
>> .../dj/foo.c.gz (195Mb) if anyone wants to find out
>> why there's a 16Gb limit compiling it...
>
> With just the following, we successfully compile your file.
>
> It takes about 25 minutes and memory use tops out around 40GB.
> Which still seems insane for a 1.6GB input file consisting
> primarily of data for a static array, but that's a
> different problem.
>
> At this point we usually have a PR to go with all stage4
> changes. But a meaningful PR is difficult to create, since
> the attachment would be too large. Perhaps a generator could
> be created, but since it wouldn't go in the testsuite it seems
> like a waste of time.
>
> Thoughts?
>
>
> r~
>
>
> * line-map.c (new_linemap): Make alloc_size a size_t.
OK.
jeff
^ permalink raw reply [flat|nested] 6+ messages in thread
* [BACKPORT 4.9/5] Fix compiling large files
2016-03-15 22:31 ` [PATCH] Fix compiling large files Richard Henderson
` (2 preceding siblings ...)
2016-03-16 16:29 ` Jeff Law
@ 2016-07-19 11:19 ` Martin Liška
2016-07-19 11:23 ` Richard Biener
3 siblings, 1 reply; 6+ messages in thread
From: Martin Liška @ 2016-07-19 11:19 UTC (permalink / raw)
To: Richard Henderson, DJ Delorie; +Cc: gcc-patches
Hello.
As mentioned in PR71920, I would like to backport the change to GCC 4.9 and 5.
May I install the patch after proper testing?
Thanks,
Martin
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [BACKPORT 4.9/5] Fix compiling large files
2016-07-19 11:19 ` [BACKPORT 4.9/5] " Martin Liška
@ 2016-07-19 11:23 ` Richard Biener
0 siblings, 0 replies; 6+ messages in thread
From: Richard Biener @ 2016-07-19 11:23 UTC (permalink / raw)
To: Martin Liška; +Cc: Richard Henderson, DJ Delorie, GCC Patches
On Tue, Jul 19, 2016 at 1:19 PM, Martin Liška <mliska@suse.cz> wrote:
> Hello.
>
> As mentioned in PR71920, I would like to backport the change to GCC 4.9 and 5.
>
> May I install the patch after proper testing?
Yes, it's pretty obvious.
Richard.
> Thanks,
> Martin
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2016-07-19 11:23 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <201603110420.u2B4KQ70018134@greed.delorie.com>
[not found] ` <56E88179.6010805@redhat.com>
2016-03-15 22:31 ` [PATCH] Fix compiling large files Richard Henderson
2016-03-15 23:02 ` Jakub Jelinek
2016-03-16 0:07 ` DJ Delorie
2016-03-16 16:29 ` Jeff Law
2016-07-19 11:19 ` [BACKPORT 4.9/5] " Martin Liška
2016-07-19 11:23 ` Richard Biener
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).