public inbox for gcc-patches@gcc.gnu.org
 help / color / mirror / Atom feed
* [PATCH] Generate gimple-match.c and generic-match.c earlier
@ 2021-05-28  4:42 Bernd Edlinger
  2021-05-28  4:51 ` Bernd Edlinger
  2021-05-28  7:26 ` Andrew Pinski
  0 siblings, 2 replies; 15+ messages in thread
From: Bernd Edlinger @ 2021-05-28  4:42 UTC (permalink / raw)
  To: gcc-patches, Richard Biener

[-- Attachment #1: Type: text/plain, Size: 599 bytes --]

Hi,

I was wondering, why gimple-match.c and generic-match.c
are not built early but always last, which slows down parallel
makes significantly.

The reason seems to be that generated_files does not
mention gimple-match.c and generic-match.c.

This comment in Makefile.in says it all:

$(ALL_HOST_OBJS) : | $(generated_files)

So this patch adds gimple-match.c generic-match.c to generated_files.


Tested on x86_64-pc-linux-gnu.
Is it OK for trunk?


Thanks
Bernd.


2021-05-28  Bernd Edlinger  <bernd.edlinger@hotmail.de>

	* Makefile.in (generated_files): Add gimple-match.c and
	generic-match.c

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Generate-gimple-match.c-and-generic-match.c-earlier.patch --]
[-- Type: text/x-patch; name="0001-Generate-gimple-match.c-and-generic-match.c-earlier.patch", Size: 1398 bytes --]

From 5c14c3f5852ddcc1d3b76e7c53260b0187604cd7 Mon Sep 17 00:00:00 2001
From: Bernd Edlinger <bernd.edlinger@hotmail.de>
Date: Fri, 28 May 2021 06:27:27 +0200
Subject: [PATCH] Generate gimple-match.c and generic-match.c earlier

I was wondering, why gimple-match.c and generic-match.c
are not built early but always last, which slows down parallel
makes significantly.

The reason seems to be that generated_files does not
mention gimple-match.c and generic-match.c.

This comment in Makefile.in says it all:

$(ALL_HOST_OBJS) : | $(generated_files)

So this patch adds gimple-match.c generic-match.c to generated_files.

2021-05-28  Bernd Edlinger  <bernd.edlinger@hotmail.de>

	* Makefile.in (generated_files): Add gimple-match.c and
	generic-match.c
---
 gcc/Makefile.in | 1 +
 1 file changed, 1 insertion(+)

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index da2ef24..4cb2966 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -2753,6 +2753,7 @@ generated_files = config.h tm.h $(TM_P_H) $(TM_D_H) $(TM_H) multilib.h \
        $(ALL_GTFILES_H) gtype-desc.c gtype-desc.h version.h \
        options.h target-hooks-def.h insn-opinit.h \
        common/common-target-hooks-def.h pass-instances.def \
+       gimple-match.c generic-match.c \
        c-family/c-target-hooks-def.h d/d-target-hooks-def.h \
        case-cfn-macros.h \
        cfn-operators.pd omp-device-properties.h
-- 
1.9.1


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH] Generate gimple-match.c and generic-match.c earlier
  2021-05-28  4:42 [PATCH] Generate gimple-match.c and generic-match.c earlier Bernd Edlinger
@ 2021-05-28  4:51 ` Bernd Edlinger
  2021-05-28  6:41   ` Richard Biener
  2021-05-28  7:26 ` Andrew Pinski
  1 sibling, 1 reply; 15+ messages in thread
From: Bernd Edlinger @ 2021-05-28  4:51 UTC (permalink / raw)
  To: gcc-patches, Richard Biener

[-- Attachment #1: Type: text/plain, Size: 997 bytes --]



On 5/28/21 6:42 AM, Bernd Edlinger wrote:
> Hi,
> 
> I was wondering, why gimple-match.c and generic-match.c
> are not built early but always last, which slows down parallel
> makes significantly.
> 
> The reason seems to be that generated_files does not
> mention gimple-match.c and generic-match.c.
> 
> This comment in Makefile.in says it all:
> 

Oh, dear, git commit did eliminate the comments
starting with "#"
the mentined comment is

# Dependency information.

# In order for parallel make to really start compiling the expensive
# objects from $(OBJS) as early as possible, build all their
# prerequisites strictly before all objects.

> $(ALL_HOST_OBJS) : | $(generated_files)
> 
> So this patch adds gimple-match.c generic-match.c to generated_files.
> 
> 
> Tested on x86_64-pc-linux-gnu.
> Is it OK for trunk?
> 
> 
> Thanks
> Bernd.
> 
> 
> 2021-05-28  Bernd Edlinger  <bernd.edlinger@hotmail.de>
> 
> 	* Makefile.in (generated_files): Add gimple-match.c and
> 	generic-match.c
> 

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Generate-gimple-match.c-and-generic-match.c-earlier.patch --]
[-- Type: text/x-patch; name="0001-Generate-gimple-match.c-and-generic-match.c-earlier.patch", Size: 1529 bytes --]

From 99eab77ebfaa02ee22263d89eb3ca812cf65263b Mon Sep 17 00:00:00 2001
From: Bernd Edlinger <bernd.edlinger@hotmail.de>
Date: Fri, 28 May 2021 06:27:27 +0200
Subject: [PATCH] Generate gimple-match.c and generic-match.c earlier

I was wondering, why gimple-match.c and generic-match.c
are not built early but always last, which slows down parallel
makes significantly.

The reason seems to be that generated_files does not
mention gimple-match.c and generic-match.c.

This comment in Makefile.in says it all:

"In order for parallel make to really start compiling the expensive
objects from $(OBJS) as early as possible, build all their
prerequisites strictly before all objects."

So this patch adds gimple-match.c generic-match.c to generated_files.

2021-05-28  Bernd Edlinger  <bernd.edlinger@hotmail.de>

	* Makefile.in (generated_files): Add gimple-match.c and
	generic-match.c
---
 gcc/Makefile.in | 1 +
 1 file changed, 1 insertion(+)

diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index da2ef24..4cb2966 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -2753,6 +2753,7 @@ generated_files = config.h tm.h $(TM_P_H) $(TM_D_H) $(TM_H) multilib.h \
        $(ALL_GTFILES_H) gtype-desc.c gtype-desc.h version.h \
        options.h target-hooks-def.h insn-opinit.h \
        common/common-target-hooks-def.h pass-instances.def \
+       gimple-match.c generic-match.c \
        c-family/c-target-hooks-def.h d/d-target-hooks-def.h \
        case-cfn-macros.h \
        cfn-operators.pd omp-device-properties.h
-- 
1.9.1


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH] Generate gimple-match.c and generic-match.c earlier
  2021-05-28  4:51 ` Bernd Edlinger
@ 2021-05-28  6:41   ` Richard Biener
  2021-05-28  6:48     ` Bernd Edlinger
  0 siblings, 1 reply; 15+ messages in thread
From: Richard Biener @ 2021-05-28  6:41 UTC (permalink / raw)
  To: Bernd Edlinger; +Cc: gcc-patches

On Fri, 28 May 2021, Bernd Edlinger wrote:

> 
> 
> On 5/28/21 6:42 AM, Bernd Edlinger wrote:
> > Hi,
> > 
> > I was wondering, why gimple-match.c and generic-match.c
> > are not built early but always last, which slows down parallel
> > makes significantly.
> > 
> > The reason seems to be that generated_files does not
> > mention gimple-match.c and generic-match.c.
> > 
> > This comment in Makefile.in says it all:
> > 
> 
> Oh, dear, git commit did eliminate the comments
> starting with "#"
> the mentined comment is
> 
> # Dependency information.
> 
> # In order for parallel make to really start compiling the expensive
> # objects from $(OBJS) as early as possible, build all their
> # prerequisites strictly before all objects.
> 
> > $(ALL_HOST_OBJS) : | $(generated_files)
> > 
> > So this patch adds gimple-match.c generic-match.c to generated_files.
> > 
> > 
> > Tested on x86_64-pc-linux-gnu.
> > Is it OK for trunk?

OK.  Does it really help though?

Thanks,
Richard.

> > 
> > 
> > Thanks
> > Bernd.
> > 
> > 
> > 2021-05-28  Bernd Edlinger  <bernd.edlinger@hotmail.de>
> > 
> > 	* Makefile.in (generated_files): Add gimple-match.c and
> > 	generic-match.c
> > 
> 

-- 
Richard Biener <rguenther@suse.de>
SUSE Software Solutions Germany GmbH, Maxfeldstrasse 5, 90409 Nuernberg,
Germany; GF: Felix Imendörffer; HRB 36809 (AG Nuernberg)

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH] Generate gimple-match.c and generic-match.c earlier
  2021-05-28  6:41   ` Richard Biener
@ 2021-05-28  6:48     ` Bernd Edlinger
  0 siblings, 0 replies; 15+ messages in thread
From: Bernd Edlinger @ 2021-05-28  6:48 UTC (permalink / raw)
  To: Richard Biener; +Cc: gcc-patches

On 5/28/21 8:41 AM, Richard Biener wrote:
> On Fri, 28 May 2021, Bernd Edlinger wrote:
> 
>>
>>
>> On 5/28/21 6:42 AM, Bernd Edlinger wrote:
>>> Hi,
>>>
>>> I was wondering, why gimple-match.c and generic-match.c
>>> are not built early but always last, which slows down parallel
>>> makes significantly.
>>>
>>> The reason seems to be that generated_files does not
>>> mention gimple-match.c and generic-match.c.
>>>
>>> This comment in Makefile.in says it all:
>>>
>>
>> Oh, dear, git commit did eliminate the comments
>> starting with "#"
>> the mentined comment is
>>
>> # Dependency information.
>>
>> # In order for parallel make to really start compiling the expensive
>> # objects from $(OBJS) as early as possible, build all their
>> # prerequisites strictly before all objects.
>>
>>> $(ALL_HOST_OBJS) : | $(generated_files)
>>>
>>> So this patch adds gimple-match.c generic-match.c to generated_files.
>>>
>>>
>>> Tested on x86_64-pc-linux-gnu.
>>> Is it OK for trunk?
> 
> OK.  Does it really help though?
> 

Yes, I guess so, at least a little bit.

Prior to this patch the whole build stage was completed for everything
then those two big files got generated,
and then there are only two large files compiled in parallel for
several minutes at least.

So a make -j8 utilizes only 25 % cpu power
and make -j16 only 12.5 % utilization.
That can certainly be a bit annoying.


Bernd.


> Thanks,
> Richard.
> 
>>>
>>>
>>> Thanks
>>> Bernd.
>>>
>>>
>>> 2021-05-28  Bernd Edlinger  <bernd.edlinger@hotmail.de>
>>>
>>> 	* Makefile.in (generated_files): Add gimple-match.c and
>>> 	generic-match.c
>>>
>>
> 

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH] Generate gimple-match.c and generic-match.c earlier
  2021-05-28  4:42 [PATCH] Generate gimple-match.c and generic-match.c earlier Bernd Edlinger
  2021-05-28  4:51 ` Bernd Edlinger
@ 2021-05-28  7:26 ` Andrew Pinski
  2021-05-28  7:33   ` Bernd Edlinger
  1 sibling, 1 reply; 15+ messages in thread
From: Andrew Pinski @ 2021-05-28  7:26 UTC (permalink / raw)
  To: Bernd Edlinger; +Cc: gcc-patches, Richard Biener

On Thu, May 27, 2021 at 10:10 PM Bernd Edlinger
<bernd.edlinger@hotmail.de> wrote:
>
> Hi,
>
> I was wondering, why gimple-match.c and generic-match.c
> are not built early but always last, which slows down parallel
> makes significantly.
>
> The reason seems to be that generated_files does not
> mention gimple-match.c and generic-match.c.
>
> This comment in Makefile.in says it all:
>
> $(ALL_HOST_OBJS) : | $(generated_files)
>
> So this patch adds gimple-match.c generic-match.c to generated_files.
>
>
> Tested on x86_64-pc-linux-gnu.
> Is it OK for trunk?

This should help for what I was complaining about in
https://gcc.gnu.org/pipermail/gcc/2021-May/235963.html . I build with
-j24 and it was stalling on compiling gimple-match.c for me.
Looks like insn-attrtab.c is missed too; I saw genattrtab was running last too.

Thanks,
Andrew

>
>
> Thanks
> Bernd.
>
>
> 2021-05-28  Bernd Edlinger  <bernd.edlinger@hotmail.de>
>
>         * Makefile.in (generated_files): Add gimple-match.c and
>         generic-match.c

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH] Generate gimple-match.c and generic-match.c earlier
  2021-05-28  7:26 ` Andrew Pinski
@ 2021-05-28  7:33   ` Bernd Edlinger
  2021-05-28  8:48     ` Jakub Jelinek
  2021-05-28 15:33     ` Michael Matz
  0 siblings, 2 replies; 15+ messages in thread
From: Bernd Edlinger @ 2021-05-28  7:33 UTC (permalink / raw)
  To: Andrew Pinski; +Cc: gcc-patches, Richard Biener

On 5/28/21 9:26 AM, Andrew Pinski wrote:
> On Thu, May 27, 2021 at 10:10 PM Bernd Edlinger
> <bernd.edlinger@hotmail.de> wrote:
>>
>> Hi,
>>
>> I was wondering, why gimple-match.c and generic-match.c
>> are not built early but always last, which slows down parallel
>> makes significantly.
>>
>> The reason seems to be that generated_files does not
>> mention gimple-match.c and generic-match.c.
>>
>> This comment in Makefile.in says it all:
>>
>> $(ALL_HOST_OBJS) : | $(generated_files)
>>
>> So this patch adds gimple-match.c generic-match.c to generated_files.
>>
>>
>> Tested on x86_64-pc-linux-gnu.
>> Is it OK for trunk?
> 
> This should help for what I was complaining about in
> https://gcc.gnu.org/pipermail/gcc/2021-May/235963.html . I build with
> -j24 and it was stalling on compiling gimple-match.c for me.
> Looks like insn-attrtab.c is missed too; I saw genattrtab was running last too.
> 

Yeah, probably insn-automata.c as well, sometimes it is picked up early sometimes not.
maybe $(simple_generated_c) should be added to generated_files,
but insn-attrtab.c is yet another exception.


Bernd.

> Thanks,
> Andrew
> 
>>
>>
>> Thanks
>> Bernd.
>>
>>
>> 2021-05-28  Bernd Edlinger  <bernd.edlinger@hotmail.de>
>>
>>         * Makefile.in (generated_files): Add gimple-match.c and
>>         generic-match.c

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH] Generate gimple-match.c and generic-match.c earlier
  2021-05-28  7:33   ` Bernd Edlinger
@ 2021-05-28  8:48     ` Jakub Jelinek
  2021-05-28 15:33     ` Michael Matz
  1 sibling, 0 replies; 15+ messages in thread
From: Jakub Jelinek @ 2021-05-28  8:48 UTC (permalink / raw)
  To: Bernd Edlinger; +Cc: Andrew Pinski, Richard Biener, gcc-patches

On Fri, May 28, 2021 at 09:33:14AM +0200, Bernd Edlinger wrote:
> > This should help for what I was complaining about in
> > https://gcc.gnu.org/pipermail/gcc/2021-May/235963.html . I build with
> > -j24 and it was stalling on compiling gimple-match.c for me.
> > Looks like insn-attrtab.c is missed too; I saw genattrtab was running last too.
> > 
> 
> Yeah, probably insn-automata.c as well, sometimes it is picked up early sometimes not.
> maybe $(simple_generated_c) should be added to generated_files,
> but insn-attrtab.c is yet another exception.

The problem with insn-automata.c is that on some targets it is very time
consuming to generate that file (I think e.g. s390x) and by putting it into
generated_files it could actually hurt the build speed because everything
will be waiting on insn-automata.c generation for 1-2 minutes or how long
before anything else can be built.

	Jakub


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH] Generate gimple-match.c and generic-match.c earlier
  2021-05-28  7:33   ` Bernd Edlinger
  2021-05-28  8:48     ` Jakub Jelinek
@ 2021-05-28 15:33     ` Michael Matz
  2021-07-14 12:47       ` Tamar Christina
  1 sibling, 1 reply; 15+ messages in thread
From: Michael Matz @ 2021-05-28 15:33 UTC (permalink / raw)
  To: Bernd Edlinger; +Cc: Andrew Pinski, Richard Biener, gcc-patches

Hello,

On Fri, 28 May 2021, Bernd Edlinger wrote:

> >> I was wondering, why gimple-match.c and generic-match.c
> >> are not built early but always last, which slows down parallel
> >> makes significantly.
> >>
> >> The reason seems to be that generated_files does not
> >> mention gimple-match.c and generic-match.c.
> >>
> >> This comment in Makefile.in says it all:
> >>
> >> $(ALL_HOST_OBJS) : | $(generated_files)
> >>
> >> So this patch adds gimple-match.c generic-match.c to generated_files.
> >>
> >>
> >> Tested on x86_64-pc-linux-gnu.
> >> Is it OK for trunk?
> > 
> > This should help for what I was complaining about in
> > https://gcc.gnu.org/pipermail/gcc/2021-May/235963.html . I build with
> > -j24 and it was stalling on compiling gimple-match.c for me.
> > Looks like insn-attrtab.c is missed too; I saw genattrtab was running last too.
> > 
> 
> Yeah, probably insn-automata.c as well, sometimes it is picked up early 
> sometimes not. maybe $(simple_generated_c) should be added to 
> generated_files, but insn-attrtab.c is yet another exception.

You can't put files in there that are sometimes slow to generate (which 
insn-{attrtab,automata}.c are on some targets), as _everything_ then waits 
for them to be created first.

Ideally there would be a way for gnumake to mark some targets as 
"ugh-slow" and back-propagate this to all dependencies so that those are 
put in front of the work queue in a parallel make.  Alas, something like 
that never came into existence :-/  (When order-only deps were introduced 
I got excited, but then came to realize that that wasn't what was really 
needed for this case, a "weak" version of it would be required at least, 
or better yet a specific facility to impose a cost with a target)


Ciao,
Michael.

> 
> 
> Bernd.
> 
> > Thanks,
> > Andrew
> > 
> >>
> >>
> >> Thanks
> >> Bernd.
> >>
> >>
> >> 2021-05-28  Bernd Edlinger  <bernd.edlinger@hotmail.de>
> >>
> >>         * Makefile.in (generated_files): Add gimple-match.c and
> >>         generic-match.c
> 

^ permalink raw reply	[flat|nested] 15+ messages in thread

* RE: [PATCH] Generate gimple-match.c and generic-match.c earlier
  2021-05-28 15:33     ` Michael Matz
@ 2021-07-14 12:47       ` Tamar Christina
  2021-07-14 13:12         ` Richard Biener
  2021-07-14 15:56         ` Bernd Edlinger
  0 siblings, 2 replies; 15+ messages in thread
From: Tamar Christina @ 2021-07-14 12:47 UTC (permalink / raw)
  To: Michael Matz, Bernd Edlinger; +Cc: gcc-patches, Richard Biener

Hi,

Ever since this commit 

commit c9114f2804b91690e030383de15a24e0b738e856
Author: Bernd Edlinger <bernd.edlinger@hotmail.de>
Date:   Fri May 28 06:27:27 2021 +0200

Various tools have been having trouble with cross compilation resulting in

make[2]: *** No rule to make target '../build-x86_64-build_pc-linux-gnu/libcpp/libcpp.a', needed by 'build/genmatch'.

(took a while to track down).  I don't understand this part of the build system well enough to know how to fix this.
It looks like `libcpp.a` has special handling for cross compilers which now seems to be broken.

I can't reproduce it with our normal cross compiler scripts. Which handles the stages on its own, but e.g.
https://github.com/crosstool-ng/crosstool-ng does reproduce the failure.

Any ideas what's going on?

Kind Regards,
Tamar

> -----Original Message-----
> From: Gcc-patches <gcc-patches-bounces@gcc.gnu.org> On Behalf Of
> Michael Matz
> Sent: Friday, May 28, 2021 4:33 PM
> To: Bernd Edlinger <bernd.edlinger@hotmail.de>
> Cc: gcc-patches@gcc.gnu.org; Richard Biener <rguenther@suse.de>
> Subject: Re: [PATCH] Generate gimple-match.c and generic-match.c earlier
> 
> Hello,
> 
> On Fri, 28 May 2021, Bernd Edlinger wrote:
> 
> > >> I was wondering, why gimple-match.c and generic-match.c are not
> > >> built early but always last, which slows down parallel makes
> > >> significantly.
> > >>
> > >> The reason seems to be that generated_files does not mention
> > >> gimple-match.c and generic-match.c.
> > >>
> > >> This comment in Makefile.in says it all:
> > >>
> > >> $(ALL_HOST_OBJS) : | $(generated_files)
> > >>
> > >> So this patch adds gimple-match.c generic-match.c to generated_files.
> > >>
> > >>
> > >> Tested on x86_64-pc-linux-gnu.
> > >> Is it OK for trunk?
> > >
> > > This should help for what I was complaining about in
> > > https://gcc.gnu.org/pipermail/gcc/2021-May/235963.html . I build
> > > with
> > > -j24 and it was stalling on compiling gimple-match.c for me.
> > > Looks like insn-attrtab.c is missed too; I saw genattrtab was running last
> too.
> > >
> >
> > Yeah, probably insn-automata.c as well, sometimes it is picked up
> > early sometimes not. maybe $(simple_generated_c) should be added to
> > generated_files, but insn-attrtab.c is yet another exception.
> 
> You can't put files in there that are sometimes slow to generate (which insn-
> {attrtab,automata}.c are on some targets), as _everything_ then waits for
> them to be created first.
> 
> Ideally there would be a way for gnumake to mark some targets as "ugh-
> slow" and back-propagate this to all dependencies so that those are put in
> front of the work queue in a parallel make.  Alas, something like that never
> came into existence :-/  (When order-only deps were introduced I got
> excited, but then came to realize that that wasn't what was really needed for
> this case, a "weak" version of it would be required at least, or better yet a
> specific facility to impose a cost with a target)
> 
> 
> Ciao,
> Michael.
> 
> >
> >
> > Bernd.
> >
> > > Thanks,
> > > Andrew
> > >
> > >>
> > >>
> > >> Thanks
> > >> Bernd.
> > >>
> > >>
> > >> 2021-05-28  Bernd Edlinger  <bernd.edlinger@hotmail.de>
> > >>
> > >>         * Makefile.in (generated_files): Add gimple-match.c and
> > >>         generic-match.c
> >

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH] Generate gimple-match.c and generic-match.c earlier
  2021-07-14 12:47       ` Tamar Christina
@ 2021-07-14 13:12         ` Richard Biener
  2021-07-14 13:15           ` Richard Biener
  2021-07-14 15:56         ` Bernd Edlinger
  1 sibling, 1 reply; 15+ messages in thread
From: Richard Biener @ 2021-07-14 13:12 UTC (permalink / raw)
  To: Tamar Christina; +Cc: Michael Matz, Bernd Edlinger, Richard Biener, gcc-patches

On Wed, Jul 14, 2021 at 2:48 PM Tamar Christina via Gcc-patches
<gcc-patches@gcc.gnu.org> wrote:
>
> Hi,
>
> Ever since this commit
>
> commit c9114f2804b91690e030383de15a24e0b738e856
> Author: Bernd Edlinger <bernd.edlinger@hotmail.de>
> Date:   Fri May 28 06:27:27 2021 +0200
>
> Various tools have been having trouble with cross compilation resulting in
>
> make[2]: *** No rule to make target '../build-x86_64-build_pc-linux-gnu/libcpp/libcpp.a', needed by 'build/genmatch'.
>
> (took a while to track down).  I don't understand this part of the build system well enough to know how to fix this.
> It looks like `libcpp.a` has special handling for cross compilers which now seems to be broken.
>
> I can't reproduce it with our normal cross compiler scripts. Which handles the stages on its own, but e.g.
> https://github.com/crosstool-ng/crosstool-ng does reproduce the failure.
>
> Any ideas what's going on?

There should be a dependence of all-gcc to all-build-libcpp, Makefile.def has

dependencies = { module=all-gcc; on=all-build-libcpp; };

so how come build-libcpp is not built when gcc/ starts?

> Kind Regards,
> Tamar
>
> > -----Original Message-----
> > From: Gcc-patches <gcc-patches-bounces@gcc.gnu.org> On Behalf Of
> > Michael Matz
> > Sent: Friday, May 28, 2021 4:33 PM
> > To: Bernd Edlinger <bernd.edlinger@hotmail.de>
> > Cc: gcc-patches@gcc.gnu.org; Richard Biener <rguenther@suse.de>
> > Subject: Re: [PATCH] Generate gimple-match.c and generic-match.c earlier
> >
> > Hello,
> >
> > On Fri, 28 May 2021, Bernd Edlinger wrote:
> >
> > > >> I was wondering, why gimple-match.c and generic-match.c are not
> > > >> built early but always last, which slows down parallel makes
> > > >> significantly.
> > > >>
> > > >> The reason seems to be that generated_files does not mention
> > > >> gimple-match.c and generic-match.c.
> > > >>
> > > >> This comment in Makefile.in says it all:
> > > >>
> > > >> $(ALL_HOST_OBJS) : | $(generated_files)
> > > >>
> > > >> So this patch adds gimple-match.c generic-match.c to generated_files.
> > > >>
> > > >>
> > > >> Tested on x86_64-pc-linux-gnu.
> > > >> Is it OK for trunk?
> > > >
> > > > This should help for what I was complaining about in
> > > > https://gcc.gnu.org/pipermail/gcc/2021-May/235963.html . I build
> > > > with
> > > > -j24 and it was stalling on compiling gimple-match.c for me.
> > > > Looks like insn-attrtab.c is missed too; I saw genattrtab was running last
> > too.
> > > >
> > >
> > > Yeah, probably insn-automata.c as well, sometimes it is picked up
> > > early sometimes not. maybe $(simple_generated_c) should be added to
> > > generated_files, but insn-attrtab.c is yet another exception.
> >
> > You can't put files in there that are sometimes slow to generate (which insn-
> > {attrtab,automata}.c are on some targets), as _everything_ then waits for
> > them to be created first.
> >
> > Ideally there would be a way for gnumake to mark some targets as "ugh-
> > slow" and back-propagate this to all dependencies so that those are put in
> > front of the work queue in a parallel make.  Alas, something like that never
> > came into existence :-/  (When order-only deps were introduced I got
> > excited, but then came to realize that that wasn't what was really needed for
> > this case, a "weak" version of it would be required at least, or better yet a
> > specific facility to impose a cost with a target)
> >
> >
> > Ciao,
> > Michael.
> >
> > >
> > >
> > > Bernd.
> > >
> > > > Thanks,
> > > > Andrew
> > > >
> > > >>
> > > >>
> > > >> Thanks
> > > >> Bernd.
> > > >>
> > > >>
> > > >> 2021-05-28  Bernd Edlinger  <bernd.edlinger@hotmail.de>
> > > >>
> > > >>         * Makefile.in (generated_files): Add gimple-match.c and
> > > >>         generic-match.c
> > >

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH] Generate gimple-match.c and generic-match.c earlier
  2021-07-14 13:12         ` Richard Biener
@ 2021-07-14 13:15           ` Richard Biener
  2021-07-14 13:19             ` Richard Biener
  0 siblings, 1 reply; 15+ messages in thread
From: Richard Biener @ 2021-07-14 13:15 UTC (permalink / raw)
  To: Tamar Christina; +Cc: Michael Matz, Bernd Edlinger, Richard Biener, gcc-patches

On Wed, Jul 14, 2021 at 3:12 PM Richard Biener
<richard.guenther@gmail.com> wrote:
>
> On Wed, Jul 14, 2021 at 2:48 PM Tamar Christina via Gcc-patches
> <gcc-patches@gcc.gnu.org> wrote:
> >
> > Hi,
> >
> > Ever since this commit
> >
> > commit c9114f2804b91690e030383de15a24e0b738e856
> > Author: Bernd Edlinger <bernd.edlinger@hotmail.de>
> > Date:   Fri May 28 06:27:27 2021 +0200
> >
> > Various tools have been having trouble with cross compilation resulting in
> >
> > make[2]: *** No rule to make target '../build-x86_64-build_pc-linux-gnu/libcpp/libcpp.a', needed by 'build/genmatch'.
> >
> > (took a while to track down).  I don't understand this part of the build system well enough to know how to fix this.
> > It looks like `libcpp.a` has special handling for cross compilers which now seems to be broken.
> >
> > I can't reproduce it with our normal cross compiler scripts. Which handles the stages on its own, but e.g.
> > https://github.com/crosstool-ng/crosstool-ng does reproduce the failure.
> >
> > Any ideas what's going on?
>
> There should be a dependence of all-gcc to all-build-libcpp, Makefile.def has
>
> dependencies = { module=all-gcc; on=all-build-libcpp; };
>
> so how come build-libcpp is not built when gcc/ starts?

Ah, I guess (gcc/Makefile.in):

CPPLIB = ../libcpp/libcpp.a
...
# For stage1 and when cross-compiling use the build libcpp which is
# built with NLS disabled.  For stage2+ use the host library and
# its dependencies.
ifeq ($(build_objdir),$(build_libobjdir))
BUILD_CPPLIB = $(build_libobjdir)/libcpp/libcpp.a
else
BUILD_CPPLIB = $(CPPLIB) $(LIBIBERTY)

is not properly reflected in above dependences.  Not sure how to fix
that though.

> > Kind Regards,
> > Tamar
> >
> > > -----Original Message-----
> > > From: Gcc-patches <gcc-patches-bounces@gcc.gnu.org> On Behalf Of
> > > Michael Matz
> > > Sent: Friday, May 28, 2021 4:33 PM
> > > To: Bernd Edlinger <bernd.edlinger@hotmail.de>
> > > Cc: gcc-patches@gcc.gnu.org; Richard Biener <rguenther@suse.de>
> > > Subject: Re: [PATCH] Generate gimple-match.c and generic-match.c earlier
> > >
> > > Hello,
> > >
> > > On Fri, 28 May 2021, Bernd Edlinger wrote:
> > >
> > > > >> I was wondering, why gimple-match.c and generic-match.c are not
> > > > >> built early but always last, which slows down parallel makes
> > > > >> significantly.
> > > > >>
> > > > >> The reason seems to be that generated_files does not mention
> > > > >> gimple-match.c and generic-match.c.
> > > > >>
> > > > >> This comment in Makefile.in says it all:
> > > > >>
> > > > >> $(ALL_HOST_OBJS) : | $(generated_files)
> > > > >>
> > > > >> So this patch adds gimple-match.c generic-match.c to generated_files.
> > > > >>
> > > > >>
> > > > >> Tested on x86_64-pc-linux-gnu.
> > > > >> Is it OK for trunk?
> > > > >
> > > > > This should help for what I was complaining about in
> > > > > https://gcc.gnu.org/pipermail/gcc/2021-May/235963.html . I build
> > > > > with
> > > > > -j24 and it was stalling on compiling gimple-match.c for me.
> > > > > Looks like insn-attrtab.c is missed too; I saw genattrtab was running last
> > > too.
> > > > >
> > > >
> > > > Yeah, probably insn-automata.c as well, sometimes it is picked up
> > > > early sometimes not. maybe $(simple_generated_c) should be added to
> > > > generated_files, but insn-attrtab.c is yet another exception.
> > >
> > > You can't put files in there that are sometimes slow to generate (which insn-
> > > {attrtab,automata}.c are on some targets), as _everything_ then waits for
> > > them to be created first.
> > >
> > > Ideally there would be a way for gnumake to mark some targets as "ugh-
> > > slow" and back-propagate this to all dependencies so that those are put in
> > > front of the work queue in a parallel make.  Alas, something like that never
> > > came into existence :-/  (When order-only deps were introduced I got
> > > excited, but then came to realize that that wasn't what was really needed for
> > > this case, a "weak" version of it would be required at least, or better yet a
> > > specific facility to impose a cost with a target)
> > >
> > >
> > > Ciao,
> > > Michael.
> > >
> > > >
> > > >
> > > > Bernd.
> > > >
> > > > > Thanks,
> > > > > Andrew
> > > > >
> > > > >>
> > > > >>
> > > > >> Thanks
> > > > >> Bernd.
> > > > >>
> > > > >>
> > > > >> 2021-05-28  Bernd Edlinger  <bernd.edlinger@hotmail.de>
> > > > >>
> > > > >>         * Makefile.in (generated_files): Add gimple-match.c and
> > > > >>         generic-match.c
> > > >

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH] Generate gimple-match.c and generic-match.c earlier
  2021-07-14 13:15           ` Richard Biener
@ 2021-07-14 13:19             ` Richard Biener
  2021-07-14 13:25               ` Tamar Christina
  0 siblings, 1 reply; 15+ messages in thread
From: Richard Biener @ 2021-07-14 13:19 UTC (permalink / raw)
  To: Tamar Christina; +Cc: Michael Matz, Bernd Edlinger, Richard Biener, gcc-patches

On Wed, Jul 14, 2021 at 3:15 PM Richard Biener
<richard.guenther@gmail.com> wrote:
>
> On Wed, Jul 14, 2021 at 3:12 PM Richard Biener
> <richard.guenther@gmail.com> wrote:
> >
> > On Wed, Jul 14, 2021 at 2:48 PM Tamar Christina via Gcc-patches
> > <gcc-patches@gcc.gnu.org> wrote:
> > >
> > > Hi,
> > >
> > > Ever since this commit
> > >
> > > commit c9114f2804b91690e030383de15a24e0b738e856
> > > Author: Bernd Edlinger <bernd.edlinger@hotmail.de>
> > > Date:   Fri May 28 06:27:27 2021 +0200
> > >
> > > Various tools have been having trouble with cross compilation resulting in
> > >
> > > make[2]: *** No rule to make target '../build-x86_64-build_pc-linux-gnu/libcpp/libcpp.a', needed by 'build/genmatch'.
> > >
> > > (took a while to track down).  I don't understand this part of the build system well enough to know how to fix this.
> > > It looks like `libcpp.a` has special handling for cross compilers which now seems to be broken.
> > >
> > > I can't reproduce it with our normal cross compiler scripts. Which handles the stages on its own, but e.g.
> > > https://github.com/crosstool-ng/crosstool-ng does reproduce the failure.
> > >
> > > Any ideas what's going on?
> >
> > There should be a dependence of all-gcc to all-build-libcpp, Makefile.def has
> >
> > dependencies = { module=all-gcc; on=all-build-libcpp; };
> >
> > so how come build-libcpp is not built when gcc/ starts?
>
> Ah, I guess (gcc/Makefile.in):
>
> CPPLIB = ../libcpp/libcpp.a
> ...
> # For stage1 and when cross-compiling use the build libcpp which is
> # built with NLS disabled.  For stage2+ use the host library and
> # its dependencies.
> ifeq ($(build_objdir),$(build_libobjdir))
> BUILD_CPPLIB = $(build_libobjdir)/libcpp/libcpp.a
> else
> BUILD_CPPLIB = $(CPPLIB) $(LIBIBERTY)
>
> is not properly reflected in above dependences.  Not sure how to fix
> that though.

I guess add

 dependencies = { module=all-gcc; on=all-libcpp; };

like it is done for libiberty.

> > > Kind Regards,
> > > Tamar
> > >
> > > > -----Original Message-----
> > > > From: Gcc-patches <gcc-patches-bounces@gcc.gnu.org> On Behalf Of
> > > > Michael Matz
> > > > Sent: Friday, May 28, 2021 4:33 PM
> > > > To: Bernd Edlinger <bernd.edlinger@hotmail.de>
> > > > Cc: gcc-patches@gcc.gnu.org; Richard Biener <rguenther@suse.de>
> > > > Subject: Re: [PATCH] Generate gimple-match.c and generic-match.c earlier
> > > >
> > > > Hello,
> > > >
> > > > On Fri, 28 May 2021, Bernd Edlinger wrote:
> > > >
> > > > > >> I was wondering, why gimple-match.c and generic-match.c are not
> > > > > >> built early but always last, which slows down parallel makes
> > > > > >> significantly.
> > > > > >>
> > > > > >> The reason seems to be that generated_files does not mention
> > > > > >> gimple-match.c and generic-match.c.
> > > > > >>
> > > > > >> This comment in Makefile.in says it all:
> > > > > >>
> > > > > >> $(ALL_HOST_OBJS) : | $(generated_files)
> > > > > >>
> > > > > >> So this patch adds gimple-match.c generic-match.c to generated_files.
> > > > > >>
> > > > > >>
> > > > > >> Tested on x86_64-pc-linux-gnu.
> > > > > >> Is it OK for trunk?
> > > > > >
> > > > > > This should help for what I was complaining about in
> > > > > > https://gcc.gnu.org/pipermail/gcc/2021-May/235963.html . I build
> > > > > > with
> > > > > > -j24 and it was stalling on compiling gimple-match.c for me.
> > > > > > Looks like insn-attrtab.c is missed too; I saw genattrtab was running last
> > > > too.
> > > > > >
> > > > >
> > > > > Yeah, probably insn-automata.c as well, sometimes it is picked up
> > > > > early sometimes not. maybe $(simple_generated_c) should be added to
> > > > > generated_files, but insn-attrtab.c is yet another exception.
> > > >
> > > > You can't put files in there that are sometimes slow to generate (which insn-
> > > > {attrtab,automata}.c are on some targets), as _everything_ then waits for
> > > > them to be created first.
> > > >
> > > > Ideally there would be a way for gnumake to mark some targets as "ugh-
> > > > slow" and back-propagate this to all dependencies so that those are put in
> > > > front of the work queue in a parallel make.  Alas, something like that never
> > > > came into existence :-/  (When order-only deps were introduced I got
> > > > excited, but then came to realize that that wasn't what was really needed for
> > > > this case, a "weak" version of it would be required at least, or better yet a
> > > > specific facility to impose a cost with a target)
> > > >
> > > >
> > > > Ciao,
> > > > Michael.
> > > >
> > > > >
> > > > >
> > > > > Bernd.
> > > > >
> > > > > > Thanks,
> > > > > > Andrew
> > > > > >
> > > > > >>
> > > > > >>
> > > > > >> Thanks
> > > > > >> Bernd.
> > > > > >>
> > > > > >>
> > > > > >> 2021-05-28  Bernd Edlinger  <bernd.edlinger@hotmail.de>
> > > > > >>
> > > > > >>         * Makefile.in (generated_files): Add gimple-match.c and
> > > > > >>         generic-match.c
> > > > >

^ permalink raw reply	[flat|nested] 15+ messages in thread

* RE: [PATCH] Generate gimple-match.c and generic-match.c earlier
  2021-07-14 13:19             ` Richard Biener
@ 2021-07-14 13:25               ` Tamar Christina
  0 siblings, 0 replies; 15+ messages in thread
From: Tamar Christina @ 2021-07-14 13:25 UTC (permalink / raw)
  To: Richard Biener; +Cc: Michael Matz, Bernd Edlinger, Richard Biener, gcc-patches


> -----Original Message-----
> From: Richard Biener <richard.guenther@gmail.com>
> Sent: Wednesday, July 14, 2021 2:19 PM
> To: Tamar Christina <Tamar.Christina@arm.com>
> Cc: Michael Matz <matz@suse.de>; Bernd Edlinger
> <bernd.edlinger@hotmail.de>; Richard Biener <rguenther@suse.de>; gcc-
> patches@gcc.gnu.org
> Subject: Re: [PATCH] Generate gimple-match.c and generic-match.c earlier
> 
> On Wed, Jul 14, 2021 at 3:15 PM Richard Biener
> <richard.guenther@gmail.com> wrote:
> >
> > On Wed, Jul 14, 2021 at 3:12 PM Richard Biener
> > <richard.guenther@gmail.com> wrote:
> > >
> > > On Wed, Jul 14, 2021 at 2:48 PM Tamar Christina via Gcc-patches
> > > <gcc-patches@gcc.gnu.org> wrote:
> > > >
> > > > Hi,
> > > >
> > > > Ever since this commit
> > > >
> > > > commit c9114f2804b91690e030383de15a24e0b738e856
> > > > Author: Bernd Edlinger <bernd.edlinger@hotmail.de>
> > > > Date:   Fri May 28 06:27:27 2021 +0200
> > > >
> > > > Various tools have been having trouble with cross compilation
> > > > resulting in
> > > >
> > > > make[2]: *** No rule to make target '../build-x86_64-build_pc-linux-
> gnu/libcpp/libcpp.a', needed by 'build/genmatch'.
> > > >
> > > > (took a while to track down).  I don't understand this part of the build
> system well enough to know how to fix this.
> > > > It looks like `libcpp.a` has special handling for cross compilers which now
> seems to be broken.
> > > >
> > > > I can't reproduce it with our normal cross compiler scripts. Which
> handles the stages on its own, but e.g.
> > > > https://github.com/crosstool-ng/crosstool-ng does reproduce the
> failure.
> > > >
> > > > Any ideas what's going on?
> > >
> > > There should be a dependence of all-gcc to all-build-libcpp,
> > > Makefile.def has
> > >
> > > dependencies = { module=all-gcc; on=all-build-libcpp; };
> > >
> > > so how come build-libcpp is not built when gcc/ starts?
> >
> > Ah, I guess (gcc/Makefile.in):
> >
> > CPPLIB = ../libcpp/libcpp.a
> > ...
> > # For stage1 and when cross-compiling use the build libcpp which is #
> > built with NLS disabled.  For stage2+ use the host library and # its
> > dependencies.
> > ifeq ($(build_objdir),$(build_libobjdir))
> > BUILD_CPPLIB = $(build_libobjdir)/libcpp/libcpp.a
> > else
> > BUILD_CPPLIB = $(CPPLIB) $(LIBIBERTY)
> >
> > is not properly reflected in above dependences.  Not sure how to fix
> > that though.
> 
> I guess add
> 
>  dependencies = { module=all-gcc; on=all-libcpp; };
> 

Isn't that already there? I see

dependencies = { module=all-gcc; on=all-libcpp; hard=true; };

already in Makefile.def

Regards,
Tamar

> like it is done for libiberty.
> 
> > > > Kind Regards,
> > > > Tamar
> > > >
> > > > > -----Original Message-----
> > > > > From: Gcc-patches <gcc-patches-bounces@gcc.gnu.org> On Behalf Of
> > > > > Michael Matz
> > > > > Sent: Friday, May 28, 2021 4:33 PM
> > > > > To: Bernd Edlinger <bernd.edlinger@hotmail.de>
> > > > > Cc: gcc-patches@gcc.gnu.org; Richard Biener <rguenther@suse.de>
> > > > > Subject: Re: [PATCH] Generate gimple-match.c and generic-match.c
> > > > > earlier
> > > > >
> > > > > Hello,
> > > > >
> > > > > On Fri, 28 May 2021, Bernd Edlinger wrote:
> > > > >
> > > > > > >> I was wondering, why gimple-match.c and generic-match.c are
> > > > > > >> not built early but always last, which slows down parallel
> > > > > > >> makes significantly.
> > > > > > >>
> > > > > > >> The reason seems to be that generated_files does not
> > > > > > >> mention gimple-match.c and generic-match.c.
> > > > > > >>
> > > > > > >> This comment in Makefile.in says it all:
> > > > > > >>
> > > > > > >> $(ALL_HOST_OBJS) : | $(generated_files)
> > > > > > >>
> > > > > > >> So this patch adds gimple-match.c generic-match.c to
> generated_files.
> > > > > > >>
> > > > > > >>
> > > > > > >> Tested on x86_64-pc-linux-gnu.
> > > > > > >> Is it OK for trunk?
> > > > > > >
> > > > > > > This should help for what I was complaining about in
> > > > > > > https://gcc.gnu.org/pipermail/gcc/2021-May/235963.html . I
> > > > > > > build with
> > > > > > > -j24 and it was stalling on compiling gimple-match.c for me.
> > > > > > > Looks like insn-attrtab.c is missed too; I saw genattrtab
> > > > > > > was running last
> > > > > too.
> > > > > > >
> > > > > >
> > > > > > Yeah, probably insn-automata.c as well, sometimes it is picked
> > > > > > up early sometimes not. maybe $(simple_generated_c) should be
> > > > > > added to generated_files, but insn-attrtab.c is yet another
> exception.
> > > > >
> > > > > You can't put files in there that are sometimes slow to generate
> > > > > (which insn- {attrtab,automata}.c are on some targets), as
> > > > > _everything_ then waits for them to be created first.
> > > > >
> > > > > Ideally there would be a way for gnumake to mark some targets as
> > > > > "ugh- slow" and back-propagate this to all dependencies so that
> > > > > those are put in front of the work queue in a parallel make.
> > > > > Alas, something like that never came into existence :-/  (When
> > > > > order-only deps were introduced I got excited, but then came to
> > > > > realize that that wasn't what was really needed for this case, a
> > > > > "weak" version of it would be required at least, or better yet a
> > > > > specific facility to impose a cost with a target)
> > > > >
> > > > >
> > > > > Ciao,
> > > > > Michael.
> > > > >
> > > > > >
> > > > > >
> > > > > > Bernd.
> > > > > >
> > > > > > > Thanks,
> > > > > > > Andrew
> > > > > > >
> > > > > > >>
> > > > > > >>
> > > > > > >> Thanks
> > > > > > >> Bernd.
> > > > > > >>
> > > > > > >>
> > > > > > >> 2021-05-28  Bernd Edlinger  <bernd.edlinger@hotmail.de>
> > > > > > >>
> > > > > > >>         * Makefile.in (generated_files): Add gimple-match.c and
> > > > > > >>         generic-match.c
> > > > > >

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH] Generate gimple-match.c and generic-match.c earlier
  2021-07-14 12:47       ` Tamar Christina
  2021-07-14 13:12         ` Richard Biener
@ 2021-07-14 15:56         ` Bernd Edlinger
  2021-07-14 16:47           ` Tamar Christina
  1 sibling, 1 reply; 15+ messages in thread
From: Bernd Edlinger @ 2021-07-14 15:56 UTC (permalink / raw)
  To: Tamar Christina, Michael Matz; +Cc: gcc-patches, Richard Biener

On 7/14/21 2:47 PM, Tamar Christina wrote:
> Hi,
> 
> Ever since this commit 
> 
> commit c9114f2804b91690e030383de15a24e0b738e856
> Author: Bernd Edlinger <bernd.edlinger@hotmail.de>
> Date:   Fri May 28 06:27:27 2021 +0200
> 
> Various tools have been having trouble with cross compilation resulting in
> 
> make[2]: *** No rule to make target '../build-x86_64-build_pc-linux-gnu/libcpp/libcpp.a', needed by 'build/genmatch'.
> 
> (took a while to track down).  I don't understand this part of the build system well enough to know how to fix this.
> It looks like `libcpp.a` has special handling for cross compilers which now seems to be broken.
> 
> I can't reproduce it with our normal cross compiler scripts. Which handles the stages on its own, but e.g.
> https://github.com/crosstool-ng/crosstool-ng does reproduce the failure.
> 

Sorry for the breakage!

I do not know this tool at all, but this here looks suspicious,
as it it by-passes the dependencies in the top-level Makefile:

https://github.com/crosstool-ng/crosstool-ng/blob/755850d07ec4e8dc44787d1a0e35fe19507f17f6/scripts/build/cc/gcc.sh#L682-L683
            CT_DoExecLog CFG make ${CT_JOBSFLAGS} configure-gcc configure-libcpp configure-build-libiberty
            CT_DoExecLog ALL make ${CT_JOBSFLAGS} all-libcpp all-build-libiberty
...
https://github.com/crosstool-ng/crosstool-ng/blob/755850d07ec4e8dc44787d1a0e35fe19507f17f6/scripts/build/cc/gcc.sh#L711-L712
        CT_DoExecLog ALL make ${CT_JOBSFLAGS} -C gcc ${libgcc_rule} \
                              ${repair_cc}


but the top-level Makefile has also a dependency to all-build-libcpp:

dependencies = { module=all-gcc; on=all-build-libcpp; };
dependencies = { module=all-gcc; on=all-libcpp; hard=true; };

Maybe this just worked by chance, when building with "make -j" started a parallel build,
might build the build-libcpp dependency eventually, but due to the patch it is
needed earlier?


Bernd.



> Any ideas what's going on?
> 
> Kind Regards,
> Tamar
> 
>> -----Original Message-----
>> From: Gcc-patches <gcc-patches-bounces@gcc.gnu.org> On Behalf Of
>> Michael Matz
>> Sent: Friday, May 28, 2021 4:33 PM
>> To: Bernd Edlinger <bernd.edlinger@hotmail.de>
>> Cc: gcc-patches@gcc.gnu.org; Richard Biener <rguenther@suse.de>
>> Subject: Re: [PATCH] Generate gimple-match.c and generic-match.c earlier
>>
>> Hello,
>>
>> On Fri, 28 May 2021, Bernd Edlinger wrote:
>>
>>>>> I was wondering, why gimple-match.c and generic-match.c are not
>>>>> built early but always last, which slows down parallel makes
>>>>> significantly.
>>>>>
>>>>> The reason seems to be that generated_files does not mention
>>>>> gimple-match.c and generic-match.c.
>>>>>
>>>>> This comment in Makefile.in says it all:
>>>>>
>>>>> $(ALL_HOST_OBJS) : | $(generated_files)
>>>>>
>>>>> So this patch adds gimple-match.c generic-match.c to generated_files.
>>>>>
>>>>>
>>>>> Tested on x86_64-pc-linux-gnu.
>>>>> Is it OK for trunk?
>>>>
>>>> This should help for what I was complaining about in
>>>> https://gcc.gnu.org/pipermail/gcc/2021-May/235963.html . I build
>>>> with
>>>> -j24 and it was stalling on compiling gimple-match.c for me.
>>>> Looks like insn-attrtab.c is missed too; I saw genattrtab was running last
>> too.
>>>>
>>>
>>> Yeah, probably insn-automata.c as well, sometimes it is picked up
>>> early sometimes not. maybe $(simple_generated_c) should be added to
>>> generated_files, but insn-attrtab.c is yet another exception.
>>
>> You can't put files in there that are sometimes slow to generate (which insn-
>> {attrtab,automata}.c are on some targets), as _everything_ then waits for
>> them to be created first.
>>
>> Ideally there would be a way for gnumake to mark some targets as "ugh-
>> slow" and back-propagate this to all dependencies so that those are put in
>> front of the work queue in a parallel make.  Alas, something like that never
>> came into existence :-/  (When order-only deps were introduced I got
>> excited, but then came to realize that that wasn't what was really needed for
>> this case, a "weak" version of it would be required at least, or better yet a
>> specific facility to impose a cost with a target)
>>
>>
>> Ciao,
>> Michael.
>>
>>>
>>>
>>> Bernd.
>>>
>>>> Thanks,
>>>> Andrew
>>>>
>>>>>
>>>>>
>>>>> Thanks
>>>>> Bernd.
>>>>>
>>>>>
>>>>> 2021-05-28  Bernd Edlinger  <bernd.edlinger@hotmail.de>
>>>>>
>>>>>         * Makefile.in (generated_files): Add gimple-match.c and
>>>>>         generic-match.c
>>>

^ permalink raw reply	[flat|nested] 15+ messages in thread

* RE: [PATCH] Generate gimple-match.c and generic-match.c earlier
  2021-07-14 15:56         ` Bernd Edlinger
@ 2021-07-14 16:47           ` Tamar Christina
  0 siblings, 0 replies; 15+ messages in thread
From: Tamar Christina @ 2021-07-14 16:47 UTC (permalink / raw)
  To: Bernd Edlinger, Michael Matz; +Cc: gcc-patches, Richard Biener


> -----Original Message-----
> From: Bernd Edlinger <bernd.edlinger@hotmail.de>
> Sent: Wednesday, July 14, 2021 4:56 PM
> To: Tamar Christina <Tamar.Christina@arm.com>; Michael Matz
> <matz@suse.de>
> Cc: gcc-patches@gcc.gnu.org; Richard Biener <rguenther@suse.de>
> Subject: Re: [PATCH] Generate gimple-match.c and generic-match.c earlier
> 
> On 7/14/21 2:47 PM, Tamar Christina wrote:
> > Hi,
> >
> > Ever since this commit
> >
> > commit c9114f2804b91690e030383de15a24e0b738e856
> > Author: Bernd Edlinger <bernd.edlinger@hotmail.de>
> > Date:   Fri May 28 06:27:27 2021 +0200
> >
> > Various tools have been having trouble with cross compilation
> > resulting in
> >
> > make[2]: *** No rule to make target '../build-x86_64-build_pc-linux-
> gnu/libcpp/libcpp.a', needed by 'build/genmatch'.
> >
> > (took a while to track down).  I don't understand this part of the build
> system well enough to know how to fix this.
> > It looks like `libcpp.a` has special handling for cross compilers which now
> seems to be broken.
> >
> > I can't reproduce it with our normal cross compiler scripts. Which handles
> the stages on its own, but e.g.
> > https://github.com/crosstool-ng/crosstool-ng does reproduce the failure.
> >
> 
> Sorry for the breakage!
> 
> I do not know this tool at all, but this here looks suspicious, as it it by-passes
> the dependencies in the top-level Makefile:
> 
> https://github.com/crosstool-ng/crosstool-
> ng/blob/755850d07ec4e8dc44787d1a0e35fe19507f17f6/scripts/build/cc/gcc.s
> h#L682-L683
>             CT_DoExecLog CFG make ${CT_JOBSFLAGS} configure-gcc configure-
> libcpp configure-build-libiberty
>             CT_DoExecLog ALL make ${CT_JOBSFLAGS} all-libcpp all-build-
> libiberty ...
> https://github.com/crosstool-ng/crosstool-
> ng/blob/755850d07ec4e8dc44787d1a0e35fe19507f17f6/scripts/build/cc/gcc.s
> h#L711-L712
>         CT_DoExecLog ALL make ${CT_JOBSFLAGS} -C gcc ${libgcc_rule} \
>                               ${repair_cc}
> 
> 
> but the top-level Makefile has also a dependency to all-build-libcpp:
> 
> dependencies = { module=all-gcc; on=all-build-libcpp; }; dependencies =
> { module=all-gcc; on=all-libcpp; hard=true; };
> 
> Maybe this just worked by chance, when building with "make -j" started a
> parallel build, might build the build-libcpp dependency eventually, but due to
> the patch it is needed earlier?

Ah I didn't notice they bypassed the top level makefile, this would explain why I can't reproduce it with my own scripts..

Thanks! I'll file a bug with them then.

Kind Regards,
Tamar

> 
> 
> Bernd.
> 
> 
> 
> > Any ideas what's going on?
> >
> > Kind Regards,
> > Tamar
> >
> >> -----Original Message-----
> >> From: Gcc-patches <gcc-patches-bounces@gcc.gnu.org> On Behalf Of
> >> Michael Matz
> >> Sent: Friday, May 28, 2021 4:33 PM
> >> To: Bernd Edlinger <bernd.edlinger@hotmail.de>
> >> Cc: gcc-patches@gcc.gnu.org; Richard Biener <rguenther@suse.de>
> >> Subject: Re: [PATCH] Generate gimple-match.c and generic-match.c
> >> earlier
> >>
> >> Hello,
> >>
> >> On Fri, 28 May 2021, Bernd Edlinger wrote:
> >>
> >>>>> I was wondering, why gimple-match.c and generic-match.c are not
> >>>>> built early but always last, which slows down parallel makes
> >>>>> significantly.
> >>>>>
> >>>>> The reason seems to be that generated_files does not mention
> >>>>> gimple-match.c and generic-match.c.
> >>>>>
> >>>>> This comment in Makefile.in says it all:
> >>>>>
> >>>>> $(ALL_HOST_OBJS) : | $(generated_files)
> >>>>>
> >>>>> So this patch adds gimple-match.c generic-match.c to generated_files.
> >>>>>
> >>>>>
> >>>>> Tested on x86_64-pc-linux-gnu.
> >>>>> Is it OK for trunk?
> >>>>
> >>>> This should help for what I was complaining about in
> >>>> https://gcc.gnu.org/pipermail/gcc/2021-May/235963.html . I build
> >>>> with
> >>>> -j24 and it was stalling on compiling gimple-match.c for me.
> >>>> Looks like insn-attrtab.c is missed too; I saw genattrtab was
> >>>> running last
> >> too.
> >>>>
> >>>
> >>> Yeah, probably insn-automata.c as well, sometimes it is picked up
> >>> early sometimes not. maybe $(simple_generated_c) should be added to
> >>> generated_files, but insn-attrtab.c is yet another exception.
> >>
> >> You can't put files in there that are sometimes slow to generate
> >> (which insn- {attrtab,automata}.c are on some targets), as
> >> _everything_ then waits for them to be created first.
> >>
> >> Ideally there would be a way for gnumake to mark some targets as
> >> "ugh- slow" and back-propagate this to all dependencies so that those
> >> are put in front of the work queue in a parallel make.  Alas,
> >> something like that never came into existence :-/  (When order-only
> >> deps were introduced I got excited, but then came to realize that
> >> that wasn't what was really needed for this case, a "weak" version of
> >> it would be required at least, or better yet a specific facility to
> >> impose a cost with a target)
> >>
> >>
> >> Ciao,
> >> Michael.
> >>
> >>>
> >>>
> >>> Bernd.
> >>>
> >>>> Thanks,
> >>>> Andrew
> >>>>
> >>>>>
> >>>>>
> >>>>> Thanks
> >>>>> Bernd.
> >>>>>
> >>>>>
> >>>>> 2021-05-28  Bernd Edlinger  <bernd.edlinger@hotmail.de>
> >>>>>
> >>>>>         * Makefile.in (generated_files): Add gimple-match.c and
> >>>>>         generic-match.c
> >>>

^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2021-07-14 16:48 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-05-28  4:42 [PATCH] Generate gimple-match.c and generic-match.c earlier Bernd Edlinger
2021-05-28  4:51 ` Bernd Edlinger
2021-05-28  6:41   ` Richard Biener
2021-05-28  6:48     ` Bernd Edlinger
2021-05-28  7:26 ` Andrew Pinski
2021-05-28  7:33   ` Bernd Edlinger
2021-05-28  8:48     ` Jakub Jelinek
2021-05-28 15:33     ` Michael Matz
2021-07-14 12:47       ` Tamar Christina
2021-07-14 13:12         ` Richard Biener
2021-07-14 13:15           ` Richard Biener
2021-07-14 13:19             ` Richard Biener
2021-07-14 13:25               ` Tamar Christina
2021-07-14 15:56         ` Bernd Edlinger
2021-07-14 16:47           ` Tamar Christina

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).