Heads up: Problems with parallel make

Heads up: Problems with parallel make

[Ken Brown]

Several of my packages that used to build fine with parallel make now 
require 'make -j1'.  Without this I either get strange errors or a hang.

I can't find any pattern to explain when the problem occurs and when it 
doesn't, but it seems to be due to a change in make, starting with 
version 4.4.  By default, make now uses a FIFO rather than a pipe in its 
jobserver implementation.  You can override this with the make flag

   --jobserver-style=pipe

In all cases where I've tried this, parallel make works again.

It wouldn't surprise me if there's a bug in Cygwin's FIFO implementation 
that's responsible for this, but I haven't yet tried to track it down.

Ken

Re: Heads up: Problems with parallel make

[Brian Inglis]

On 2023-02-13 16:05, Ken Brown via Cygwin-apps wrote:
> Several of my packages that used to build fine with parallel make now require 
> 'make -j1'.  Without this I either get strange errors or a hang.
> I can't find any pattern to explain when the problem occurs and when it doesn't, 
> but it seems to be due to a change in make, starting with version 4.4.  By 
> default, make now uses a FIFO rather than a pipe in its jobserver 
> implementation.  You can override this with the make flag
>    --jobserver-style=pipe
> In all cases where I've tried this, parallel make works again.
> It wouldn't surprise me if there's a bug in Cygwin's FIFO implementation that's 
> responsible for this, but I haven't yet tried to track it down.

There is some glibc? compatibility issue also with this recommended workaround.

-- 
Take care. Thanks, Brian Inglis			Calgary, Alberta, Canada

La perfection est atteinte			Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter	not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer	but when there is no more to cut
			-- Antoine de Saint-Exupéry

Re: Heads up: Problems with parallel make

[Ken Brown]

On 2/13/2023 6:44 PM, Brian Inglis wrote:
> On 2023-02-13 16:05, Ken Brown via Cygwin-apps wrote:
>> Several of my packages that used to build fine with parallel make now 
>> require 'make -j1'.  Without this I either get strange errors or a hang.
>> I can't find any pattern to explain when the problem occurs and when 
>> it doesn't, but it seems to be due to a change in make, starting with 
>> version 4.4.  By default, make now uses a FIFO rather than a pipe in 
>> its jobserver implementation.  You can override this with the make flag
>>    --jobserver-style=pipe
>> In all cases where I've tried this, parallel make works again.
>> It wouldn't surprise me if there's a bug in Cygwin's FIFO 
>> implementation that's responsible for this, but I haven't yet tried to 
>> track it down.
> 
> There is some glibc? compatibility issue also with this recommended 
> workaround.

Could you elaborate?  I don't know what issue you're seeing.

Ken

Re: Heads up: Problems with parallel make

[Brian Inglis]

On 2023-02-13 17:38, Ken Brown via Cygwin-apps wrote:
> On 2/13/2023 6:44 PM, Brian Inglis wrote:
>> On 2023-02-13 16:05, Ken Brown via Cygwin-apps wrote:
>>> Several of my packages that used to build fine with parallel make now require 
>>> 'make -j1'.  Without this I either get strange errors or a hang.
>>> I can't find any pattern to explain when the problem occurs and when it 
>>> doesn't, but it seems to be due to a change in make, starting with version 
>>> 4.4.  By default, make now uses a FIFO rather than a pipe in its jobserver 
>>> implementation.  You can override this with the make flag
>>>    --jobserver-style=pipe
>>> In all cases where I've tried this, parallel make works again.
>>> It wouldn't surprise me if there's a bug in Cygwin's FIFO implementation 
>>> that's responsible for this, but I haven't yet tried to track it down.
>>
>> There is some glibc? compatibility issue also with this recommended workaround.
> 
> Could you elaborate?  I don't know what issue you're seeing.

There have been reports from various projects mentioning problems in other 
packages ending up at make 4.4 and FIFOs solved by reverting to pipes.

There is a make 4.4.1 release candidate 4.40.0.90 with patches that may help to 
deal with this:

	https://alpha.gnu.org/gnu/make/make-4.4.0.90.tar.lz{,.sig}

https://sv.gnu.org/bugs/index.php?group=make&report_id=111&fix_release_id=110&set=custom

-- 
Take care. Thanks, Brian Inglis			Calgary, Alberta, Canada

La perfection est atteinte			Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter	not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer	but when there is no more to cut
			-- Antoine de Saint-Exupéry

Re: Heads up: Problems with parallel make

[Marco Atzeri]

On 14.02.2023 17:49, Brian Inglis via Cygwin-apps wrote:
> On 2023-02-13 17:38, Ken Brown via Cygwin-apps wrote:
>> On 2/13/2023 6:44 PM, Brian Inglis wrote:
>>> On 2023-02-13 16:05, Ken Brown via Cygwin-apps wrote:
>>>> Several of my packages that used to build fine with parallel make 
>>>> now require 'make -j1'.  Without this I either get strange errors or 
>>>> a hang.
>>>> I can't find any pattern to explain when the problem occurs and when 
>>>> it doesn't, but it seems to be due to a change in make, starting 
>>>> with version 4.4.  By default, make now uses a FIFO rather than a 
>>>> pipe in its jobserver implementation.  You can override this with 
>>>> the make flag
>>>>    --jobserver-style=pipe
>>>> In all cases where I've tried this, parallel make works again.
>>>> It wouldn't surprise me if there's a bug in Cygwin's FIFO 
>>>> implementation that's responsible for this, but I haven't yet tried 
>>>> to track it down.
>>>
>>> There is some glibc? compatibility issue also with this recommended 
>>> workaround.
>>
>> Could you elaborate?  I don't know what issue you're seeing.
> 
> There have been reports from various projects mentioning problems in 
> other packages ending up at make 4.4 and FIFOs solved by reverting to 
> pipes.
> 
> There is a make 4.4.1 release candidate 4.40.0.90 with patches that may 
> help to deal with this:
> 
>      https://alpha.gnu.org/gnu/make/make-4.4.0.90.tar.lz{,.sig}
> 
> https://sv.gnu.org/bugs/index.php?group=make&report_id=111&fix_release_id=110&set=custom
> 

test version built and on the way

please check

Re: Heads up: Problems with parallel make

[Ken Brown]

On 2/14/2023 4:23 PM, Marco Atzeri via Cygwin-apps wrote:
> On 14.02.2023 17:49, Brian Inglis via Cygwin-apps wrote:
>> There is a make 4.4.1 release candidate 4.40.0.90 with patches that 
>> may help to deal with this:
>>
>>      https://alpha.gnu.org/gnu/make/make-4.4.0.90.tar.lz{,.sig}
>>
>> https://sv.gnu.org/bugs/index.php?group=make&report_id=111&fix_release_id=110&set=custom
>>
> 
> test version built and on the way

The problems I reported with make 4.4 still exist in 4.4.0.90.

Until someone can figure out why this happens, you might want to 
consider building make so that it doesn't default to using a FIFO for 
its jobserver.  I suggested to the make developers that they provide a 
configure option for this purpose.  If they don't do this, I think you 
could accomplish the same thing by configuring make with 
ac_cv_func_mkfifo=no.

Ken

Re: Heads up: Problems with parallel make

[Marco Atzeri]

On 15.02.2023 19:09, Ken Brown via Cygwin-apps wrote:
> On 2/14/2023 4:23 PM, Marco Atzeri via Cygwin-apps wrote:
>> On 14.02.2023 17:49, Brian Inglis via Cygwin-apps wrote:
>>> There is a make 4.4.1 release candidate 4.40.0.90 with patches that 
>>> may help to deal with this:
>>>
>>>      https://alpha.gnu.org/gnu/make/make-4.4.0.90.tar.lz{,.sig}
>>>
>>> https://sv.gnu.org/bugs/index.php?group=make&report_id=111&fix_release_id=110&set=custom
>>>
>>
>> test version built and on the way
> 
> The problems I reported with make 4.4 still exist in 4.4.0.90.
> 
> Until someone can figure out why this happens, you might want to 
> consider building make so that it doesn't default to using a FIFO for 
> its jobserver.  I suggested to the make developers that they provide a 
> configure option for this purpose.  If they don't do this, I think you 
> could accomplish the same thing by configuring make with 
> ac_cv_func_mkfifo=no.
> 
> Ken

version 2 uploaded.


Regards
Marco

Re: Heads up: Problems with parallel make

[Brian Inglis]

On 2023-02-16 00:47, Marco Atzeri via Cygwin-apps wrote:
> On 15.02.2023 19:09, Ken Brown via Cygwin-apps wrote:
>> On 2/14/2023 4:23 PM, Marco Atzeri via Cygwin-apps wrote:
>>> On 14.02.2023 17:49, Brian Inglis via Cygwin-apps wrote:
>>>> There is a make 4.4.1 release candidate 4.40.0.90 with patches that may help 
>>>> to deal with this:
>>>>      https://alpha.gnu.org/gnu/make/make-4.4.0.90.tar.lz{,.sig}
>>>> https://sv.gnu.org/bugs/index.php?group=make&report_id=111&fix_release_id=110&set=custom

>>> test version built and on the way

>> The problems I reported with make 4.4 still exist in 4.4.0.90.
>> Until someone can figure out why this happens, you might want to consider 
>> building make so that it doesn't default to using a FIFO for its jobserver.  I 
>> suggested to the make developers that they provide a configure option for this 
>> purpose.  If they don't do this, I think you could accomplish the same thing 
>> by configuring make with ac_cv_func_mkfifo=no.

> version 2 uploaded.

Hi guys,

Might be good to follow up on bug-make at gnu dot org to let them know that 
pre-release does not help us, and it has been rereleased building with 
ac_cv_func_mkfifo=no which does.

I notice that posix_spawn is defaulted, any chance it could be implicated?

-- 
Take care. Thanks, Brian Inglis			Calgary, Alberta, Canada

La perfection est atteinte			Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter	not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer	but when there is no more to cut
			-- Antoine de Saint-Exupéry

Re: Heads up: Problems with parallel make

[Marco Atzeri]

On 16.02.2023 17:49, Brian Inglis via Cygwin-apps wrote:
> On 2023-02-16 00:47, Marco Atzeri via Cygwin-apps wrote:
>> On 15.02.2023 19:09, Ken Brown via Cygwin-apps wrote:
>>> On 2/14/2023 4:23 PM, Marco Atzeri via Cygwin-apps wrote:
>>>> On 14.02.2023 17:49, Brian Inglis via Cygwin-apps wrote:
>>>>> There is a make 4.4.1 release candidate 4.40.0.90 with patches that 
>>>>> may help to deal with this:
>>>>>      https://alpha.gnu.org/gnu/make/make-4.4.0.90.tar.lz{,.sig}
>>>>> https://sv.gnu.org/bugs/index.php?group=make&report_id=111&fix_release_id=110&set=custom
> 
>>>> test version built and on the way
> 
>>> The problems I reported with make 4.4 still exist in 4.4.0.90.
>>> Until someone can figure out why this happens, you might want to 
>>> consider building make so that it doesn't default to using a FIFO for 
>>> its jobserver.  I suggested to the make developers that they provide 
>>> a configure option for this purpose.  If they don't do this, I think 
>>> you could accomplish the same thing by configuring make with 
>>> ac_cv_func_mkfifo=no.
> 
>> version 2 uploaded.
> 
> Hi guys,
> 
> Might be good to follow up on bug-make at gnu dot org to let them know 
> that pre-release does not help us, and it has been rereleased building 
> with ac_cv_func_mkfifo=no which does.
> 
> I notice that posix_spawn is defaulted, any chance it could be implicated?
> 

what are you using as test ?

Standard build tests are fine for both -1 and -2

Marco

Re: Heads up: Problems with parallel make

[Brian Inglis]

On 2023-02-16 11:57, Marco Atzeri via Cygwin-apps wrote:
> On 16.02.2023 17:49, Brian Inglis via Cygwin-apps wrote:
>> On 2023-02-16 00:47, Marco Atzeri via Cygwin-apps wrote:
>>> On 15.02.2023 19:09, Ken Brown via Cygwin-apps wrote:
>>>> On 2/14/2023 4:23 PM, Marco Atzeri via Cygwin-apps wrote:
>>>>> On 14.02.2023 17:49, Brian Inglis via Cygwin-apps wrote:
>>>>>> There is a make 4.4.1 release candidate 4.40.0.90 with patches that may 
>>>>>> help to deal with this:
>>>>>>      https://alpha.gnu.org/gnu/make/make-4.4.0.90.tar.lz{,.sig}
>>>>>> https://sv.gnu.org/bugs/index.php?group=make&report_id=111&fix_release_id=110&set=custom

>>>>> test version built and on the way

>>>> The problems I reported with make 4.4 still exist in 4.4.0.90.
>>>> Until someone can figure out why this happens, you might want to consider 
>>>> building make so that it doesn't default to using a FIFO for its jobserver.  
>>>> I suggested to the make developers that they provide a configure option for 
>>>> this purpose.  If they don't do this, I think you could accomplish the same 
>>>> thing by configuring make with ac_cv_func_mkfifo=no.

>>> version 2 uploaded.

>> Might be good to follow up on bug-make at gnu dot org to let them know that 
>> pre-release does not help us, and it has been rereleased building with 
>> ac_cv_func_mkfifo=no which does.

>> I notice that posix_spawn is defaulted, any chance it could be implicated?

> what are you using as test ?
> Standard build tests are fine for both -1 and -2

I'm not seeing make issues so far! x fingers - touch wood - no jinxes! ;^>

Are there high parallel make tests that show the issues with FIFO?

Just aware that if there are issues, sometimes flipping a config option gives a 
different result, and occasionally a fix, or a broken build. :-(

-- 
Take care. Thanks, Brian Inglis			Calgary, Alberta, Canada

La perfection est atteinte			Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter	not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer	but when there is no more to cut
			-- Antoine de Saint-Exupéry

Re: Heads up: Problems with parallel make

[Ken Brown]

On 2/16/2023 2:47 AM, Marco Atzeri via Cygwin-apps wrote:
> On 15.02.2023 19:09, Ken Brown via Cygwin-apps wrote:
>> On 2/14/2023 4:23 PM, Marco Atzeri via Cygwin-apps wrote:
>>> On 14.02.2023 17:49, Brian Inglis via Cygwin-apps wrote:
>>>> There is a make 4.4.1 release candidate 4.40.0.90 with patches that 
>>>> may help to deal with this:
>>>>
>>>>      https://alpha.gnu.org/gnu/make/make-4.4.0.90.tar.lz{,.sig}
>>>>
>>>> https://sv.gnu.org/bugs/index.php?group=make&report_id=111&fix_release_id=110&set=custom
>>>>
>>>
>>> test version built and on the way
>>
>> The problems I reported with make 4.4 still exist in 4.4.0.90.
>>
>> Until someone can figure out why this happens, you might want to 
>> consider building make so that it doesn't default to using a FIFO for 
>> its jobserver.  I suggested to the make developers that they provide a 
>> configure option for this purpose.  If they don't do this, I think you 
>> could accomplish the same thing by configuring make with 
>> ac_cv_func_mkfifo=no.
>>
>> Ken
> 
> version 2 uploaded.

Thanks, Marco.  As expected, that fixes the problem for my test case 
(building TeX Live).  Obviously it would be better if the make 
developers would provide a configure option to use a pipe for the 
jobserver, but this is a good workaround if they don't.

Ken

Re: Heads up: Problems with parallel make

[Ken Brown]

On 2/16/2023 9:21 PM, Ken Brown via Cygwin-apps wrote:
> On 2/16/2023 2:47 AM, Marco Atzeri via Cygwin-apps wrote:
>> On 15.02.2023 19:09, Ken Brown via Cygwin-apps wrote:
>>> On 2/14/2023 4:23 PM, Marco Atzeri via Cygwin-apps wrote:
>>>> On 14.02.2023 17:49, Brian Inglis via Cygwin-apps wrote:
>>>>> There is a make 4.4.1 release candidate 4.40.0.90 with patches that 
>>>>> may help to deal with this:
>>>>>
>>>>>      https://alpha.gnu.org/gnu/make/make-4.4.0.90.tar.lz{,.sig}
>>>>>
>>>>> https://sv.gnu.org/bugs/index.php?group=make&report_id=111&fix_release_id=110&set=custom
>>>>>
>>>>
>>>> test version built and on the way
>>>
>>> The problems I reported with make 4.4 still exist in 4.4.0.90.
>>>
>>> Until someone can figure out why this happens, you might want to 
>>> consider building make so that it doesn't default to using a FIFO for 
>>> its jobserver.  I suggested to the make developers that they provide 
>>> a configure option for this purpose.  If they don't do this, I think 
>>> you could accomplish the same thing by configuring make with 
>>> ac_cv_func_mkfifo=no.
>>>
>>> Ken
>>
>> version 2 uploaded.
> 
> Thanks, Marco.  As expected, that fixes the problem for my test case 
> (building TeX Live).  Obviously it would be better if the make 
> developers would provide a configure option to use a pipe for the 
> jobserver, but this is a good workaround if they don't.

Just to finish this off, there's now a new release candidate 4.4.0.91 
for make, which makes the jobserver use a pipe by default on Cygwin. 
You can override this default when building make by defining 
JOBSERVER_USE_FIFO to 1, and you can override it when running make by 
using the make option --jobserver-style=fifo.  But there's no longer a 
need to build make with ac_cv_func_mkfifo=no, and there's no longer a 
need to use --jobserver-style=pipe when running make.

I'm very impressed with how responsive the make developer (Paul Smith) was.

Ken

Re: Heads up: Problems with parallel make

[Achim Gratz]

Ken Brown via Cygwin-apps writes:
> Thanks, Marco.  As expected, that fixes the problem for my test case
> (building TeX Live).  Obviously it would be better if the make
> developers would provide a configure option to use a pipe for the
> jobserver, but this is a good workaround if they don't.

Does it actually do a parallel build?  I've quickly tested a cmake based
project with it and it completely serializes the build.  Reverting to
4.4 resolves that problem.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Waldorf MIDI Implementation & additional documentation:
http://Synth.Stromeko.net/Downloads.html#WaldorfDocs

Re: Heads up: Problems with parallel make

[Ken Brown]

On 2/20/2023 12:00 PM, Achim Gratz via Cygwin-apps wrote:
> Ken Brown via Cygwin-apps writes:
>> Thanks, Marco.  As expected, that fixes the problem for my test case
>> (building TeX Live).  Obviously it would be better if the make
>> developers would provide a configure option to use a pipe for the
>> jobserver, but this is a good workaround if they don't.
> 
> Does it actually do a parallel build?  I've quickly tested a cmake based
> project with it and it completely serializes the build.  Reverting to
> 4.4 resolves that problem.

It does for my build of TeX Live.  I just tried 'make -j13 check', and I 
saw many make processes and many sh processes running at once.  [But I'm 
currently testing a self-built make 4.4.0.91.]

Ken

Re: Heads up: Problems with parallel make

[Marco Atzeri]

On 20.02.2023 18:14, Ken Brown via Cygwin-apps wrote:
> On 2/20/2023 12:00 PM, Achim Gratz via Cygwin-apps wrote:
>> Ken Brown via Cygwin-apps writes:
>>> Thanks, Marco.  As expected, that fixes the problem for my test case
>>> (building TeX Live).  Obviously it would be better if the make
>>> developers would provide a configure option to use a pipe for the
>>> jobserver, but this is a good workaround if they don't.
>>
>> Does it actually do a parallel build?  I've quickly tested a cmake based
>> project with it and it completely serializes the build.  Reverting to
>> 4.4 resolves that problem.

I noticed the same with another cmake build

make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent 
make rule.

but I do not see the message on Autoconf build

> 
> It does for my build of TeX Live.  I just tried 'make -j13 check', and I 
> saw many make processes and many sh processes running at once.  [But I'm 
> currently testing a self-built make 4.4.0.91.]
> 
> Ken

I will build it and test again

Regards
Marco

Re: Heads up: Problems with parallel make

[Marco Atzeri]

On 20.02.2023 18:34, Marco Atzeri wrote:
> On 20.02.2023 18:14, Ken Brown via Cygwin-apps wrote:
>> On 2/20/2023 12:00 PM, Achim Gratz via Cygwin-apps wrote:
>>> Ken Brown via Cygwin-apps writes:
>>>> Thanks, Marco.  As expected, that fixes the problem for my test case
>>>> (building TeX Live).  Obviously it would be better if the make
>>>> developers would provide a configure option to use a pipe for the
>>>> jobserver, but this is a good workaround if they don't.
>>>
>>> Does it actually do a parallel build?  I've quickly tested a cmake based
>>> project with it and it completely serializes the build.  Reverting to
>>> 4.4 resolves that problem.
> 

uploaded as test 4.4.0.91-1


following message is still present while missing in 4.4-1

> make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent 
> make rule.
> 

and I do not see clear evidence of parallelism like in 4.4-1

>>
>> It does for my build of TeX Live.  I just tried 'make -j13 check', and 
>> I saw many make processes and many sh processes running at once.  [But 
>> I'm currently testing a self-built make 4.4.0.91.]
>>
>> Ken
> 

How are you configuring your build ?

Re: Heads up: Problems with parallel make

[Ken Brown]

On 2/20/2023 2:12 PM, Marco Atzeri via Cygwin-apps wrote:
> 
> 
> On 20.02.2023 18:34, Marco Atzeri wrote:
>> On 20.02.2023 18:14, Ken Brown via Cygwin-apps wrote:
>>> On 2/20/2023 12:00 PM, Achim Gratz via Cygwin-apps wrote:
>>>> Ken Brown via Cygwin-apps writes:
>>>>> Thanks, Marco.  As expected, that fixes the problem for my test case
>>>>> (building TeX Live).  Obviously it would be better if the make
>>>>> developers would provide a configure option to use a pipe for the
>>>>> jobserver, but this is a good workaround if they don't.
>>>>
>>>> Does it actually do a parallel build?  I've quickly tested a cmake 
>>>> based
>>>> project with it and it completely serializes the build.  Reverting to
>>>> 4.4 resolves that problem.
>>
> 
> uploaded as test 4.4.0.91-1
> 
> 
> following message is still present while missing in 4.4-1
> 
>> make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent 
>> make rule.
>>
> 
> and I do not see clear evidence of parallelism like in 4.4-1
> 
>>>
>>> It does for my build of TeX Live.  I just tried 'make -j13 check', 
>>> and I saw many make processes and many sh processes running at once.  
>>> [But I'm currently testing a self-built make 4.4.0.91.]
>>>
>>> Ken
>>
> 
> How are you configuring your build ?

Just a plain vanilla build, no configure options.

Ken

Re: Heads up: Problems with parallel make

[Ken Brown]

On 2/20/2023 2:12 PM, Marco Atzeri via Cygwin-apps wrote:
> uploaded as test 4.4.0.91-1
> 
> 
> following message is still present while missing in 4.4-1
> 
>> make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent 
>> make rule.
>>
> 
> and I do not see clear evidence of parallelism like in 4.4-1

I doubt if this is related to the pipe vs. fifo issue, but maybe you 
should test 4.4.0.90-1 to be sure.

Do you have a simple way to reproduce the problem?  For example, does it 
happen if you just make a toy cmake project?

Ken

Re: Heads up: Problems with parallel make

[Ken Brown]

On 2/20/2023 5:00 PM, Ken Brown via Cygwin-apps wrote:
> On 2/20/2023 2:12 PM, Marco Atzeri via Cygwin-apps wrote:
>> uploaded as test 4.4.0.91-1
>>
>>
>> following message is still present while missing in 4.4-1
>>
>>> make[1]: warning: jobserver unavailable: using -j1.  Add '+' to 
>>> parent make rule.
>>>
>>
>> and I do not see clear evidence of parallelism like in 4.4-1
> 
> I doubt if this is related to the pipe vs. fifo issue, but maybe you 
> should test 4.4.0.90-1 to be sure.
> 
> Do you have a simple way to reproduce the problem?  For example, does it 
> happen if you just make a toy cmake project?

Ignore what I said about 'make -j13 check' working right in the TeX Live 
build.  I just retried it and saw the "jobserver unavailable" message. 
When I saw several makes running at once, I think it was because of 
recursion, not parallelism.  So there's a real problem that can be 
reproduced without cmake.

Ken

Re: Heads up: Problems with parallel make

[Ken Brown]

On 2/20/2023 6:29 PM, Ken Brown via Cygwin-apps wrote:
> On 2/20/2023 5:00 PM, Ken Brown via Cygwin-apps wrote:
>> On 2/20/2023 2:12 PM, Marco Atzeri via Cygwin-apps wrote:
>>> uploaded as test 4.4.0.91-1
>>>
>>>
>>> following message is still present while missing in 4.4-1
>>>
>>>> make[1]: warning: jobserver unavailable: using -j1.  Add '+' to 
>>>> parent make rule.
>>>>
>>>
>>> and I do not see clear evidence of parallelism like in 4.4-1
>>
>> I doubt if this is related to the pipe vs. fifo issue, but maybe you 
>> should test 4.4.0.90-1 to be sure.
>>
>> Do you have a simple way to reproduce the problem?  For example, does 
>> it happen if you just make a toy cmake project?
> 
> Ignore what I said about 'make -j13 check' working right in the TeX Live 
> build.  I just retried it and saw the "jobserver unavailable" message. 
> When I saw several makes running at once, I think it was because of 
> recursion, not parallelism.  So there's a real problem that can be 
> reproduced without cmake.

I'm going to report this upstream, as a continuation of the upstream bug 
report that I filed earlier.

Ken

Re: Heads up: Problems with parallel make

[Brian Inglis]

On 2023-02-20 17:48, Ken Brown via Cygwin-apps wrote:
> On 2/20/2023 6:29 PM, Ken Brown via Cygwin-apps wrote:
>> On 2/20/2023 5:00 PM, Ken Brown via Cygwin-apps wrote:
>>> On 2/20/2023 2:12 PM, Marco Atzeri via Cygwin-apps wrote:
>>>> uploaded as test 4.4.0.91-1
>>>> following message is still present while missing in 4.4-1
>>>>> make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make 
>>>>> rule.
>>>> and I do not see clear evidence of parallelism like in 4.4-1

>>> I doubt if this is related to the pipe vs. fifo issue, but maybe you should 
>>> test 4.4.0.90-1 to be sure.
>>> Do you have a simple way to reproduce the problem?  For example, does it 
>>> happen if you just make a toy cmake project?

>> Ignore what I said about 'make -j13 check' working right in the TeX Live 
>> build.  I just retried it and saw the "jobserver unavailable" message. When I 
>> saw several makes running at once, I think it was because of recursion, not 
>> parallelism.  So there's a real problem that can be reproduced without cmake.

> I'm going to report this upstream, as a continuation of the upstream bug report 
> that I filed earlier.

Looks like the 4.4.1 release disables FIFO on Hurd and Cygwin.
Will be interesting to see if it is based on config test, for clues to what it 
does not like, or just system?

-- 
Take care. Thanks, Brian Inglis              Calgary, Alberta, Canada

La perfection est atteinte                   Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter  not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer     but when there is no more to cut
                                 -- Antoine de Saint-Exupéry

Re: Heads up: Problems with parallel make

[Ken Brown]

On 2/27/2023 4:41 PM, Brian Inglis via Cygwin-apps wrote:
> Looks like the 4.4.1 release disables FIFO on Hurd and Cygwin.
> Will be interesting to see if it is based on config test, for clues to 
> what it does not like, or just system?

In the case of HURD, it's because of failing tests in the test suite. 
In the case of Cygwin, it's because of the problem I reported, in which 
parallel make using a FIFO sometimes hangs when building TeX Live.

Anyone who wants to test this with their own builds can build make with

   CPPFLAGS=-DJOBSERVER_USE_FIFO=1

That will re-enable the use a FIFO for the jobserver.  One can then 
force make to use a pipe on a given project by using the make option

   --jobserver-style=pipe

I plan to do this myself, since I hope to debug the FIFO problem the 
next time I see it.  It's not easy to catch, because it doesn't happen 
with every build of TeX Live.  And when it does happen, the build has 
typically been going on for close to two hours before it hangs.

P.S. Marco, I don't know how much trouble you want to go to on this, but 
when you release make-4.4.1, you might also make a test release with 
CPPFLAGS set as above.  That would make it easier for adventurous people 
to try builds that use a FIFO.

Re: Heads up: Problems with parallel make

[Marco Atzeri]

On 27.02.2023 23:45, Ken Brown via Cygwin-apps wrote:
> On 2/27/2023 4:41 PM, Brian Inglis via Cygwin-apps wrote:
>> Looks like the 4.4.1 release disables FIFO on Hurd and Cygwin.
>> Will be interesting to see if it is based on config test, for clues to 
>> what it does not like, or just system?
> 
> In the case of HURD, it's because of failing tests in the test suite. In 
> the case of Cygwin, it's because of the problem I reported, in which 
> parallel make using a FIFO sometimes hangs when building TeX Live.
> 
> Anyone who wants to test this with their own builds can build make with
> 
>    CPPFLAGS=-DJOBSERVER_USE_FIFO=1
> 
> That will re-enable the use a FIFO for the jobserver.  One can then 
> force make to use a pipe on a given project by using the make option
> 
>    --jobserver-style=pipe
> 
> I plan to do this myself, since I hope to debug the FIFO problem the 
> next time I see it.  It's not easy to catch, because it doesn't happen 
> with every build of TeX Live.  And when it does happen, the build has 
> typically been going on for close to two hours before it hangs.
> but it could be 
> P.S. Marco, I don't know how much trouble you want to go to on this, but 
> when you release make-4.4.1, you might also make a test release with 
> CPPFLAGS set as above.  That would make it easier for adventurous people 
> to try builds that use a FIFO.

uploaded 2 versions

4.4.1-1 default is built with CPPFLAGS=-DJOBSERVER_USE_FIFO=1
so it should behave like 4.4-1 previous build

4.4.1-2 test is vanilla build, so it should be built with pipe

the build logs are not very clear, so I need to dig more on them.

 From my tests building qpdf, both are paralleling and maybe the
-2 is slight faster, but it could be due on machine load


real    19m38.493s
user    106m36.519s
sys     7m58.323s

real    18m51.680s
user    105m14.277s
sys     7m12.061s


Regards
Marco

Re: Heads up: Problems with parallel make

[Brian Inglis]

On 2023-02-20 10:14, Ken Brown via Cygwin-apps wrote:
> On 2/20/2023 12:00 PM, Achim Gratz via Cygwin-apps wrote:
>> Ken Brown via Cygwin-apps writes:
>>> Thanks, Marco.  As expected, that fixes the problem for my test case
>>> (building TeX Live).  Obviously it would be better if the make
>>> developers would provide a configure option to use a pipe for the
>>> jobserver, but this is a good workaround if they don't.
>>
>> Does it actually do a parallel build?  I've quickly tested a cmake based
>> project with it and it completely serializes the build.  Reverting to
>> 4.4 resolves that problem.
> 
> It does for my build of TeX Live.  I just tried 'make -j13 check', and I saw 
> many make processes and many sh processes running at once.  [But I'm currently 
> testing a self-built make 4.4.0.91.]

Definitely parallel jobs visible in top with make 4.4.0.90-1

-- 
Take care. Thanks, Brian Inglis			Calgary, Alberta, Canada

La perfection est atteinte			Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter	not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer	but when there is no more to cut
			-- Antoine de Saint-Exupéry

Re: Heads up: Problems with parallel make

[Ken Brown]

On 2/13/2023 6:05 PM, Ken Brown via Cygwin-apps wrote:
> Several of my packages that used to build fine with parallel make now 
> require 'make -j1'.  Without this I either get strange errors or a hang.
> 
> I can't find any pattern to explain when the problem occurs and when it 
> doesn't, but it seems to be due to a change in make, starting with 
> version 4.4.  By default, make now uses a FIFO rather than a pipe in its 
> jobserver implementation.  You can override this with the make flag
> 
>    --jobserver-style=pipe
> 
> In all cases where I've tried this, parallel make works again.
> 
> It wouldn't surprise me if there's a bug in Cygwin's FIFO implementation 
> that's responsible for this, but I haven't yet tried to track it down.

I've now reported this upstream:

   https://lists.gnu.org/archive/html/bug-make/2023-02/msg00029.html

Ken