* [Bug other/42980] GCC parllel "make install" failures
2010-02-05 21:46 [Bug other/42980] New: GCC parllel "make install" failures gary at intrepid dot com
@ 2010-02-05 22:38 ` pinskia at gcc dot gnu dot org
2010-02-05 23:03 ` gary at intrepid dot com
` (11 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: pinskia at gcc dot gnu dot org @ 2010-02-05 22:38 UTC (permalink / raw)
To: gcc-bugs
------- Comment #1 from pinskia at gcc dot gnu dot org 2010-02-05 22:37 -------
Related to PR 33119. I don't many people who install with -j.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42980
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug other/42980] GCC parllel "make install" failures
2010-02-05 21:46 [Bug other/42980] New: GCC parllel "make install" failures gary at intrepid dot com
2010-02-05 22:38 ` [Bug other/42980] " pinskia at gcc dot gnu dot org
@ 2010-02-05 23:03 ` gary at intrepid dot com
2010-02-08 18:18 ` [Bug other/42980] GCC parallel " rwild at gcc dot gnu dot org
` (10 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: gary at intrepid dot com @ 2010-02-05 23:03 UTC (permalink / raw)
To: gcc-bugs
------- Comment #2 from gary at intrepid dot com 2010-02-05 23:03 -------
(In reply to comment #1)
> Related to PR 33119. I don't many people who install with -j.
>
It resulted from "make install" being invoked from a Makefile, where
the overall make was run in parallel, and then a last step did this:
$(MAKE) install
It inherited the -j setting from the top-level make invocation.
I checked the install docs, and didn't see where it warned away from
doing a parallel make install, fyi.
The automake aspect of this bug seems like a bug in the way
automake handles certain install rules in a multi-lib setting. Certain
rules should only be run once, and not re-run in the multi-lib'd directories.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42980
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug other/42980] GCC parallel "make install" failures
2010-02-05 21:46 [Bug other/42980] New: GCC parllel "make install" failures gary at intrepid dot com
2010-02-05 22:38 ` [Bug other/42980] " pinskia at gcc dot gnu dot org
2010-02-05 23:03 ` gary at intrepid dot com
@ 2010-02-08 18:18 ` rwild at gcc dot gnu dot org
2010-02-08 23:11 ` gary at intrepid dot com
` (9 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: rwild at gcc dot gnu dot org @ 2010-02-08 18:18 UTC (permalink / raw)
To: gcc-bugs
------- Comment #3 from rwild at gcc dot gnu dot org 2010-02-08 18:18 -------
Can you make a bit more of the output of such a failed install available,
say, about 50 lines around each of the two different failures?
Wrt. the failure with headers, it seems GNU coreutils install does not
allow concurrent installs of the same file. I wasn't aware of this
limitation, will ask on bug-coreutils whether that is on purpose.
And no, the two issues mentioned in this bug do not look like they are the
same as in PR 33119.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42980
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug other/42980] GCC parallel "make install" failures
2010-02-05 21:46 [Bug other/42980] New: GCC parllel "make install" failures gary at intrepid dot com
` (2 preceding siblings ...)
2010-02-08 18:18 ` [Bug other/42980] GCC parallel " rwild at gcc dot gnu dot org
@ 2010-02-08 23:11 ` gary at intrepid dot com
2010-02-21 16:13 ` rwild at gcc dot gnu dot org
` (8 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: gary at intrepid dot com @ 2010-02-08 23:11 UTC (permalink / raw)
To: gcc-bugs
------- Comment #4 from gary at intrepid dot com 2010-02-08 23:11 -------
Created an attachment (id=19827)
--> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=19827&action=view)
Log excerpts of parallel make install failures
Excerpts from parallel install (ie, "make -j6 install") failures. Each file is
separated by "=== <file> ===" strings. Search for the word "cannot" to find
the point of failure. These are mainly libiberty failures, with a single
libssp failure. We saw a similar libgomp failure to the libssp failure, but
apparently didn't save the log file. The problem can be replicated by creating
a loop that repeatedly runs "make -j6 install", and that saves the log when a
failure is detected. The -j6 is arbitrary - this turns out to be the "sweet
spot" for parallel makes on our 4 core system.
Something like this:
#!/bin/sh
dest=`pwd`/rls
for i in `seq 1 1000`; do
for j in 6; do
echo "$i: make -j$j install"
rm -rf $dest
logfile="install-${i}-${j}.log"
make -j$j -C bld DESTDIR=$dest install >& $logfile
if [ $? -ne 0 ]; then
echo "$i: make -j$j install failed."
else
rm -f $logfile
fi
done
done
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42980
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug other/42980] GCC parallel "make install" failures
2010-02-05 21:46 [Bug other/42980] New: GCC parllel "make install" failures gary at intrepid dot com
` (3 preceding siblings ...)
2010-02-08 23:11 ` gary at intrepid dot com
@ 2010-02-21 16:13 ` rwild at gcc dot gnu dot org
2010-02-22 17:48 ` gary at intrepid dot com
` (7 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: rwild at gcc dot gnu dot org @ 2010-02-21 16:13 UTC (permalink / raw)
To: gcc-bugs
------- Comment #5 from rwild at gcc dot gnu dot org 2010-02-21 16:13 -------
Thanks for the logs. I don't understand the libiberty installation failure
yet. Can you please run the following, and provide the log file and the number
of runs needed, in case it provokes a failure?
while make -j install-target-libiberty SHELL="/bin/sh -x" > log-file 2>&1; do
echo $((n++))
done
Otherwise, you might need to just interrupt this after a while. Thanks.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42980
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug other/42980] GCC parallel "make install" failures
2010-02-05 21:46 [Bug other/42980] New: GCC parllel "make install" failures gary at intrepid dot com
` (4 preceding siblings ...)
2010-02-21 16:13 ` rwild at gcc dot gnu dot org
@ 2010-02-22 17:48 ` gary at intrepid dot com
2010-02-24 22:15 ` rwild at gcc dot gnu dot org
` (6 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: gary at intrepid dot com @ 2010-02-22 17:48 UTC (permalink / raw)
To: gcc-bugs
------- Comment #6 from gary at intrepid dot com 2010-02-22 17:48 -------
Created an attachment (id=19936)
--> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=19936&action=view)
parallel "make -j6 install" failure logs
Attached, a collection of install logs run with SHELL="/bin/sh -x", where the
install failed. The first log is for a libgomp install failure, and the rest
are for libiberty. Note that it is difficult to reproduce the failures. I
found that "make -j6" led to more failures than "make -j" on our system. Also,
couldn't get the install-target-libiberty target to fail, but perhaps didn't
wait long enough. Instead, ran "make -j6 install". Also, tried running this
test on a "ram fs", but couldn't make the installs fail, likely due to
different interlocks. Similarly, the installs were more likely to fail on our
RAID-ed disk subsystem, populated with high speed drives, than on a file system
on a single drive. (These tests were run using the gcc-4.5-20100114 (gcc core)
snapshot.)
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42980
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug other/42980] GCC parallel "make install" failures
2010-02-05 21:46 [Bug other/42980] New: GCC parllel "make install" failures gary at intrepid dot com
` (5 preceding siblings ...)
2010-02-22 17:48 ` gary at intrepid dot com
@ 2010-02-24 22:15 ` rwild at gcc dot gnu dot org
2010-02-24 22:17 ` rwild at gcc dot gnu dot org
` (5 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: rwild at gcc dot gnu dot org @ 2010-02-24 22:15 UTC (permalink / raw)
To: gcc-bugs
------- Comment #7 from rwild at gcc dot gnu dot org 2010-02-24 22:15 -------
Thank you very much for the logs, and the note that install-target-libiberty
alone wasn't sufficient to provoke a race, that provided the needed clue.
Please try the following three patches (split up because they're likely to go
into the tree at different times) to ensure there are no more likely issues
left. Thanks.
--
rwild at gcc dot gnu dot org changed:
What |Removed |Added
----------------------------------------------------------------------------
AssignedTo|unassigned at gcc dot gnu |rwild at gcc dot gnu dot org
|dot org |
Status|UNCONFIRMED |ASSIGNED
Ever Confirmed|0 |1
Last reconfirmed|0000-00-00 00:00:00 |2010-02-24 22:15:05
date| |
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42980
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug other/42980] GCC parallel "make install" failures
2010-02-05 21:46 [Bug other/42980] New: GCC parllel "make install" failures gary at intrepid dot com
` (6 preceding siblings ...)
2010-02-24 22:15 ` rwild at gcc dot gnu dot org
@ 2010-02-24 22:17 ` rwild at gcc dot gnu dot org
2010-02-24 22:18 ` rwild at gcc dot gnu dot org
` (4 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: rwild at gcc dot gnu dot org @ 2010-02-24 22:17 UTC (permalink / raw)
To: gcc-bugs
------- Comment #8 from rwild at gcc dot gnu dot org 2010-02-24 22:17 -------
Created an attachment (id=19953)
--> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=19953&action=view)
properly propagate (parallel) make flags in libgcc install rule
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42980
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug other/42980] GCC parallel "make install" failures
2010-02-05 21:46 [Bug other/42980] New: GCC parllel "make install" failures gary at intrepid dot com
` (7 preceding siblings ...)
2010-02-24 22:17 ` rwild at gcc dot gnu dot org
@ 2010-02-24 22:18 ` rwild at gcc dot gnu dot org
2010-02-24 22:22 ` rwild at gcc dot gnu dot org
` (3 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: rwild at gcc dot gnu dot org @ 2010-02-24 22:18 UTC (permalink / raw)
To: gcc-bugs
------- Comment #9 from rwild at gcc dot gnu dot org 2010-02-24 22:18 -------
Created an attachment (id=19954)
--> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=19954&action=view)
Fix parallel libiberty install failure
This patch should fix the bulk of the failures, and is hopefully simple enough
to go in before 4.6.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42980
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug other/42980] GCC parallel "make install" failures
2010-02-05 21:46 [Bug other/42980] New: GCC parllel "make install" failures gary at intrepid dot com
` (8 preceding siblings ...)
2010-02-24 22:18 ` rwild at gcc dot gnu dot org
@ 2010-02-24 22:22 ` rwild at gcc dot gnu dot org
2010-02-28 9:57 ` rwild at gcc dot gnu dot org
` (2 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: rwild at gcc dot gnu dot org @ 2010-02-24 22:22 UTC (permalink / raw)
To: gcc-bugs
------- Comment #10 from rwild at gcc dot gnu dot org 2010-02-24 22:22 -------
Created an attachment (id=19955)
--> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=19955&action=view)
Changes proposed for automake-generated files (pending upstream fix)
This patch is just a diff against files generated by automake, more precisely,
a fix that hasn't gone upstream yet (discussion to follow on gcc-patches).
After you apply this, due to PR 43171, you either need to start a new build
tree or remove x86_64-unknown-linux-gnu/*/Makefile and
x86_64-unknown-linux-gnu/32/*/Makefile before rerunning make.
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42980
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug other/42980] GCC parallel "make install" failures
2010-02-05 21:46 [Bug other/42980] New: GCC parllel "make install" failures gary at intrepid dot com
` (9 preceding siblings ...)
2010-02-24 22:22 ` rwild at gcc dot gnu dot org
@ 2010-02-28 9:57 ` rwild at gcc dot gnu dot org
2010-02-28 10:41 ` pluto at agmk dot net
2010-03-02 6:10 ` rwild at gcc dot gnu dot org
12 siblings, 0 replies; 14+ messages in thread
From: rwild at gcc dot gnu dot org @ 2010-02-28 9:57 UTC (permalink / raw)
To: gcc-bugs
------- Comment #11 from rwild at gcc dot gnu dot org 2010-02-28 09:57 -------
Patches posted at <http://gcc.gnu.org/ml/gcc-patches/2010-02/msg01236.html>
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42980
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug other/42980] GCC parallel "make install" failures
2010-02-05 21:46 [Bug other/42980] New: GCC parllel "make install" failures gary at intrepid dot com
` (10 preceding siblings ...)
2010-02-28 9:57 ` rwild at gcc dot gnu dot org
@ 2010-02-28 10:41 ` pluto at agmk dot net
2010-03-02 6:10 ` rwild at gcc dot gnu dot org
12 siblings, 0 replies; 14+ messages in thread
From: pluto at agmk dot net @ 2010-02-28 10:41 UTC (permalink / raw)
To: gcc-bugs
------- Comment #12 from pluto at agmk dot net 2010-02-28 10:41 -------
*** Bug 38388 has been marked as a duplicate of this bug. ***
--
pluto at agmk dot net changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |pluto at agmk dot net
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42980
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Bug other/42980] GCC parallel "make install" failures
2010-02-05 21:46 [Bug other/42980] New: GCC parllel "make install" failures gary at intrepid dot com
` (11 preceding siblings ...)
2010-02-28 10:41 ` pluto at agmk dot net
@ 2010-03-02 6:10 ` rwild at gcc dot gnu dot org
12 siblings, 0 replies; 14+ messages in thread
From: rwild at gcc dot gnu dot org @ 2010-03-02 6:10 UTC (permalink / raw)
To: gcc-bugs
------- Comment #13 from rwild at gcc dot gnu dot org 2010-03-02 06:10 -------
Subject: Bug 42980
Author: rwild
Date: Tue Mar 2 06:09:56 2010
New Revision: 157159
URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=157159
Log:
Small multilib rule fixups.
libgcc/:
PR other/42980
* Makefile.in (install): Use $(MAKE) string in rule, for
parallel make.
libiberty/:
* Makefile.in (all): Do not use exec.
Modified:
trunk/libgcc/ChangeLog
trunk/libgcc/Makefile.in
trunk/libiberty/ChangeLog
trunk/libiberty/Makefile.in
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42980
^ permalink raw reply [flat|nested] 14+ messages in thread