[Bug other/38388] parallel install failures in install-{libiberty,gnatlib}

[Bug other/38388] parallel install failures in install-{libiberty,gnatlib}

[pluto at agmk dot net]

------- Comment #2 from pluto at agmk dot net  2008-12-03 18:34 -------
Created an attachment (id=16821)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=16821&action=view)
log


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38388

[Bug other/38388] New: parallel install failures in install-{libiberty,gnatlib}

[pluto at agmk dot net]

during -j8 install i can observe failures.


-- 
           Summary: parallel install failures in install-{libiberty,gnatlib}
           Product: gcc
           Version: 4.4.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: other
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: pluto at agmk dot net
GCC target triplet: x86_64-gnu-linux


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38388

[Bug other/38388] parallel install failures in install-{libiberty,gnatlib}

[pluto at agmk dot net]

------- Comment #1 from pluto at agmk dot net  2008-12-03 18:34 -------
Created an attachment (id=16820)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=16820&action=view)
log


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38388

[Bug other/38388] parallel install failures in install-{libiberty,gnatlib}

[pluto at agmk dot net]

------- Comment #3 from pluto at agmk dot net  2008-12-03 18:35 -------
this is a trunk/r142396


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38388

[Bug bootstrap/38388] parallel install failures in install-{libiberty,gnatlib}

[pinskia at gcc dot gnu dot org]

------- Comment #4 from pinskia at gcc dot gnu dot org  2008-12-03 18:38 -------
Hardly anybody installs using make -j.


-- 

pinskia at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|other                       |bootstrap


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38388

[Bug bootstrap/38388] parallel install failures in install-{libiberty,gnatlib}

[pluto at agmk dot net]

------- Comment #5 from pluto at agmk dot net  2010-02-28 10:41 -------


*** This bug has been marked as a duplicate of 42980 ***


-- 

pluto at agmk dot net changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|                            |DUPLICATE


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38388

[Bug bootstrap/38388] parallel install failures in install-{libiberty,gnatlib}

[rwild at gcc dot gnu dot org]

------- Comment #6 from rwild at gcc dot gnu dot org  2010-02-28 11:08 -------
Actually, except for the libiberty failure, these failures are different, and
not fixed by the patches for PR 42980.  De-duplicating and reopening.

Can you still reproduce the Ada-related failures? It looks like serialization
between toplevel install-libada and some installation below gcc/ada may be
needed.


-- 

rwild at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |UNCONFIRMED
         Resolution|DUPLICATE                   |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38388

[Bug bootstrap/38388] parallel install failures in install-{libiberty,gnatlib}

[pluto at agmk dot net]

------- Comment #7 from pluto at agmk dot net  2010-03-10 22:37 -------
(In reply to comment #6)

> Can you still reproduce the Ada-related failures? It looks like serialization
> between toplevel install-libada and some installation below gcc/ada may be
> needed.

currently i can't test install due to PR43328.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38388

[Bug bootstrap/38388] parallel install failures in install-{libiberty,gnatlib}

[rwild at gcc dot gnu dot org]

-- 

rwild at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |WAITING


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38388

[Bug bootstrap/38388] parallel install failures in install-{libiberty,gnatlib}

[pluto at agmk dot net]

------- Comment #8 from pluto at agmk dot net  2010-03-13 12:32 -------
current 4.4 branch and trunk install correctly with make -jN.


-- 

pluto at agmk dot net changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |RESOLVED
         Resolution|                            |FIXED


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38388

[Bug bootstrap/38388] parallel install failures in install-{libiberty,gnatlib}

[davek at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38388

Dave Korn <davek at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Target|x86_64-gnu-linux            |
             Status|RESOLVED                    |REOPENED
   Last reconfirmed|                            |2011-12-06
                 CC|                            |davek at gcc dot gnu.org
            Version|4.4.0                       |4.7.0
         Resolution|FIXED                       |
     Ever Confirmed|0                           |1

--- Comment #9 from Dave Korn <davek at gcc dot gnu.org> 2011-12-06 22:43:46 UTC ---
[ Reopened because the gnatlib bug is still present.  Removed target, because
it is not target-specific.  Not sure if bootstrap is still the right component
or if it should be ada, so will leave that to discretion of bug maintainers. ]

I saw this myself earlier, and asked about it on the GCC list at
http://gcc.gnu.org/ml/gcc/2011-12/msg00050.html - 

On 03/12/2011 12:16, Dave Korn wrote:

>   Running "make -j8 install" in a fresh build of head, I saw loads of the
> following error messages coming out in the log:
> 
>> cp: cannot create regular file
>> `/gnu/gcc/install.obj3/gnu/usr/lib/gcc/i686-pc-cygwin/4.7.0/adainclude/a-ztmoau.adb':
>> File exists cp: cannot create regular file
>> `/gnu/gcc/install.obj3/gnu/usr/lib/gcc/i686-pc-cygwin/4.7.0/adainclude/a-ztmoio.adb':
>> File exists cp: cannot create regular file
>> `/gnu/gcc/install.obj3/gnu/usr/lib/gcc/i686-pc-cygwin/4.7.0/adainclude/a-zttest.ads':
>> File exists cp: cannot create regular file
>> `/gnu/gcc/install.obj3/gnu/usr/lib/gcc/i686-pc-cygwin/4.7.0/adainclude/a-zzboio.ads':
>> File exists cp: cannot create regular file
>> `/gnu/gcc/install.obj3/gnu/usr/lib/gcc/i686-pc-cygwin/4.7.0/adainclude/ada.ads':
>> File exists cp: cannot create regular file
>> `/gnu/gcc/install.obj3/gnu/usr/lib/gcc/i686-pc-cygwin/4.7.0/adainclude/directio.ads':
>> File exists
>       [ ... snip ... ]
> 
>   Sure enough, all the files did exist, so I guess it must have been trying to
> install them twice.  Before I try digging deeper, I thought I'd quickly ask:
> Does anyone else see this?  (Is it perhaps something that wouldn't show up on
> a different host, such as linux, owing to differing filesystem semantics?)

  Okay, confirmed; I wonder why this isn't causing problems for anyone else?

  The issue is that there are two dependency chains that lead to the
"install-gnatlib" target in the gcc/ada/gcc-interface/Makefile.in-derived
Makefile in $objdir:

[1] top level "make install" -> $objdir/$target/libada/Makefile "install:
install-gnatlib" -> "install-gnatlib:" -> $objdir/gcc/ada/Makefile(*)
"install-gnatlib:" -> $objdir/gcc/ada/gcc-interface/Makefile
"install-gnatlib:".

[2] top level "make install" -> $objdir/gcc/Makefile "install: install-common"
-> "install-common: lang.install-common" -> "lang.install-common:
ada.install-common" -> $(srcdir)/ada/gcc-interface/Make-lang.in
"ada.install-common" -> "install-gnatlib" -> $objdir/gcc/ada/Makefile(*)
"install-gnatlib:" -> $objdir/gcc/ada/gcc-interface/Makefile
"install-gnatlib:".

  The two paths merge at (*), and so a parallel top-level make can end up
running both of them at the same time.  That gets broken, because the
install-gnatlib target begins by wiping and recreating the install dirs:

> install-gnatlib: ../stamp-gnatlib-$(RTSDIR)
> #	Create the directory before deleting it, in case the directory is
> #	a list of directories (as it may be on VMS). This ensures we are
> #	deleting the right one.
> 	-$(MKDIR) $(DESTDIR)$(ADA_RTL_OBJ_DIR)
> 	-$(MKDIR) $(DESTDIR)$(ADA_INCLUDE_DIR)
> 	$(RMDIR) $(DESTDIR)$(ADA_RTL_OBJ_DIR)
> 	$(RMDIR) $(DESTDIR)$(ADA_INCLUDE_DIR)
> 	-$(MKDIR) $(DESTDIR)$(ADA_RTL_OBJ_DIR)
> 	-$(MKDIR) $(DESTDIR)$(ADA_INCLUDE_DIR)

  Depending how out-of-sync the two separate sub-makes are, this results in an
incomplete installation.  Here's what happened in my latest test:

> make[2]: Entering directory `/gnu/gcc/obj3/i686-pc-cygwin/libada'
> make -C ../.././gcc/ada "MAKEOVERRIDES=" "LDFLAGS=" "LN_S=ln -s" "SHELL=/bin/sh" "GNATLIBFLAGS=-W -Wall -gnatpg -nostdinc " "GNATLIBCFLAGS=-g -O2 " "GNATLIBCFLAGS_FOR_C=-W -Wall -g -O2  -fexceptions -DIN_RTS -DHAVE_GETIPINFO " "PICFLAG_FOR_TARGET=" "THREAD_KIND=native" "TRACE=no" "MULTISUBDIR=" "libsubdir=/gnu/usr/lib/gcc/i686-pc-cygwin/4.7.0" "objext=.o" "prefix=/gnu/usr" "exeext=.exeext.should.not.be.used " 'CC=the.host.compiler.should.not.be.needed' "GCC_FOR_TARGET=/gnu/gcc/obj3/./gcc/xgcc -B/gnu/gcc/obj3/./gcc/ -B/gnu/usr/i686-pc-cygwin/bin/ -B/gnu/usr/i686-pc-cygwin/lib/ -isystem /gnu/usr/i686-pc-cygwin/include -isystem /gnu/usr/i686-pc-cygwin/sys-include   " "CFLAGS=-g -O2" install-gnatlib
> make[3]: Entering directory `/gnu/gcc/obj3/gcc/ada'
> mkdir -p /gnu/gcc/install.obj3/gnu/usr/lib/gcc/i686-pc-cygwin/4.7.0/adalib
> mkdir -p /gnu/gcc/install.obj3/gnu/usr/lib/gcc/i686-pc-cygwin/4.7.0/adainclude
> rm -rf /gnu/gcc/install.obj3/gnu/usr/lib/gcc/i686-pc-cygwin/4.7.0/adalib
> rm -rf /gnu/gcc/install.obj3/gnu/usr/lib/gcc/i686-pc-cygwin/4.7.0/adainclude
> mkdir -p /gnu/gcc/install.obj3/gnu/usr/lib/gcc/i686-pc-cygwin/4.7.0/adalib
> mkdir -p /gnu/gcc/install.obj3/gnu/usr/lib/gcc/i686-pc-cygwin/4.7.0/adainclude
> for file in rts/*.ali; do \
> 	    cp -p $file /gnu/gcc/install.obj3/gnu/usr/lib/gcc/i686-pc-cygwin/4.7.0/adalib; \
> 	done

  The target libada make install runs first, recurses into gcc/ada and starts
copying .ali files.  Shortly afterwards, ...

> make[4]: Entering directory `/gnu/gcc/obj3/gcc/ada'
> mkdir -p /gnu/gcc/install.obj3/gnu/usr/lib/gcc/i686-pc-cygwin/4.7.0/adalib
> mkdir -p /gnu/gcc/install.obj3/gnu/usr/lib/gcc/i686-pc-cygwin/4.7.0/adainclude

... make install running in the gcc subdir starts running the install-gnatlib
target too, ...

> rm -rf /gnu/gcc/install.obj3/gnu/usr/lib/gcc/i686-pc-cygwin/4.7.0/adalib
> rm: cannot remove `/gnu/gcc/install.obj3/gnu/usr/lib/gcc/i686-pc-cygwin/4.7.0/adalib': Directory not empty
> make[4]: *** [install-gnatlib] Error 1
> make[4]: Leaving directory `/gnu/gcc/obj3/gcc/ada'
> make[3]: *** [install-gnatlib] Error 2
> make[3]: Leaving directory `/gnu/gcc/obj3/gcc'

... and fails, but not after having deleted all the files that the target
libada make had managed to install so far.

> cd rts; for file in *.a;do \
> 	    /usr/bin/install -c -m 644 $file /gnu/gcc/install.obj3/gnu/usr/lib/gcc/i686-pc-cygwin/4.7.0/adalib; \
> 	    /gnu/usr/i686-pc-cygwin/bin/ranlib /gnu/gcc/install.obj3/gnu/usr/lib/gcc/i686-pc-cygwin/4.7.0/adalib/$file; \
> 	done

  Meanwhile the target libada install continues to run...

> if [ -f gcov.exe ]; \
> 	then \
> 	    rm -f /gnu/gcc/install.obj3/gnu/usr/bin/gcov-4.exe; \
> 	    /usr/bin/install -c gcov.exe /gnu/gcc/install.obj3/gnu/usr/bin/gcov-4.exe; \
> 	fi
> make[2]: Leaving directory `/gnu/gcc/obj3/gcc'
> make[1]: Leaving directory `/gnu/gcc/obj3'
> 
> real	7m48.235s
> user	4m48.897s
> sys	3m35.738s

... and eventually the top-level make completes - apparently without error, so
I guess the lower-level $objdir/gcc/ada make isn't propagating errors back up
properly.

  I hope a build-system maintainer can advise.  Would removing the
install-gnatlib from the gcc/ada language directory and having the files
installed only from the $target/libada directory be correct w.r.t multilibs?