proposal for automatic dependency tracking

proposal for automatic dependency tracking

[Tom Tromey]

After tracking down a few missing dependencies, I decided to resurrect
an old automatic dependency tracking patch I had lying around.

Before finishing it, I wanted to see if there were any objections to
the approach.

This patch adopts the Automake approach, which is to compute
dependencies as a side effect of compilation.  In the normal case,
where gcc is being built with itself, this adds no overhead to the
build.

One weakness of the Automake scheme is that it doesn't handle
generated header files very gracefully.  The problem is that if a
compilation requires a generated header, then you need an explicit
dependency (because otherwise the header won't exist when the
compilation is run).  But, the whole goal is not to have exist
dependencies...

To work around this, we use a recursive make invocation.  First we
build the generated headers, then we go ahead and build xgcc.

The patch below is a rough draft of this -- enough to see how it will
work.  I only converted java/Make-lang.in to the new scheme.  As you
can see, this part consists mostly of deletions.

Initially I'd only convert the objects that go into xgcc.  Build tools
would still require explicit dependencies.  (This is fixable, though.)

Please comment.  Thanks.

Tom

Index: Makefile.in
===================================================================
--- Makefile.in	(revision 129967)
+++ Makefile.in	(working copy)
@@ -263,6 +263,11 @@
 # UNSORTED
 # --------
 
+# Dependency tracking stuff.
+DEPMODE = @CCDEPMODE@
+DEPDIR = @DEPDIR@
+depcomp = $(SHELL) $(srcdir)/../depcomp
+
 # Some compilers can't handle cc -c blah.c -o foo/blah.o.
 # In stage2 and beyond, we force this to "-o $@" since we know we're using gcc.
 OUTPUT_OPTION = @OUTPUT_OPTION@
@@ -725,6 +730,15 @@
 GTM_H     = tm.h      $(tm_file_list)
 TM_H      = $(GTM_H) insn-constants.h insn-flags.h options.h
 
+# All generated headers which might be needed while building xgcc
+# itself.  Generated headers needed by build tools and the like should
+# not be mentioned here.
+XGCC_GEN_HEADERS = config.h tm.h tm_p.h multilib.h options.h \
+	$(simple_generated_h) $(simple_generated_c) \
+	tree-check.h genrtl.h insn-modes.h tm-preds.h tm-constrs.h \
+	$(ALL_GTFILES_H) gtype-desc.c gtype-desc.h
+
+
 # Variables for version information.
 BASEVER     := $(srcdir)/BASE-VER  # 4.x.y
 DEVPHASE    := $(srcdir)/DEV-PHASE # experimental, prerelease, ""
@@ -898,9 +912,20 @@
 	   -I$(srcdir)/../include @INCINTL@ \
 	   $(CPPINC) $(GMPINC) $(DECNUMINC)
 
+COMPILE.base = $(CC) -c $(ALL_CFLAGS) $(ALL_CPPFLAGS)
+ifeq ($(DEPMODE),depmode=gcc3)
+COMPILE = $(COMPILE.base) -o $@ -MT $@ -MMD -MP -MF $(DEPDIR)/$*.Po
+else
+COMPILE = source='$<' object='$@' libtool=no \
+    DEPDIR=$(DEPDIR) $(DEPMODE) $(depcomp) $(COMPILE.base)
+endif
+
 .c.o:
-	$(CC) -c $(ALL_CFLAGS) $(ALL_CPPFLAGS) $< $(OUTPUT_OPTION)
+	$(COMPILE) $<
 
+# Include the auto-generated dependencies for all host objects.
+-include $(patsubst %.o, $(DEPDIR)/%.Po, $(ALL_HOST_OBJS))
+
 #\f
 # Support for additional languages (other than C).
 # C can be supported this way too (leave for later).
@@ -1248,6 +1273,14 @@
 
 OBJS-onestep = libbackend.o $(OBJS-archive)
 
+# This lists all host object files, whether they are included in this
+# compilation or not.  This is used for dependency tracking.
+# fixme: cxx_target_objs
+# $(EXTRA_GCC_OBJS) ?
+ALL_HOST_OBJS = $(GCC_OBJS) $(C_OBJS) $(OBJS) libbackend.o \
+  @TREEBROWSER@ main.o gccspec.o version.o intl.o prefix.o cppspec.o \
+  $(foreach v,$(CONFIG_LANGUAGES),$($(v)_OBJS))
+
 BACKEND = main.o @TREEBROWSER@ libbackend.a $(CPPLIB) $(LIBDECNUMBER)
 
 MOSTLYCLEANFILES = insn-flags.h insn-config.h insn-codes.h \
@@ -1492,7 +1525,7 @@
 all.cross: native gcc-cross$(exeext) cpp$(exeext) specs \
 	libgcc-support lang.all.cross doc @GENINSRC@ srcextra
 # This is what must be made before installing GCC and converting libraries.
-start.encap: native xgcc$(exeext) cpp$(exeext) specs \
+start.encap: native prebuild-xgcc specs \
 	libgcc-support lang.start.encap @GENINSRC@ srcextra
 # These can't be made until after GCC can run.
 rest.encap: $(STMP_FIXPROTO) lang.rest.encap
@@ -1501,6 +1534,12 @@
 native: config.status auto-host.h build-@POSUB@ $(LANGUAGES) \
 	$(EXTRA_PASSES) $(EXTRA_PROGRAMS) $(COLLECT2)
 
+# First build things that must exist before we can try to build xgcc.
+# This is basically generated source files.  We do this to avoid
+# having hand-maintained dependencies for object files.
+prebuild-xgcc: $(XGCC_GEN_HEADERS)
+	$(MAKE) xgcc$(exeext) cpp$(exeext)
+
 # Define the names for selecting languages in LANGUAGES.
 c: cc1$(exeext)
 
@@ -1525,7 +1564,7 @@
 # to avoid confusion if the current directory is in the path
 # and CC is `gcc'.  It is renamed to `gcc' when it is installed.
 xgcc$(exeext): $(GCC_OBJS) gccspec.o version.o intl.o prefix.o \
-   version.o $(LIBDEPS) $(EXTRA_GCC_OBJS)
+   $(LIBDEPS) $(EXTRA_GCC_OBJS)
 	$(CC) $(ALL_CFLAGS) $(LDFLAGS) -o $@ $(GCC_OBJS) gccspec.o \
 	  intl.o prefix.o version.o $(EXTRA_GCC_OBJS) $(LIBS)
 
@@ -1533,7 +1572,7 @@
 # The only difference from xgcc is that it's linked with cppspec.o
 # instead of gccspec.o.
 cpp$(exeext): $(GCC_OBJS) cppspec.o version.o intl.o prefix.o \
-   version.o $(LIBDEPS) $(EXTRA_GCC_OBJS)
+   $(LIBDEPS) $(EXTRA_GCC_OBJS)
 	$(CC) $(ALL_CFLAGS) $(LDFLAGS) -o $@ $(GCC_OBJS) cppspec.o \
 	  intl.o prefix.o version.o $(EXTRA_GCC_OBJS) $(LIBS)
 
Index: java/Make-lang.in
===================================================================
--- java/Make-lang.in	(revision 129967)
+++ java/Make-lang.in	(working copy)
@@ -77,28 +77,28 @@
 java.srcextra:
 
 # Executables built by this Makefile:
-JAVA_OBJS = java/class.o java/decl.o java/expr.o \
-  java/constants.o java/lang.o java/typeck.o java/except.o \
-  java/verify-glue.o java/verify-impl.o \
-  java/zextract.o java/jcf-io.o java/win32-host.o java/jcf-parse.o java/mangle.o \
-  java/mangle_name.o java/builtins.o java/resource.o \
-  java/jcf-depend.o \
-  java/jcf-path.o java/boehm.o java/java-gimplify.o
+JAVA_OBJS = java/class.o java/decl.o java/expr.o java/constants.o \
+    java/lang.o java/typeck.o java/except.o java/verify-glue.o \
+    java/verify-impl.o java/zextract.o java/jcf-io.o java/win32-host.o \
+    java/jcf-parse.o java/mangle.o java/mangle_name.o java/builtins.o \
+    java/resource.o java/jcf-depend.o java/jcf-path.o java/boehm.o \
+    java/java-gimplify.o
 
-JCFDUMP_OBJS = java/jcf-dump.o java/jcf-io.o java/jcf-depend.o java/jcf-path.o \
-		java/win32-host.o java/zextract.o errors.o version.o ggc-none.o intl.o
+JCFDUMP_OBJS = java/jcf-dump.o java/jcf-io.o java/jcf-depend.o \
+    java/jcf-path.o java/win32-host.o java/zextract.o errors.o version.o \
+    ggc-none.o intl.o
 
 JVGENMAIN_OBJS = java/jvgenmain.o java/mangle_name.o errors.o intl.o
 
+# Used by dependency tracking.  Sort makes the list unique.
+java_OBJS = $(sort $(JAVA_OBJS) $(JCFDUMP_OBJS) $(JVGENMAIN_OBJS))
+
 # Use strict warnings for this front end.
 java-warn = $(STRICT_WARN)
 
 # String length warnings
 jvspec.o-warn = -Wno-error
 
-# Bison-1.75 output often yields (harmless) -Wtraditional warnings
-java/parse.o-warn = -Wno-error
-
 jc1$(exeext): $(JAVA_OBJS) $(BACKEND) $(LIBDEPS) attribs.o
 	rm -f $@
 	$(CC) $(ALL_CFLAGS) $(LDFLAGS) -o $@ \
@@ -239,76 +239,16 @@
 	-mv java/*$(objext) stagefeedback/java
 
 #\f
-# .o:.h dependencies.
-JAVA_TREE_H = $(TREE_H) $(HASHTAB_H) java/java-tree.h java/java-tree.def
 
-java/jcf-dump.o: $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) $(JAVA_TREE_H) \
-  java/jcf-dump.c java/jcf-reader.c java/jcf.h java/javaop.h java/javaop.def \
-  version.h $(GGC_H) intl.h java/zipfile.h
-java/boehm.o: java/boehm.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
-  $(TREE_H) $(JAVA_TREE_H) java/parse.h toplev.h
-java/builtins.o: java/builtins.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
-  $(JAVA_TREE_H) $(GGC_H) $(FLAGS_H) $(OPTABS_H) $(EXPR_H) langhooks.h \
-  gt-java-builtins.h
-java/class.o: java/class.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
-  $(JAVA_TREE_H) $(RTL_H) java/jcf.h java/parse.h toplev.h output.h $(GGC_H) \
-  $(TARGET_H) $(FUNCTION_H) gt-java-class.h $(CGRAPH_H)
-java/constants.o: java/constants.c $(CONFIG_H) $(JAVA_TREE_H) java/jcf.h \
-  toplev.h $(SYSTEM_H) coretypes.h $(TM_H) $(GGC_H) gt-java-constants.h
-java/decl.o: java/decl.c $(CONFIG_H) $(JAVA_TREE_H) $(RTL_H) java/jcf.h \
-  toplev.h $(FLAGS_H) $(SYSTEM_H) coretypes.h $(TM_H) $(FUNCTION_H) $(EXPR_H) \
-  libfuncs.h except.h java/java-except.h $(GGC_H) $(REAL_H) gt-java-decl.h \
-  $(TARGET_H) $(CGRAPH_H) langhooks.h
-java/except.o: java/except.c $(CONFIG_H) $(JAVA_TREE_H) java/jcf.h $(REAL_H) \
-  $(RTL_H) java/javaop.h java/java-opcodes.h except.h java/java-except.h \
-  toplev.h $(SYSTEM_H) coretypes.h $(TM_H) $(FUNCTION_H) 
-java/expr.o: java/expr.c $(CONFIG_H) $(JAVA_TREE_H) java/jcf.h $(REAL_H) \
-  $(RTL_H) $(EXPR_H) java/javaop.h java/java-opcodes.h except.h \
-  java/java-except.h java/java-except.h java/parse.h toplev.h \
-  $(SYSTEM_H) coretypes.h $(TM_H) $(GGC_H) gt-java-expr.h $(TARGET_H)
-java/jcf-depend.o: java/jcf-depend.c $(CONFIG_H) $(SYSTEM_H) coretypes.h \
-  $(TM_H) java/jcf.h
-java/jcf-parse.o: java/jcf-parse.c $(CONFIG_H) $(JAVA_TREE_H) $(FLAGS_H) \
-  input.h java/java-except.h $(SYSTEM_H) coretypes.h $(TM_H) toplev.h \
-  java/parse.h $(GGC_H) debug.h $(REAL_H) gt-java-jcf-parse.h $(TM_P_H) \
-  java/jcf-reader.c java/zipfile.h java/jcf.h
-java/jvgenmain.o: java/jvgenmain.c $(CONFIG_H) $(JAVA_TREE_H) $(SYSTEM_H) \
-  coretypes.h $(TM_H) intl.h
-java/lang.o: java/lang.c $(CONFIG_H) $(JAVA_TREE_H) java/jcf.h input.h \
-  toplev.h $(SYSTEM_H) coretypes.h $(TM_H) $(RTL_H) $(EXPR_H) $(DIAGNOSTIC_H) \
-  langhooks.h $(LANGHOOKS_DEF_H) gt-java-lang.h opts.h options.h
-java/mangle.o: java/mangle.c $(CONFIG_H) java/jcf.h $(JAVA_TREE_H) $(SYSTEM_H) \
-  coretypes.h $(TM_H) toplev.h $(GGC_H) gt-java-mangle.h $(LANGHOOKS_DEF_H)
-java/mangle_name.o: java/mangle_name.c $(CONFIG_H) java/jcf.h $(JAVA_TREE_H) \
-  $(SYSTEM_H) coretypes.h $(TM_H) toplev.h $(GGC_H)
-java/resource.o: java/resource.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
-  $(JAVA_TREE_H) $(RTL_H) java/jcf.h java/parse.h toplev.h output.h $(GGC_H) \
-  $(TARGET_H) $(FUNCTION_H) gt-java-resource.h $(EXPR_H)
-java/typeck.o: java/typeck.c $(CONFIG_H) $(JAVA_TREE_H) java/jcf.h \
-  toplev.h $(SYSTEM_H) coretypes.h $(TM_H) $(GGC_H) $(REAL_H)
-java/win32-host.o: java/win32-host.c $(CONFIG_H) $(SYSTEM_H) coretypes.h java/jcf.h
-java/verify-glue.o: java/verify-glue.c $(CONFIG_H) $(SYSTEM_H) $(JAVA_TREE_H) \
-  coretypes.h $(TM_H) java/verify.h toplev.h
-java/verify-impl.o: java/verify-impl.c $(CONFIG_H) java/verify.h $(SYSTEM_H) \
-  coretypes.h  java/jcf.h $(JAVA_TREE_H)
-java/zextract.o: java/zextract.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
-  java/zipfile.h
-java/java-gimplify.o: java/java-gimplify.c $(CONFIG_H) $(SYSTEM_H) \
-  coretypes.h $(TM_H) $(JAVA_TREE_H) $(TREE_GIMPLE_H) toplev.h
-
 # jcf-io.o needs $(ZLIBINC) added to cflags.
-java/jcf-io.o: java/jcf-io.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
-  $(JAVA_TREE_H) java/zipfile.h
-	$(CC) -c $(ALL_CFLAGS) $(ALL_CPPFLAGS) $(INCLUDES) $(ZLIBINC) \
-		$(srcdir)/java/jcf-io.c $(OUTPUT_OPTION)
+java/jcf-io.o: \
+	ALL_CPPFLAGS := $(ALL_CPPFLAGS) $(ZLIBINC)
 
 # jcf-path.o needs a -D.
-java/jcf-path.o: java/jcf-path.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) \
-  java/jcf.h
-	$(CC) -c $(ALL_CFLAGS) $(ALL_CPPFLAGS) $(INCLUDES) \
+java/jcf-path.o: \
+     ALL_CPPFLAGS := $(ALL_CPPFLAGS) \
 	  -DLIBGCJ_ZIP_FILE='"$(datadir)/java/libgcj-$(version).jar"' \
-	  -DDEFAULT_TARGET_VERSION=\"$(version)\" \
-	  $(srcdir)/java/jcf-path.c $(OUTPUT_OPTION)
+	  -DDEFAULT_TARGET_VERSION=\"$(version)\"
 
 TEXI_JAVA_FILES = java/gcj.texi $(gcc_docdir)/include/fdl.texi		\
 	 $(gcc_docdir)/include/gpl.texi $(gcc_docdir)/include/gcc-common.texi   \
Index: configure.ac
===================================================================
--- configure.ac	(revision 129967)
+++ configure.ac	(working copy)
@@ -308,6 +308,12 @@
 AC_CHECK_TYPES([long long], [AC_CHECK_SIZEOF(long long)])
 AC_CHECK_TYPES([__int64], [AC_CHECK_SIZEOF(__int64)])
 
+# Dependency checking.
+ZW_CREATE_DEPDIR
+## FIXME!!
+mkdir -p .deps/cp .deps/ada .deps/java .deps/treelang .deps/fortran .deps/objc .deps/objcp
+ZW_PROG_COMPILER_DEPENDENCIES([CC])
+
 # ---------------------
 # Warnings and checking
 # ---------------------

Re: proposal for automatic dependency tracking

[Diego Novillo]

Tom Tromey wrote:

> Please comment.  Thanks.

Only one coment: Thanks!

I don't have any opinions on the code itself.  I'll leave that up to the 
build maintainers.  As a user, this looks a lot easier and less 
error-prone than the current system.  Build failures with -jN would now 
be minimized, I expect.


Diego.