Re: Buildbot failure in Wildebeest Builder on whole buildset

[Mark Wielaard <mark@klomp.org>]

[-- Attachment #1: Type: text/plain, Size: 1090 bytes --]

Hi,

On Wed, 2021-05-05 at 20:03 +0000, buildbot@builder.wildebeest.org wrote:
> The Buildbot has detected a failed build on builder whole buildset while building debugedit.
> Full details are available at:
>     https://builder.wildebeest.org/buildbot/#builders/53/builds/14
> 
> Buildbot URL: https://builder.wildebeest.org/buildbot/
> 
> Worker for this Build: centos-x86_64
> 
> Build Reason: <unknown>
> Blamelist: Mark Wielaard <mark@klomp.org>
> 
> BUILD FAILED: failed compile (failure)

This was an interesting failure. Turns out that we were too clever with
our help2man dependencies. It might fail when we do a parallel build
because our recursive make invocation might clash with the running
parallel make one. The attached patch fixes it by a trick found in
guix. Now we do depend on the executables, but only invoke the help2man
rule if the dependency that changed wasn't the executable itself.

See attached.

Cheers,

Mark

P.S. Yes, there are still 3 buildbot workers that don't have help2man,
they are still failing. Will be fixed tomorrow.

[-- Attachment #2: 0001-Makefile.am-Don-t-try-to-recursively-make-binaries-t.patch --]
[-- Type: text/x-patch, Size: 3060 bytes --]

From 4bb332c7a8e53f9e2ee7aefb5a33d19aeed482cf Mon Sep 17 00:00:00 2001
From: Mark Wielaard <mark@klomp.org>
Date: Wed, 5 May 2021 23:33:14 +0200
Subject: [PATCH] Makefile.am: Don't try to recursively make binaries to run
 help2man

We try to avoid having to run help2man unnecessary for a build from
a source distribution (which include the generated manpages). We did
this by not depending on the actual executable, but recursively calling
make to generate it when necessary. This causes trouble with parallel
makes because the two make processes don't know which objects have,
have not or are being build. Fix this by depending on the executable
but only running help2man if one of the sources has changed, not when
the executable has been regenerated.

Signed-off-by: Mark Wielaard <mark@klomp.org>
---
 Makefile.am | 37 ++++++++++++++++++++++---------------
 1 file changed, 22 insertions(+), 15 deletions(-)

diff --git a/Makefile.am b/Makefile.am
index ef972e8..584e532 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -41,23 +41,30 @@ sepdebugcrcfix_LDADD = @LIBELF_LIBS@
 # Manual pages are generated for dist
 dist_man_MANS = debugedit.1 sepdebugcrcfix.1 find-debuginfo.sh.1
 
-# Common dependency for man pages. Man pages are only regenerated when
-# their main source is newer, or configure.ac (version) is updated.
-TOP_CONFIGURE_AC = $(top_srcdir)/configure.ac
+# The 'case' ensures the man pages are only generated if the corresponding
+# source script (the first prerequisite) or configure.ac (for the version)
+# has been changed.  The executable prerequisite is solely meant to force
+# these docs to be made only after the executable has been compiled.
+# This makes sure help2man is not normally necessary (since the generated
+# man pages are distributed).
+debugedit.1: tools/debugedit.c configure.ac debugedit$(EXEEXT)
+	@case '$?' in \
+	  *$<* | *configure.ac* ) $(HELP2MAN) -N --output=$@ \
+		--name='debug source path manipulation tool' \
+		./debugedit$(EXEEXT) ;; \
+	  * ) : ;; \
+	esac
 
-debugedit.1: tools/debugedit.c $(TOP_CONFIGURE_AC)
-	$(MAKE) $(AM_MAKEFLAGS) debugedit$(EXEEXT)
-	$(HELP2MAN) -N --output=$@ \
-	  --name='debug source path manipulation tool' \
-	  ./debugedit$(EXEEXT)
-
-sepdebugcrcfix.1: tools/sepdebugcrcfix.c $(TOP_CONFIGURE_AC)
-	$(MAKE) $(AM_MAKEFLAGS) sepdebugcrcfix$(EXEEXT)
-	$(HELP2MAN) -N --output=$@ \
-	  --name='fixes CRC for separate .debug files' \
-	  ./sepdebugcrcfix$(EXEEXT)
+sepdebugcrcfix.1: tools/sepdebugcrcfix.c configure.ac sepdebugcrcfix$(EXEEXT)
+	@case '$?' in \
+	  *$<* | *configure.ac* ) $(HELP2MAN) -N --output=$@ \
+		--name='fixes CRC for separate .debug files' \
+		./sepdebugcrcfix$(EXEEXT) ;;\
+	  * ) : ;; \
+	esac
 
-find-debuginfo.sh.1: $(top_srcdir)/scripts/find-debuginfo.sh $(TOP_CONFIGURE_AC)
+# Since the script isn't generated this doesn't need any special casing.
+find-debuginfo.sh.1: $(top_srcdir)/scripts/find-debuginfo.sh
 	$(HELP2MAN) -N --output=$@ \
 	  --name='finds debuginfo and processes it' \
 	  $(top_srcdir)/scripts/find-debuginfo.sh
-- 
2.18.4