[PATCH 0/4] gdb/doc: build fixes and improvements

[PATCH 0/4] gdb/doc: build fixes and improvements

[Andrew Burgess]

The first patch in this series is a minor bug fix for my recent
doc/Makefile.in patch -- it's not serious, we end up rebuilding some
things unnecessarily, but everything does build correctly.

The second patch removes some dead code from the Makefile.in.

The third and fourth patches fix some build issues when trying to
build all the docs in parallel.  Normally, when building gdb we only
build the info pages from the docs/ directory, and this works fine in
parallel.  But if you try 'make -j20 -C gdb/doc all-docs' you'll find
there are some issues with recipes not working when run in parallel.
These should all be fixed with this series.

--

Andrew Burgess (4):
  gdb/doc: don't delete *.pod files too early
  gdb/doc: don't try to copy GDBvn.texi from the source tree
  gdb/doc: fix parallel build of refcard related targets
  gdb/doc: fix parallel build of pdf and dvi files

 gdb/configure       |  33 +++++++++++++-
 gdb/configure.ac    |  21 +++++++++
 gdb/doc/Makefile.in | 104 ++++++++++++++++++++++----------------------
 3 files changed, 103 insertions(+), 55 deletions(-)


base-commit: a4f76c0765a0b9c643dc91d5a398a1cd9519572b
-- 
2.25.4

[PATCH 1/4] gdb/doc: don't delete *.pod files too early

[Andrew Burgess]

When doing 'make -C gdb/doc man' to build the man pages, I noticed
that the outputs were being rebuilt each time the make command was
rerun, even when the input files hadn't changed.

This was caused by this commit:

  commit 824083f34c222aa7419e2ea58e82d6f230d5f531
  Date:   Fri Apr 12 17:47:20 2024 +0100

      gdb/doc: use silent-rules.mk in the Makefile

Which split the generation of the .pod file from the actual creation
of the man page file.  Prior to this split it was OK to delete the
.pod file at the end of the recipe, the rule depending on the .texi
input file, and output was the .1 or .5 man page file.

Now however, with the split, the man page creation depends on the .pod
file, if we delete this after creating the .1 or .5 man page file then
the next time we run 'make' the .pod file is missing and is
regenerated, which in turn triggers the regeneration of the man page
file.

Fix this by leaving the .pod file around, and only cleaning up these
files in the 'mostlyclean' target.

Which leads to a second problem, the POD_FILE_TMPS is not created
correctly, so we don't actually clean up the .pod files!  This too is
fixed in this commit.

After this commit running 'make -C gdb/doc man' will build the manual
pages the first time, and each subsequent run will do nothing.

Running 'make -C gdb/doc mostlyclean' will now delete the .pod files.
---
 gdb/doc/Makefile.in | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/gdb/doc/Makefile.in b/gdb/doc/Makefile.in
index 28d829fdfee..3f3fe7b7ed9 100644
--- a/gdb/doc/Makefile.in
+++ b/gdb/doc/Makefile.in
@@ -190,8 +190,8 @@ MANS = $(MAN1S) $(MAN5S)
 
 # The pod files that are generated as a side effect of creating the
 # man pages.
-POD_FILE_TMPS = $(patsubst %.1,%.pod,$MAN1S) \
-		$(patsubst %.5,%.pod,$MAN1S)
+POD_FILE_TMPS = $(patsubst %.1,%.pod,$(MAN1S)) \
+		$(patsubst %.5,%.pod,$(MAN5S))
 
 HAVE_NATIVE_GCORE_TARGET = @HAVE_NATIVE_GCORE_TARGET@
 HAVE_NATIVE_GCORE_HOST = @HAVE_NATIVE_GCORE_HOST@
@@ -665,12 +665,10 @@ annotate/index.html: $(ANNOTATE_DOC_FILES)
 $(MAN1S) : %.1 : %.pod $(GDB_DOC_FILES)
 	$(ECHO_TEXI2MAN) ($(POD2MAN1) $*.pod | sed -e '/^.if n .na/d' > $@.T$$$$ && \
 		mv -f $@.T$$$$ $@) || (rm -f $@.T$$$$ && exit 1)
-	$(SILENCE) rm -f $*.pod
 
 $(MAN5S) : %.5 : %.pod $(GDB_DOC_FILES)
 	$(ECHO_TEXI2MAN) ($(POD2MAN1) $*.pod | sed -e '/^.if n .na/d' > $@.T$$$$ && \
 		mv -f $@.T$$$$ $@) || (rm -f $@.T$$$$ && exit 1)
-	$(SILENCE) rm -f $*.pod
 
 force:
 
-- 
2.25.4

[PATCH 2/4] gdb/doc: don't try to copy GDBvn.texi from the source tree

[Andrew Burgess]

Long ago we used to carry a copy of GDBvn.texi in the source tree.
The build recipe for gdb.dvi and gdb.pdf contained instructions for
copying this file from the source tree if the file had not been
generated.

That's right, we also have a rule that builds GDBvn.texi, and
GDBvn.texi is in the dependency list for both gdb.dvi and gdb.pdf.

On top of this, GDBvn.texi was removed from the source tree in 2000,
in this commit:

  commit 1171f8d754f71e35fd2f6c4e82b00f3c685e515f
  Date:   Tue Feb 8 03:37:34 2000 +0000

      2000-02-07  Jason Molenda  (jsm@bugshack.cygnus.com)

I think we can assume that everyone is happy generating GDBvn.texi, so
lets remove the redundant copying code.
---
 gdb/doc/Makefile.in | 17 -----------------
 1 file changed, 17 deletions(-)

diff --git a/gdb/doc/Makefile.in b/gdb/doc/Makefile.in
index 3f3fe7b7ed9..9e3ce018e86 100644
--- a/gdb/doc/Makefile.in
+++ b/gdb/doc/Makefile.in
@@ -477,15 +477,6 @@ gdb-cfg.texi: ${srcdir}/${DOC_CONFIG}-cfg.texi
 	ln ${srcdir}/${DOC_CONFIG}-cfg.texi gdb-cfg.texi || \
 	cp ${srcdir}/${DOC_CONFIG}-cfg.texi gdb-cfg.texi
 
-# GDB MANUAL: texinfo source, using @set/@clear/@value/@ifset/@ifclear
-# If your texinfo or makeinfo don't support these, get a new texinfo release
-#
-# The nonsense with GDBvn.texi gets this to run with both Sun and GNU make.
-#   Note that we can *generate* GDBvn.texi, but since we distribute one in the
-#   source directory for the benefit of people who *don't* use this makefile,
-#   VPATH will often tell make not to bother building it, because the one
-#   in the srcdir is up to date.  (if not, then make should build one here).
-
 # Clean these up before each run.  Avoids a catch 22 with not being
 # able to re-generate these files (to fix a corruption) because these
 # files contain a corruption.
@@ -494,10 +485,6 @@ GDB_TEX_TMPS = gdb.aux gdb.cp* gdb.fn* gdb.ky* gdb.log gdb.pg* gdb.toc \
 
 # GDB MANUAL: TeX dvi file
 gdb.dvi: ${GDB_DOC_FILES}
-	$(SILENCE) if [ ! -f ./GDBvn.texi ]; then \
-		(test "$(LN_S)" = "ln -s" && ln -s $(srcdir)/GDBvn.texi .) || \
-		ln $(srcdir)/GDBvn.texi . || \
-		cp $(srcdir)/GDBvn.texi . ; else true; fi
 	$(SILENCE) rm -f $(GDB_TEX_TMPS)
 	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) $(READLINE_TEXI_INCFLAG) \
 		-I ${GDBMI_DIR} -I $(srcdir) $(srcdir)/gdb.texinfo
@@ -506,10 +493,6 @@ gdb.ps: gdb.dvi
 	$(ECHO_DVIPS) $(DVIPS) $(SILENT_Q_FLAG) -o $@ $?
 
 gdb.pdf: ${GDB_DOC_FILES}
-	$(SILENCE) if [ ! -f ./GDBvn.texi ]; then \
-		(test "$(LN_S)" = "ln -s" && ln -s $(srcdir)/GDBvn.texi .) || \
-		ln $(srcdir)/GDBvn.texi . || \
-		cp $(srcdir)/GDBvn.texi . ; else true; fi
 	$(SILENCE) rm -f $(GDB_TEX_TMPS)
 	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) --pdf \
 		$(READLINE_TEXI_INCFLAG) -I ${GDBMI_DIR} -I $(srcdir) \
-- 
2.25.4

[PATCH 3/4] gdb/doc: fix parallel build of refcard related targets

[Andrew Burgess]

There are two problems we encounter when trying to build the refcard
related target in parallel, i.e.:

  $ make -j20 -C gdb/doc/ refcard.dvi refcard.ps refcard.pdf

These problems are:

(1) The refcard.dvi and refcard.pdf targets both try and generate the
    tmp.sed and sedref.tex files.  If two make threads end up trying
    to create these files at the same time then the result is these
    files become corrupted.

    I've fixed this by creating a new rule that creates sedref.tex,
    both refcard.dvi and refcard.pdf now depend on this, and make will
    build sedref.tex just once.  The tmp.sed file is now generated as
    refcard.sed, this is generated and deleted as a temporary file
    within the sedref.tex recipe.

(2) Having created sedref.tex the recipes for refcard.dvi and
    refcard.pdf both run various LaTeX based tools with sedref.tex as
    the input file.  The problem with this is that these tools all
    rely on creating temporary files calls sedref.*.

    If the refcard.dvi and refcard.pdf rules run at the same time then
    these temporary files clash and overwrite each other causing the
    build to fail.

    We already copy the result file in order to rename it, our input
    file is sedref.tex which results in an output file named
    sedref.dvi or sedref.pdf, but we actually want refcard.dvi or
    refcard.pdf.  So within the recipe for refcard.dvi I copy the
    input file from sedref.tex to sedref_dvi.tex.  Now all the temp
    files are named sedref_dvi.* and the output is sedref_dvi.dvi, I
    then rename this new output file to refcard.dvi.

    I've done the same thing for refcard.pdf, but I copy the input
    to sedref_pdf.tex.

    In this way the temp files no longer clash, and both recipes can
    safely run in parallel.

After this commit I was able to reliably build all of the refcard
targets in parallel.  There should be no change in the final file.
---
 gdb/doc/Makefile.in | 39 ++++++++++++++++++++-------------------
 1 file changed, 20 insertions(+), 19 deletions(-)

diff --git a/gdb/doc/Makefile.in b/gdb/doc/Makefile.in
index 9e3ce018e86..c05e561edeb 100644
--- a/gdb/doc/Makefile.in
+++ b/gdb/doc/Makefile.in
@@ -410,31 +410,32 @@ de-stage3: force
 	-(cd stage3 ; mv -f * ..)
 	-rmdir stage3
 
-# GDB QUICK REFERENCE (dvi output)
-refcard.dvi : refcard.tex $(REFEDITS)
-	$(SILENCE) echo > tmp.sed
+
+sedref.tex : refcard.tex $(REFEDITS)
+	$(SILENCE) echo > refcard.sed
 	$(SILENCE) for f in x $(REFEDITS) ; do \
 		test x$$f = xx && continue ; \
-		cat $(srcdir)/$$f >>tmp.sed ; \
+		cat $(srcdir)/$$f >>refcard.sed ; \
 	done
-	$(SILENCE) sed -f tmp.sed $(srcdir)/refcard.tex >sedref.tex
-	$(ECHO_TEX) $(SET_TEXINPUTS) $(TEX) sedref.tex
-	$(SILENCE) mv sedref.dvi refcard.dvi
-	$(SILENCE) rm -f sedref.log sedref.tex tmp.sed
+	$(ECHO_GEN) sed -f refcard.sed $(srcdir)/refcard.tex >$@
+	$(SILENCE) rm -f refcard.sed
+
+
+# GDB QUICK REFERENCE (dvi output)
+refcard.dvi : sedref.tex
+	$(SILENCE) rm -f sedref_dvi.*
+	$(SILENCE) cp $< sedref_dvi.tex
+	$(ECHO_TEX) $(SET_TEXINPUTS) $(TEX) sedref_dvi.tex && \
+		mv sedref_dvi.dvi $@
 
 refcard.ps : refcard.dvi
 	$(ECHO_DVIPS) $(DVIPS) $(SILENT_Q_FLAG) -t landscape -o $@ $?
 
-refcard.pdf : refcard.tex $(REFEDITS)
-	$(SILENCE) echo > tmp.sed
-	$(SILENCE) for f in x $(REFEDITS) ; do \
-		test x$$f = xx && continue ; \
-		cat $(srcdir)/$$f >>tmp.sed ; \
-	done
-	$(SILENCE) sed -f tmp.sed $(srcdir)/refcard.tex >sedref.tex
-	$(ECHO_PDFTEX) $(SET_TEXINPUTS) $(PDFTEX) sedref.tex
-	$(SILENCE) mv sedref.pdf refcard.pdf
-	$(SILENCE) rm -f sedref.log sedref.tex tmp.sed
+refcard.pdf : sedref.tex
+	$(SILENCE) rm -f sedref_pdf.*
+	$(SILENCE) cp $< sedref_pdf.tex
+	$(ECHO_PDFTEX) $(SET_TEXINPUTS) $(PDFTEX) sedref_pdf.tex && \
+		mv sedref_pdf.pdf $@
 
 # File to record current GDB version number.
 GDBvn.texi : version.subst
@@ -667,7 +668,7 @@ mostlyclean:
 	rm -f $(GDB_TEX_TMPS)
 	rm -f $(STABS_TEX_TMPS)
 	rm -f $(ANNOTATE_TEX_TMPS)
-	rm -f sedref.dvi sedref.tex tmp.sed sedref.log
+	rm -f refcard.sed sedref.tex sedref_dvi.* sedref_pdf.*
 	rm -f $(POD_FILE_TMPS)
 
 clean: mostlyclean
-- 
2.25.4

[PATCH 4/4] gdb/doc: fix parallel build of pdf and dvi files

[Andrew Burgess]

When building with 'make -j20 -C gdb/doc all-doc' I often see problems
caused from trying to build some dvi files in parallel with some pdf
files.  The problem files are: gdb.dvi and gdb.pdf; stabs.dvi and
stabs.pdf; and annotate.dvi and annotate.pdf.

The problem is that building these files create temporary files in the
local directory.  There's already a race here that two make threads
might try to create these files at the same time.

But it gets worse, to avoid issues where a failed build could leave
these temporary files in a corrupted state, and so prevent the next
build from succeeding, the recipe for each of these files delete all
the temporary files first, this obviously causes problems if some
other thread has already started the build and is relying on these
temporary files.

To work around this problem I propose we start using the --build flag
for texi2dvi (which is the same tool used to create the pdf files).
The --build flag allows for the temporary files to be placed into a
sub-directory, e.g. creating gdb.pdf will create the temporary files
in gdb.t2d/pdf/ and gdb.dvi uses gdb.t2d/dvi/, in this way the
temporary files will never clash, and we can easily delete the
specific sub-directory at the start of the recipe to ensure a
successful build.

Very old versions of texi2dvi don't support --build, so I've added a
configure check for this option.  If the option is not supported then
we don't use it.  This means we fall back to placing temporary files
in the local directory, which means parallel builds will remain
broken.  The --build option is definitely present in texi2dvi version
6.1, from 2016, so any version after that should be fine.

For me, with this fix in place, I can now run 'make -j20 -C gdb/doc
all-doc' without seeing any build problems.
---
 gdb/configure       | 33 +++++++++++++++++++++++++++++++--
 gdb/configure.ac    | 21 +++++++++++++++++++++
 gdb/doc/Makefile.in | 42 +++++++++++++++++++++++++++++-------------
 3 files changed, 81 insertions(+), 15 deletions(-)

diff --git a/gdb/configure b/gdb/configure
index 98cd488a737..f6e8a387172 100755
--- a/gdb/configure
+++ b/gdb/configure
@@ -765,6 +765,7 @@ ENABLE_BFD_64_BIT_TRUE
 subdirs
 GDB_DATADIR
 DEBUGDIR
+TEXI2DVI_EXTRA_FLAGS
 MAKEINFO_EXTRA_FLAGS
 MAKEINFOFLAGS
 MAKEINFO
@@ -11499,7 +11500,7 @@ else
   lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
   lt_status=$lt_dlunknown
   cat > conftest.$ac_ext <<_LT_EOF
-#line 11502 "configure"
+#line 11503 "configure"
 #include "confdefs.h"
 
 #if HAVE_DLFCN_H
@@ -11605,7 +11606,7 @@ else
   lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
   lt_status=$lt_dlunknown
   cat > conftest.$ac_ext <<_LT_EOF
-#line 11608 "configure"
+#line 11609 "configure"
 #include "confdefs.h"
 
 #if HAVE_DLFCN_H
@@ -24601,6 +24602,34 @@ if test x"$gdb_cv_have_makeinfo_click" = xyes; then
 fi
 
 
+## Figure out if texi2dvi supports the --build option.  Older versions
+## didn't.  If this option is not supported then parallel building in
+## the docs/ directory is probably going to be broken.  But single
+## threaded build should be fine.
+TEXI2DVI=texi2dvi
+TEXI2DVI_EXTRA_FLAGS=""
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether $TEXI2DVI supports --build" >&5
+$as_echo_n "checking whether $TEXI2DVI supports --build... " >&6; }
+if ${gdb_cv_have_texi2dvi_build_opt+:} false; then :
+  $as_echo_n "(cached) " >&6
+else
+  echo '\input texinfo' >conftest.texi
+   echo 'abc' >>conftest.texi
+   echo '@bye' >>conftest.texi
+   cp conftest.texi /home/andrew/tmp/
+   if eval "$TEXI2DVI --build=tidy conftest.texi >&5 2>&5"; then
+     gdb_cv_have_texi2dvi_build_opt=yes
+   else
+     gdb_cv_have_texi2dvi_build_opt=no
+   fi
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $gdb_cv_have_texi2dvi_build_opt" >&5
+$as_echo "$gdb_cv_have_texi2dvi_build_opt" >&6; }
+if test x"$gdb_cv_have_texi2dvi_build_opt" = xyes; then
+  TEXI2DVI_EXTRA_FLAGS="$TEXI2DVI_EXTRA_FLAGS --build=tidy"
+fi
+
+
 
 
 # Check whether --with-separate-debug-dir was given.
diff --git a/gdb/configure.ac b/gdb/configure.ac
index 62ff09cea20..3b726e5f0aa 100644
--- a/gdb/configure.ac
+++ b/gdb/configure.ac
@@ -117,6 +117,27 @@ if test x"$gdb_cv_have_makeinfo_click" = xyes; then
 fi
 AC_SUBST(MAKEINFO_EXTRA_FLAGS)
 
+## Figure out if texi2dvi supports the --build option.  Older versions
+## didn't.  If this option is not supported then parallel building in
+## the docs/ directory is probably going to be broken.  But single
+## threaded build should be fine.
+TEXI2DVI=texi2dvi
+TEXI2DVI_EXTRA_FLAGS=""
+AC_CACHE_CHECK([whether $TEXI2DVI supports --build], gdb_cv_have_texi2dvi_build_opt,
+  [echo '\input texinfo' >conftest.texi
+   echo 'abc' >>conftest.texi
+   echo '@bye' >>conftest.texi
+   cp conftest.texi /home/andrew/tmp/
+   if eval "$TEXI2DVI --build=tidy conftest.texi >&5 2>&5"; then
+     gdb_cv_have_texi2dvi_build_opt=yes
+   else
+     gdb_cv_have_texi2dvi_build_opt=no
+   fi])
+if test x"$gdb_cv_have_texi2dvi_build_opt" = xyes; then
+  TEXI2DVI_EXTRA_FLAGS="$TEXI2DVI_EXTRA_FLAGS --build=tidy"
+fi
+AC_SUBST(TEXI2DVI_EXTRA_FLAGS)
+
 GDB_AC_WITH_DIR(DEBUGDIR, separate-debug-dir,
     [look for global separate debug info in this path @<:@LIBDIR/debug@:>@],
     [${libdir}/debug])
diff --git a/gdb/doc/Makefile.in b/gdb/doc/Makefile.in
index c05e561edeb..5836d6ce024 100644
--- a/gdb/doc/Makefile.in
+++ b/gdb/doc/Makefile.in
@@ -68,6 +68,8 @@ TEXI2ROFF=texi2roff
 
 # where to find texi2dvi, ditto
 TEXI2DVI=texi2dvi
+TEXI2DVI_EXTRA_FLAGS=@TEXI2DVI_EXTRA_FLAGS@
+TEXI2DVI_CMD = $(TEXI2DVI) $(TEXI2DVI_EXTRA_FLAGS)
 
 # Package to install the docs under
 PACKAGE = @PACKAGE@
@@ -481,21 +483,26 @@ gdb-cfg.texi: ${srcdir}/${DOC_CONFIG}-cfg.texi
 # Clean these up before each run.  Avoids a catch 22 with not being
 # able to re-generate these files (to fix a corruption) because these
 # files contain a corruption.
+#
+# If TEXI2DVI_CMD includes the --build option then these files should
+# not be created in the current directory, but attempting to delete
+# them is harmless.
 GDB_TEX_TMPS = gdb.aux gdb.cp* gdb.fn* gdb.ky* gdb.log gdb.pg* gdb.toc \
 	gdb.tp* gdb.vr*
 
 # GDB MANUAL: TeX dvi file
 gdb.dvi: ${GDB_DOC_FILES}
-	$(SILENCE) rm -f $(GDB_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) $(READLINE_TEXI_INCFLAG) \
-		-I ${GDBMI_DIR} -I $(srcdir) $(srcdir)/gdb.texinfo
+	$(SILENCE) rm -fr $(GDB_TEX_TMPS) gdb.t2d/dvi/
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) \
+		$(READLINE_TEXI_INCFLAG) -I ${GDBMI_DIR} -I $(srcdir) \
+		$(srcdir)/gdb.texinfo
 
 gdb.ps: gdb.dvi
 	$(ECHO_DVIPS) $(DVIPS) $(SILENT_Q_FLAG) -o $@ $?
 
 gdb.pdf: ${GDB_DOC_FILES}
-	$(SILENCE) rm -f $(GDB_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) --pdf \
+	$(SILENCE) rm -fr $(GDB_TEX_TMPS) gdb.t2d/pdf/
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) --pdf \
 		$(READLINE_TEXI_INCFLAG) -I ${GDBMI_DIR} -I $(srcdir) \
 		$(srcdir)/gdb.texinfo
 
@@ -596,41 +603,49 @@ stabs/index.html: $(STABS_DOC_FILES)
 # Clean these up before each run.  Avoids a catch 22 with not being
 # able to re-generate these files (to fix a corruption) because these
 # files contain a corruption.
+#
+# If TEXI2DVI_CMD includes the --build option then these files should
+# not be created in the current directory, but attempting to delete
+# them is harmless.
 STABS_TEX_TMPS = stabs.aux stabs.cp* stabs.fn* stabs.ky* \
 	stabs.log stabs.pg* stabs.toc stabs.tp* stabs.vr*
 
 # STABS DOCUMENTATION: TeX dvi file
 stabs.dvi : $(STABS_DOC_FILES)
-	$(SILENCE) rm -f $(STABS_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) -I $(srcdir) \
+	$(SILENCE) rm -fr $(STABS_TEX_TMPS) stabs.t2d/dvi/
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) -I $(srcdir) \
 		$(srcdir)/stabs.texinfo
 
 stabs.ps: stabs.dvi
 	$(ECHO_DVIPS) $(DVIPS) $(SILENT_Q_FLAG) -o $@ $?
 
 stabs.pdf: $(STABS_DOC_FILES)
-	$(SILENCE) rm -f $(STABS_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) --pdf -I $(srcdir) \
+	$(SILENCE) rm -fr $(STABS_TEX_TMPS) stabs.t2d/pdf/
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) --pdf -I $(srcdir) \
 		$(srcdir)/stabs.texinfo
 
 # Clean these up before each run.  Avoids a catch 22 with not being
 # able to re-generate these files (to fix a corruption) because these
 # files contain a corruption.
+#
+# If TEXI2DVI_CMD includes the --build option then these files should
+# not be created in the current directory, but attempting to delete
+# them is harmless.
 ANNOTATE_TEX_TMPS = annotate.aux annotate.cp* annotate.fn* annotate.ky* \
 	annotate.log annotate.pg* annotate.toc annotate.tp* annotate.vr*
 
 # ANNOTATE DOCUMENTATION: TeX dvi file
 annotate.dvi : $(ANNOTATE_DOC_FILES)
-	$(SILENCE) rm -f $(ANNOTATE_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) -I $(srcdir) \
+	$(SILENCE) rm -fr $(ANNOTATE_TEX_TMPS) annotate.t2d/dvi/
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) -I $(srcdir) \
 		$(srcdir)/annotate.texinfo
 
 annotate.ps: annotate.dvi
 	$(ECHO_DVIPS) $(DVIPS) $(SILENT_Q_FLAG) -o $@ $?
 
 annotate.pdf: $(ANNOTATE_DOC_FILES)
-	$(SILENCE) rm -f $(ANNOTATE_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) --pdf -I $(srcdir) \
+	$(SILENCE) rm -fr $(ANNOTATE_TEX_TMPS) annotate.t2d/pdf/
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) --pdf -I $(srcdir) \
 		$(srcdir)/annotate.texinfo
 
 annotate.info: $(ANNOTATE_DOC_FILES)
@@ -665,6 +680,7 @@ Makefile: Makefile.in $(host_makefile_frag) ../config.status
 
 mostlyclean:
 	rm -f gdb.mm gdb.ms gdb.me links2roff
+	rm -fr annotate.t2d/ stabs.t2d/ gdb.t2d/
 	rm -f $(GDB_TEX_TMPS)
 	rm -f $(STABS_TEX_TMPS)
 	rm -f $(ANNOTATE_TEX_TMPS)
-- 
2.25.4

Re: [PATCH 2/4] gdb/doc: don't try to copy GDBvn.texi from the source tree

[Tom Tromey]

>>>>> "Andrew" == Andrew Burgess <aburgess@redhat.com> writes:

Andrew> That's right, we also have a rule that builds GDBvn.texi, and
Andrew> GDBvn.texi is in the dependency list for both gdb.dvi and gdb.pdf.

Andrew> On top of this, GDBvn.texi was removed from the source tree in 2000,
Andrew> in this commit:

Andrew>   commit 1171f8d754f71e35fd2f6c4e82b00f3c685e515f
Andrew>   Date:   Tue Feb 8 03:37:34 2000 +0000

Andrew>       2000-02-07  Jason Molenda  (jsm@bugshack.cygnus.com)

Andrew> I think we can assume that everyone is happy generating GDBvn.texi, so
Andrew> lets remove the redundant copying code.

I am not sure about this change.

I downloaded gdb-14.2.tar.xz and I do see gdb/doc/GDBvn.texi in the
source tree there.  I think this is done because we also ship the .info
files, so that users won't have to have makeinfo in order to build.

(I'm not sure how well that plan really works since there used to be
open bugs about not building when makeinfo is missing ... not sure of
the current state.)

Tom

Re: [PATCH 1/4] gdb/doc: don't delete *.pod files too early

[Tom Tromey]

>>>>> "Andrew" == Andrew Burgess <aburgess@redhat.com> writes:

Andrew> After this commit running 'make -C gdb/doc man' will build the manual
Andrew> pages the first time, and each subsequent run will do nothing.

Andrew> Running 'make -C gdb/doc mostlyclean' will now delete the .pod files.

This looks good to me.
Approved-By: Tom Tromey <tom@tromey.com>

Tom

Re: [PATCH 3/4] gdb/doc: fix parallel build of refcard related targets

[Tom Tromey]

>>>>> "Andrew" == Andrew Burgess <aburgess@redhat.com> writes:

Andrew> After this commit I was able to reliably build all of the refcard
Andrew> targets in parallel.  There should be no change in the final file.

Looks good, thank you.
Approved-By: Tom Tromey <tom@tromey.com>

Tom

Re: [PATCH 4/4] gdb/doc: fix parallel build of pdf and dvi files

[Tom Tromey]

>>>>> "Andrew" == Andrew Burgess <aburgess@redhat.com> writes:

Andrew> When building with 'make -j20 -C gdb/doc all-doc' I often see problems
Andrew> caused from trying to build some dvi files in parallel with some pdf
Andrew> files.  The problem files are: gdb.dvi and gdb.pdf; stabs.dvi and
Andrew> stabs.pdf; and annotate.dvi and annotate.pdf.

Andrew> The problem is that building these files create temporary files in the
Andrew> local directory.  There's already a race here that two make threads
Andrew> might try to create these files at the same time.

Does this mean that without --build the tools here use fixed temporary
file names?  That seems bad.

Anyway the patch seems ok to me.  The configure check is maybe overkill
IMO but considering that it exists it also seems ok.

Approved-By: Tom Tromey <tom@tromey.com>

Tom

Re: [PATCH 2/4] gdb/doc: don't try to copy GDBvn.texi from the source tree

[Eli Zaretskii]

> From: Tom Tromey <tom@tromey.com>
> Cc: gdb-patches@sourceware.org
> Date: Mon, 13 May 2024 10:12:35 -0600
> 
> >>>>> "Andrew" == Andrew Burgess <aburgess@redhat.com> writes:
> 
> Andrew> That's right, we also have a rule that builds GDBvn.texi, and
> Andrew> GDBvn.texi is in the dependency list for both gdb.dvi and gdb.pdf.
> 
> Andrew> On top of this, GDBvn.texi was removed from the source tree in 2000,
> Andrew> in this commit:
> 
> Andrew>   commit 1171f8d754f71e35fd2f6c4e82b00f3c685e515f
> Andrew>   Date:   Tue Feb 8 03:37:34 2000 +0000
> 
> Andrew>       2000-02-07  Jason Molenda  (jsm@bugshack.cygnus.com)
> 
> Andrew> I think we can assume that everyone is happy generating GDBvn.texi, so
> Andrew> lets remove the redundant copying code.
> 
> I am not sure about this change.
> 
> I downloaded gdb-14.2.tar.xz and I do see gdb/doc/GDBvn.texi in the
> source tree there.  I think this is done because we also ship the .info
> files, so that users won't have to have makeinfo in order to build.

Right.  So I don't think we should remove GDBvn.texi from the tarball.

> (I'm not sure how well that plan really works since there used to be
> open bugs about not building when makeinfo is missing ... not sure of
> the current state.)

Building GDB doesn't require running makeinfo, at least it didn't in
GDB 14.

Re: [PATCH 2/4] gdb/doc: don't try to copy GDBvn.texi from the source tree

[Andrew Burgess]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Tom Tromey <tom@tromey.com>
>> Cc: gdb-patches@sourceware.org
>> Date: Mon, 13 May 2024 10:12:35 -0600
>> 
>> >>>>> "Andrew" == Andrew Burgess <aburgess@redhat.com> writes:
>> 
>> Andrew> That's right, we also have a rule that builds GDBvn.texi, and
>> Andrew> GDBvn.texi is in the dependency list for both gdb.dvi and gdb.pdf.
>> 
>> Andrew> On top of this, GDBvn.texi was removed from the source tree in 2000,
>> Andrew> in this commit:
>> 
>> Andrew>   commit 1171f8d754f71e35fd2f6c4e82b00f3c685e515f
>> Andrew>   Date:   Tue Feb 8 03:37:34 2000 +0000
>> 
>> Andrew>       2000-02-07  Jason Molenda  (jsm@bugshack.cygnus.com)
>> 
>> Andrew> I think we can assume that everyone is happy generating GDBvn.texi, so
>> Andrew> lets remove the redundant copying code.
>> 
>> I am not sure about this change.
>> 
>> I downloaded gdb-14.2.tar.xz and I do see gdb/doc/GDBvn.texi in the
>> source tree there.  I think this is done because we also ship the .info
>> files, so that users won't have to have makeinfo in order to build.
>
> Right.  So I don't think we should remove GDBvn.texi from the tarball.

OK, I'll drop this patch.  I'll likely post the inverse patch at some
point -- we have other recipes that use GDBvn.texi, but don't include
the copy code, so maybe they should?

I'll take another look, including trying with a release tar ball, maybe
there's a reason why some rules that depend on GDBvn.texi, while others
don't.... but that can all wait for some future patch.

>
>> (I'm not sure how well that plan really works since there used to be
>> open bugs about not building when makeinfo is missing ... not sure of
>> the current state.)
>
> Building GDB doesn't require running makeinfo, at least it didn't in
> GDB 14.

I'll double check the status of this after these changes.

Thanks,
Andrew

Re: [PATCH 4/4] gdb/doc: fix parallel build of pdf and dvi files

[Andrew Burgess]

Tom Tromey <tom@tromey.com> writes:

>>>>>> "Andrew" == Andrew Burgess <aburgess@redhat.com> writes:
>
> Andrew> When building with 'make -j20 -C gdb/doc all-doc' I often see problems
> Andrew> caused from trying to build some dvi files in parallel with some pdf
> Andrew> files.  The problem files are: gdb.dvi and gdb.pdf; stabs.dvi and
> Andrew> stabs.pdf; and annotate.dvi and annotate.pdf.
>
> Andrew> The problem is that building these files create temporary files in the
> Andrew> local directory.  There's already a race here that two make threads
> Andrew> might try to create these files at the same time.
>
> Does this mean that without --build the tools here use fixed temporary
> file names?  That seems bad.

Yes, building gdb.dvi produces files matching:

  gdb.aux gdb.cp* gdb.fn* gdb.ky* gdb.log gdb.pg* gdb.toc gdb.tp* gdb.vr*

And building gdb.pdf produces the same set of files.  These two targets
use the same tool with different flags (to create dvi vs pdf).

Indeed this isn't great, which is why (I guess) the tool gained the
--build option.

The other option would be to use the same trick I used with the refcard,
copy the input file to some unique name, e.g. gdb.texi -> gdb_dvi.texi,
then the temporary files will not clash between dvi/pdf targets.

The reason I used that trick for refcard, is that is invoking
(basically) LaTeX directly, which doesn't have a --build option or
equivalent (that I could find).  For these targets (e.g. gdb.{dvi,pdf})
we're using "texi2dvi" or "texi2dvi --pdf", which does have an
"official" way to solve this problem, so I figured using that was
better.

> Anyway the patch seems ok to me.  The configure check is maybe overkill
> IMO but considering that it exists it also seems ok.

It seemed cheap to add, and there are some folk who build on older
systems.  I was worried they might not have this option.

>
> Approved-By: Tom Tromey <tom@tromey.com>

Thanks,
Andrew

Re: [PATCH 1/4] gdb/doc: don't delete *.pod files too early

[Andrew Burgess]

Tom Tromey <tom@tromey.com> writes:

>>>>>> "Andrew" == Andrew Burgess <aburgess@redhat.com> writes:
>
> Andrew> After this commit running 'make -C gdb/doc man' will build the manual
> Andrew> pages the first time, and each subsequent run will do nothing.
>
> Andrew> Running 'make -C gdb/doc mostlyclean' will now delete the .pod files.
>
> This looks good to me.
> Approved-By: Tom Tromey <tom@tromey.com>

Pushed.

Thanks,
Andrew

[PATCHv2 0/5] gdb/doc: parallel build fixes and improvements

[Andrew Burgess]

After feedback on the v1 series I merged the first patch.  I've now
rewritten the commit message for what was the second patch to better
explain why this change is fine.

I've added some additional fixes I found, these are the new patches #2
and #3, then the final two patches are unchanged from v1 (and already
got approved).

The goal of this series is to fix 'make -j20 -C gdb/doc all-doc', but
I've also fixed some doc rebuild issues that currently exist with our
releases.

For testing, as well as a parallel build of 'all-doc' in a git
checkout, I've also been creating a GDB release tar file, unpacking
that into an empty directory and doing the usual configure, make,
'make install' process to ensure that works fine.  The things I'm
checking for in this last step are:

  1. From a freshly unpacked GDB release, by default we don't build
     the man or info pages when doing make and 'make install', we just
     install the copies in the source tree.

  2. If I then 'touch gdb/doc/*.texinfo' in the unpacked release tree
     and then 'make install' the info and man pages are successfully
     rebuilt using the GDBvn.texi and version.subst files in the
     source tree.

  3. If I 'touch gdb/doc/version.subst' then 'make install' we
     correctly recreate GDBvn.texi in the build directory and then
     rebuild the docs using this new file.

  4. And finally, if I 'touch bfd/version.h' then 'make install' we
     correctly rebuild version.subst into the build directory and then
     rebuild everything using this file.

Steps 2, 3, and 4 don't currently work with previous GDB releases.

In V2:

  - Patch #1: the content of this patch is unchanged from v1, however
    the commit message is completely rewritten.  I believe the change
    I made was good, but my previous justification was completely
    wrong!

  - Patches #2 and #3: these patches are completely new in v2.

  - Patches #4 and #5: these are completely unchanged from v1 where
    they were already approved.  Further feedback is of course
    welcome, but if you want to skip looking at these, that's fine
    too.

Thanks,
Andrew

---

Andrew Burgess (5):
  gdb/doc: don't try to copy GDBvn.texi from the source tree
  gdb/doc: allow for version.subst in the source tree
  gdb/doc: also look in srcdir when running TEXI2POD
  gdb/doc: fix parallel build of refcard related targets
  gdb/doc: fix parallel build of pdf and dvi files

 gdb/configure       |  33 +++++++++++-
 gdb/configure.ac    |  21 ++++++++
 gdb/doc/Makefile.in | 121 ++++++++++++++++++++++++--------------------
 3 files changed, 118 insertions(+), 57 deletions(-)


base-commit: 40acbd34527648e0c375b965b16ab5b7f2ecae6c
-- 
2.25.4

[PATCHv2 1/5] gdb/doc: don't try to copy GDBvn.texi from the source tree

[Andrew Burgess]

The build recipe for gdb.dvi and gdb.pdf contains instructions for
copying the GDBvn.texi file from the source tree into the build
directory if the GDBvn.texi file doesn't already exist in the build
directory.

The gdb.dvi and gdb.pdf targets also have a dependency on GDBvn.texi,
and we have a recipe for building GDBvn.texi.

What's happening here is this:

  - In a git checkout of the source tree there is no GDBvn.texi in the
    source tree, the GDBvn.texi dependency will trigger a rebuild of
    GDBvn.texi, which is then used to build gdb.dvi and/or gdb.pdf.

  - In a release tar file we do include a copy of GDBvn.texi.  This
    file will appear to be up to date, and so no copy of GDBvn.texi is
    created within the build directory.  Now when building gdb.dvi
    and/or gdb.pdf we copy (or symlink) the version of GDBvn.texi from
    the source tree into the build directory.

However, copying GDBvn.texi into the source directory is completely
unnecessary.  The gdb.dvi/gdb.pdf recipes both invoke texi2dvi and
pass '-I $(srcdir)' as an argument, this means that texi2dvi will look
in the $(srcdir) to find included files, including GDBvn.texi.

As such I believe we can remove the code that copies GDBvn.texi from
the source tree into the build tree.

I've tested with a release build; creating a release with:

  ./src-release gdb

Then in an empty directory, unpacking the resulting .tar file,
creating a parallel build directory and doing the usual configure,
make, and 'make install'.

Having done this I can then 'touch gdb/doc/*.texinfo' in the unpacked
source tree, and do 'make -C gdb/doc pdf dvi' to rebuild all the pdf
and dvi files, this works fine without having to either build or copy
GDBvn.texi into the build directory.
---
 gdb/doc/Makefile.in | 17 -----------------
 1 file changed, 17 deletions(-)

diff --git a/gdb/doc/Makefile.in b/gdb/doc/Makefile.in
index 6a112b98d0e..3a179e9799a 100644
--- a/gdb/doc/Makefile.in
+++ b/gdb/doc/Makefile.in
@@ -477,15 +477,6 @@ gdb-cfg.texi: ${srcdir}/${DOC_CONFIG}-cfg.texi
 	ln ${srcdir}/${DOC_CONFIG}-cfg.texi gdb-cfg.texi || \
 	cp ${srcdir}/${DOC_CONFIG}-cfg.texi gdb-cfg.texi
 
-# GDB MANUAL: texinfo source, using @set/@clear/@value/@ifset/@ifclear
-# If your texinfo or makeinfo don't support these, get a new texinfo release
-#
-# The nonsense with GDBvn.texi gets this to run with both Sun and GNU make.
-#   Note that we can *generate* GDBvn.texi, but since we distribute one in the
-#   source directory for the benefit of people who *don't* use this makefile,
-#   VPATH will often tell make not to bother building it, because the one
-#   in the srcdir is up to date.  (if not, then make should build one here).
-
 # Clean these up before each run.  Avoids a catch 22 with not being
 # able to re-generate these files (to fix a corruption) because these
 # files contain a corruption.
@@ -494,10 +485,6 @@ GDB_TEX_TMPS = gdb.aux gdb.cp* gdb.fn* gdb.ky* gdb.log gdb.pg* gdb.toc \
 
 # GDB MANUAL: TeX dvi file
 gdb.dvi: ${GDB_DOC_FILES}
-	$(SILENCE) if [ ! -f ./GDBvn.texi ]; then \
-		(test "$(LN_S)" = "ln -s" && ln -s $(srcdir)/GDBvn.texi .) || \
-		ln $(srcdir)/GDBvn.texi . || \
-		cp $(srcdir)/GDBvn.texi . ; else true; fi
 	$(SILENCE) rm -f $(GDB_TEX_TMPS)
 	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) $(READLINE_TEXI_INCFLAG) \
 		-I ${GDBMI_DIR} -I $(srcdir) $(srcdir)/gdb.texinfo
@@ -506,10 +493,6 @@ gdb.ps: gdb.dvi
 	$(ECHO_DVIPS) $(DVIPS) $(SILENT_Q_FLAG) -o $@ $?
 
 gdb.pdf: ${GDB_DOC_FILES}
-	$(SILENCE) if [ ! -f ./GDBvn.texi ]; then \
-		(test "$(LN_S)" = "ln -s" && ln -s $(srcdir)/GDBvn.texi .) || \
-		ln $(srcdir)/GDBvn.texi . || \
-		cp $(srcdir)/GDBvn.texi . ; else true; fi
 	$(SILENCE) rm -f $(GDB_TEX_TMPS)
 	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) --pdf \
 		$(READLINE_TEXI_INCFLAG) -I ${GDBMI_DIR} -I $(srcdir) \
-- 
2.25.4

[PATCHv2 2/5] gdb/doc: allow for version.subst in the source tree

[Andrew Burgess]

In a git checkout of the source code we don't have a version.subst
file in the gdb/doc directory.  When building the GDB docs the
version.subst file is generated on demand (we have a recipe for that).

However, in a release tar file we do include a copy of the
version.subst file in the source tree, as a result the version.subst
recipe will not be run.

If, in a release build, we force the running of any recipe that
depends on version.subst then we run into a problem.  For example,
slightly confusingly, if we 'touch gdb/doc/version.subst' within the
unpacked source tree of a release, then 'make -C gdb/doc GDBvn.texi'
in the build tree, we'll see:

  make: Entering directory '/tmp/build/build/gdb/doc'
    GEN    GDBvn.texi
  sed: can't read version.subst: No such file or directory
  make: Leaving directory '/tmp/build/build/gdb/doc'

The problem is that every reference to version.subst in GDB's Makefile
assumes that the version.subst file will always be in the build
directory.

In this commit I replace direct references to version.subst with a
small shell snippet which returns the path to the version.subst file
to use.  If there is a version.subst file in the build directory then
that file is selected, otherwise we return the path to a version.subst
file in the source directory.

As the POD2MAN1 and POD2MAN5 commands both depend on version.subst I
have added version.subst as a dependency to the two recipes that use
this variable.
---
 gdb/doc/Makefile.in | 21 ++++++++++++++++-----
 1 file changed, 16 insertions(+), 5 deletions(-)

diff --git a/gdb/doc/Makefile.in b/gdb/doc/Makefile.in
index 3a179e9799a..e5f8a11008e 100644
--- a/gdb/doc/Makefile.in
+++ b/gdb/doc/Makefile.in
@@ -178,10 +178,21 @@ MANCONF = -Dman
 TEXI2POD = perl $(srcdir)/../../etc/texi2pod.pl \
 		$(MAKEINFOFLAGS) $(MAKEINFO_EXTRA_FLAGS)
 
+# Shell snippet which will provide the full filename of the
+# version.subst file to use.  If there is a copy in the build
+# directory then that is preferred, otherwise we assume there is a
+# copy in the source tree.
+VERSION_SUBST = $$(if [ -r version.subst ]; \
+                   then \
+                      (echo $$PWD/version.subst); \
+                   else \
+                      (builtin cd $(srcdir) && echo $$PWD/version.subst); \
+                   fi)
+
 POD2MAN1 = pod2man --center="GNU Development Tools" \
-		   --release="gdb-`sed q version.subst`" --section=1
+		   --release="gdb-`sed q $(VERSION_SUBST)`" --section=1
 POD2MAN5 = pod2man --center="GNU Development Tools" \
-		   --release="gdb-`sed q version.subst`" --section=5
+		   --release="gdb-`sed q $(VERSION_SUBST)`" --section=5
 
 # List of man pages generated from gdb.texi
 MAN1S = gdb.1 gdbserver.1 gcore.1 gdb-add-index.1
@@ -439,7 +450,7 @@ refcard.pdf : refcard.tex $(REFEDITS)
 # File to record current GDB version number.
 GDBvn.texi : version.subst
 	$(ECHO_GEN)
-	$(SILENCE) echo "@set GDBVN `sed q version.subst`" > ./GDBvn.new
+	$(SILENCE) echo "@set GDBVN `sed q $(VERSION_SUBST)`" > ./GDBvn.new
 	$(SILENCE) if [ -n "$(PKGVERSION)" ]; then \
 	  echo "@set VERSION_PACKAGE $(PKGVERSION)" >> ./GDBvn.new; \
 	fi
@@ -654,13 +665,13 @@ annotate/index.html: $(ANNOTATE_DOC_FILES)
 # pages, then the .pod files must become a dependency, this will
 # trigger an attempt to rebuild these files while building and
 # installing a release of GDB, which is something we don't want.
-$(MAN1S) : %.1 : $(GDB_DOC_FILES)
+$(MAN1S) : %.1 : $(GDB_DOC_FILES) version.subst
 	$(ECHO_TEXI2POD) $(TEXI2POD) $(MANCONF) -D$* < $(srcdir)/gdb.texinfo > $*.pod
 	$(ECHO_TEXI2MAN) ($(POD2MAN1) $*.pod | sed -e '/^.if n .na/d' > $@.T$$$$ && \
 		mv -f $@.T$$$$ $@) || (rm -f $@.T$$$$ && exit 1)
 	$(SILENCE) rm -f $*.pod
 
-$(MAN5S) : %.5 : $(GDB_DOC_FILES)
+$(MAN5S) : %.5 : $(GDB_DOC_FILES) version.subst
 	$(ECHO_TEXI2POD) $(TEXI2POD) $(MANCONF) -D$* < $(srcdir)/gdb.texinfo > $*.pod
 	$(ECHO_TEXI2MAN) ($(POD2MAN1) $*.pod | sed -e '/^.if n .na/d' > $@.T$$$$ && \
 		mv -f $@.T$$$$ $@) || (rm -f $@.T$$$$ && exit 1)
-- 
2.25.4

[PATCHv2 3/5] gdb/doc: also look in srcdir when running TEXI2POD

[Andrew Burgess]

In gdb/doc/Makefile.in the TEXI2POD variable is used to invoke
texi2pod.pl, which process the .texinfo files.  This also handles the
'include' directives within the .texinfo files.

Like the texi2dvi and texi2pdf tools, texi2pod.pl handles the -I flag
to add search directories for resolving 'include' directives within
.texinfo files.

When GDB runs TEXI2POD we include gdb-cfg.texi, which then includes
GDBvn.texi.

When building from a git checkout the gdb-cfg.texi files and
GDBvn.texi files will be created in the build directory, which is
where texi2pod.pl is invoked, so the files will be found just fine.

However, for a GDB release we ship gdb-cfg.texi and GDBvn.texi in the
source tree, along with the generated manual (.1 and .5) files.

So when building a release, what normally happens is that we spot that
the .1 and .5 man files are up to date, and don't run the recipe to
regenerate these files.

However, if we deliberately touch the *.texinfo files in a release
source tree, and then try to rebuild the man files, we'll get an error
like this:

  make: Entering directory '/tmp/release-build/build/gdb/doc'
    TEXI2POD gdb.1
  cannot find GDBvn.texi at ../../../gdb-16.0.50.20240529/gdb/doc/../../etc/texi2pod.pl line 251, <GEN0> line 16.
  make: *** [Makefile:664: gdb.1] Error 2
  make: Leaving directory '/tmp/release-build/build/gdb/doc'

The problem is that texi2pod.pl doesn't know to look in the source
tree for the GDBvn.texi file.

If we compare this to the recipe for creating (for example) gdb.dvi,
which uses texi2dvi, this recipe adds '-I $(srcdir)' to the texi2dvi
command line, which allows texi2dvi to find GDBvn.texi in the source
tree.

In this commit I add a similar -I option to the texi2pod.pl command
line.  After this, given a GDB release, it is possible to edit (or
just touch) the gdb.texinfo file and rebuild the man pages, the
GDBvn.texi will be picked up from the source tree.

If however a dependency for GDBvn.texi is changed in a release tree
then GDBvn.texi will be regenerated into the build directory and this
will be picked up in preference to the GDBvn.texi in the source tree,
just as you would want.
---
 gdb/doc/Makefile.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gdb/doc/Makefile.in b/gdb/doc/Makefile.in
index e5f8a11008e..e5304404ba8 100644
--- a/gdb/doc/Makefile.in
+++ b/gdb/doc/Makefile.in
@@ -176,7 +176,7 @@ ANNOTATE_DOC_FILES = \
 MANCONF = -Dman
 
 TEXI2POD = perl $(srcdir)/../../etc/texi2pod.pl \
-		$(MAKEINFOFLAGS) $(MAKEINFO_EXTRA_FLAGS)
+		$(MAKEINFOFLAGS) $(MAKEINFO_EXTRA_FLAGS) -I $(srcdir)
 
 # Shell snippet which will provide the full filename of the
 # version.subst file to use.  If there is a copy in the build
-- 
2.25.4

[PATCHv2 4/5] gdb/doc: fix parallel build of refcard related targets

[Andrew Burgess]

There are two problems we encounter when trying to build the refcard
related target in parallel, i.e.:

  $ make -j20 -C gdb/doc/ refcard.dvi refcard.ps refcard.pdf

These problems are:

(1) The refcard.dvi and refcard.pdf targets both try and generate the
    tmp.sed and sedref.tex files.  If two make threads end up trying
    to create these files at the same time then the result is these
    files become corrupted.

    I've fixed this by creating a new rule that creates sedref.tex,
    both refcard.dvi and refcard.pdf now depend on this, and make will
    build sedref.tex just once.  The tmp.sed file is now generated as
    refcard.sed, this is generated and deleted as a temporary file
    within the sedref.tex recipe.

(2) Having created sedref.tex the recipes for refcard.dvi and
    refcard.pdf both run various LaTeX based tools with sedref.tex as
    the input file.  The problem with this is that these tools all
    rely on creating temporary files calls sedref.*.

    If the refcard.dvi and refcard.pdf rules run at the same time then
    these temporary files clash and overwrite each other causing the
    build to fail.

    We already copy the result file in order to rename it, our input
    file is sedref.tex which results in an output file named
    sedref.dvi or sedref.pdf, but we actually want refcard.dvi or
    refcard.pdf.  So within the recipe for refcard.dvi I copy the
    input file from sedref.tex to sedref_dvi.tex.  Now all the temp
    files are named sedref_dvi.* and the output is sedref_dvi.dvi, I
    then rename this new output file to refcard.dvi.

    I've done the same thing for refcard.pdf, but I copy the input
    to sedref_pdf.tex.

    In this way the temp files no longer clash, and both recipes can
    safely run in parallel.

After this commit I was able to reliably build all of the refcard
targets in parallel.  There should be no change in the final file.

Approved-By: Tom Tromey <tom@tromey.com>
---
 gdb/doc/Makefile.in | 39 ++++++++++++++++++++-------------------
 1 file changed, 20 insertions(+), 19 deletions(-)

diff --git a/gdb/doc/Makefile.in b/gdb/doc/Makefile.in
index e5304404ba8..3d64526f542 100644
--- a/gdb/doc/Makefile.in
+++ b/gdb/doc/Makefile.in
@@ -421,31 +421,32 @@ de-stage3: force
 	-(cd stage3 ; mv -f * ..)
 	-rmdir stage3
 
-# GDB QUICK REFERENCE (dvi output)
-refcard.dvi : refcard.tex $(REFEDITS)
-	$(SILENCE) echo > tmp.sed
+
+sedref.tex : refcard.tex $(REFEDITS)
+	$(SILENCE) echo > refcard.sed
 	$(SILENCE) for f in x $(REFEDITS) ; do \
 		test x$$f = xx && continue ; \
-		cat $(srcdir)/$$f >>tmp.sed ; \
+		cat $(srcdir)/$$f >>refcard.sed ; \
 	done
-	$(SILENCE) sed -f tmp.sed $(srcdir)/refcard.tex >sedref.tex
-	$(ECHO_TEX) $(SET_TEXINPUTS) $(TEX) sedref.tex
-	$(SILENCE) mv sedref.dvi refcard.dvi
-	$(SILENCE) rm -f sedref.log sedref.tex tmp.sed
+	$(ECHO_GEN) sed -f refcard.sed $(srcdir)/refcard.tex >$@
+	$(SILENCE) rm -f refcard.sed
+
+
+# GDB QUICK REFERENCE (dvi output)
+refcard.dvi : sedref.tex
+	$(SILENCE) rm -f sedref_dvi.*
+	$(SILENCE) cp $< sedref_dvi.tex
+	$(ECHO_TEX) $(SET_TEXINPUTS) $(TEX) sedref_dvi.tex && \
+		mv sedref_dvi.dvi $@
 
 refcard.ps : refcard.dvi
 	$(ECHO_DVIPS) $(DVIPS) $(SILENT_Q_FLAG) -t landscape -o $@ $?
 
-refcard.pdf : refcard.tex $(REFEDITS)
-	$(SILENCE) echo > tmp.sed
-	$(SILENCE) for f in x $(REFEDITS) ; do \
-		test x$$f = xx && continue ; \
-		cat $(srcdir)/$$f >>tmp.sed ; \
-	done
-	$(SILENCE) sed -f tmp.sed $(srcdir)/refcard.tex >sedref.tex
-	$(ECHO_PDFTEX) $(SET_TEXINPUTS) $(PDFTEX) sedref.tex
-	$(SILENCE) mv sedref.pdf refcard.pdf
-	$(SILENCE) rm -f sedref.log sedref.tex tmp.sed
+refcard.pdf : sedref.tex
+	$(SILENCE) rm -f sedref_pdf.*
+	$(SILENCE) cp $< sedref_pdf.tex
+	$(ECHO_PDFTEX) $(SET_TEXINPUTS) $(PDFTEX) sedref_pdf.tex && \
+		mv sedref_pdf.pdf $@
 
 # File to record current GDB version number.
 GDBvn.texi : version.subst
@@ -691,7 +692,7 @@ mostlyclean:
 	rm -f $(GDB_TEX_TMPS)
 	rm -f $(STABS_TEX_TMPS)
 	rm -f $(ANNOTATE_TEX_TMPS)
-	rm -f sedref.dvi sedref.tex tmp.sed sedref.log
+	rm -f refcard.sed sedref.tex sedref_dvi.* sedref_pdf.*
 	rm -f $(POD_FILE_TMPS)
 
 clean: mostlyclean
-- 
2.25.4

[PATCHv2 5/5] gdb/doc: fix parallel build of pdf and dvi files

[Andrew Burgess]

When building with 'make -j20 -C gdb/doc all-doc' I often see problems
caused from trying to build some dvi files in parallel with some pdf
files.  The problem files are: gdb.dvi and gdb.pdf; stabs.dvi and
stabs.pdf; and annotate.dvi and annotate.pdf.

The problem is that building these files create temporary files in the
local directory.  There's already a race here that two make threads
might try to create these files at the same time.

But it gets worse, to avoid issues where a failed build could leave
these temporary files in a corrupted state, and so prevent the next
build from succeeding, the recipe for each of these files delete all
the temporary files first, this obviously causes problems if some
other thread has already started the build and is relying on these
temporary files.

To work around this problem I propose we start using the --build flag
for texi2dvi (which is the same tool used to create the pdf files).
The --build flag allows for the temporary files to be placed into a
sub-directory, e.g. creating gdb.pdf will create the temporary files
in gdb.t2d/pdf/ and gdb.dvi uses gdb.t2d/dvi/, in this way the
temporary files will never clash, and we can easily delete the
specific sub-directory at the start of the recipe to ensure a
successful build.

Very old versions of texi2dvi don't support --build, so I've added a
configure check for this option.  If the option is not supported then
we don't use it.  This means we fall back to placing temporary files
in the local directory, which means parallel builds will remain
broken.  The --build option is definitely present in texi2dvi version
6.1, from 2016, so any version after that should be fine.

For me, with this fix in place, I can now run 'make -j20 -C gdb/doc
all-doc' without seeing any build problems.

Approved-By: Tom Tromey <tom@tromey.com>
---
 gdb/configure       | 33 +++++++++++++++++++++++++++++++--
 gdb/configure.ac    | 21 +++++++++++++++++++++
 gdb/doc/Makefile.in | 42 +++++++++++++++++++++++++++++-------------
 3 files changed, 81 insertions(+), 15 deletions(-)

diff --git a/gdb/configure b/gdb/configure
index 66a7ad8d256..8d870d50de3 100755
--- a/gdb/configure
+++ b/gdb/configure
@@ -765,6 +765,7 @@ ENABLE_BFD_64_BIT_TRUE
 subdirs
 GDB_DATADIR
 DEBUGDIR
+TEXI2DVI_EXTRA_FLAGS
 MAKEINFO_EXTRA_FLAGS
 MAKEINFOFLAGS
 MAKEINFO
@@ -11499,7 +11500,7 @@ else
   lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
   lt_status=$lt_dlunknown
   cat > conftest.$ac_ext <<_LT_EOF
-#line 11502 "configure"
+#line 11503 "configure"
 #include "confdefs.h"
 
 #if HAVE_DLFCN_H
@@ -11605,7 +11606,7 @@ else
   lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
   lt_status=$lt_dlunknown
   cat > conftest.$ac_ext <<_LT_EOF
-#line 11608 "configure"
+#line 11609 "configure"
 #include "confdefs.h"
 
 #if HAVE_DLFCN_H
@@ -24609,6 +24610,34 @@ if test x"$gdb_cv_have_makeinfo_click" = xyes; then
 fi
 
 
+## Figure out if texi2dvi supports the --build option.  Older versions
+## didn't.  If this option is not supported then parallel building in
+## the docs/ directory is probably going to be broken.  But single
+## threaded build should be fine.
+TEXI2DVI=texi2dvi
+TEXI2DVI_EXTRA_FLAGS=""
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether $TEXI2DVI supports --build" >&5
+$as_echo_n "checking whether $TEXI2DVI supports --build... " >&6; }
+if ${gdb_cv_have_texi2dvi_build_opt+:} false; then :
+  $as_echo_n "(cached) " >&6
+else
+  echo '\input texinfo' >conftest.texi
+   echo 'abc' >>conftest.texi
+   echo '@bye' >>conftest.texi
+   cp conftest.texi /home/andrew/tmp/
+   if eval "$TEXI2DVI --build=tidy conftest.texi >&5 2>&5"; then
+     gdb_cv_have_texi2dvi_build_opt=yes
+   else
+     gdb_cv_have_texi2dvi_build_opt=no
+   fi
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $gdb_cv_have_texi2dvi_build_opt" >&5
+$as_echo "$gdb_cv_have_texi2dvi_build_opt" >&6; }
+if test x"$gdb_cv_have_texi2dvi_build_opt" = xyes; then
+  TEXI2DVI_EXTRA_FLAGS="$TEXI2DVI_EXTRA_FLAGS --build=tidy"
+fi
+
+
 
 
 # Check whether --with-separate-debug-dir was given.
diff --git a/gdb/configure.ac b/gdb/configure.ac
index 62ff09cea20..3b726e5f0aa 100644
--- a/gdb/configure.ac
+++ b/gdb/configure.ac
@@ -117,6 +117,27 @@ if test x"$gdb_cv_have_makeinfo_click" = xyes; then
 fi
 AC_SUBST(MAKEINFO_EXTRA_FLAGS)
 
+## Figure out if texi2dvi supports the --build option.  Older versions
+## didn't.  If this option is not supported then parallel building in
+## the docs/ directory is probably going to be broken.  But single
+## threaded build should be fine.
+TEXI2DVI=texi2dvi
+TEXI2DVI_EXTRA_FLAGS=""
+AC_CACHE_CHECK([whether $TEXI2DVI supports --build], gdb_cv_have_texi2dvi_build_opt,
+  [echo '\input texinfo' >conftest.texi
+   echo 'abc' >>conftest.texi
+   echo '@bye' >>conftest.texi
+   cp conftest.texi /home/andrew/tmp/
+   if eval "$TEXI2DVI --build=tidy conftest.texi >&5 2>&5"; then
+     gdb_cv_have_texi2dvi_build_opt=yes
+   else
+     gdb_cv_have_texi2dvi_build_opt=no
+   fi])
+if test x"$gdb_cv_have_texi2dvi_build_opt" = xyes; then
+  TEXI2DVI_EXTRA_FLAGS="$TEXI2DVI_EXTRA_FLAGS --build=tidy"
+fi
+AC_SUBST(TEXI2DVI_EXTRA_FLAGS)
+
 GDB_AC_WITH_DIR(DEBUGDIR, separate-debug-dir,
     [look for global separate debug info in this path @<:@LIBDIR/debug@:>@],
     [${libdir}/debug])
diff --git a/gdb/doc/Makefile.in b/gdb/doc/Makefile.in
index 3d64526f542..cdb046bd90f 100644
--- a/gdb/doc/Makefile.in
+++ b/gdb/doc/Makefile.in
@@ -68,6 +68,8 @@ TEXI2ROFF=texi2roff
 
 # where to find texi2dvi, ditto
 TEXI2DVI=texi2dvi
+TEXI2DVI_EXTRA_FLAGS=@TEXI2DVI_EXTRA_FLAGS@
+TEXI2DVI_CMD = $(TEXI2DVI) $(TEXI2DVI_EXTRA_FLAGS)
 
 # Package to install the docs under
 PACKAGE = @PACKAGE@
@@ -492,21 +494,26 @@ gdb-cfg.texi: ${srcdir}/${DOC_CONFIG}-cfg.texi
 # Clean these up before each run.  Avoids a catch 22 with not being
 # able to re-generate these files (to fix a corruption) because these
 # files contain a corruption.
+#
+# If TEXI2DVI_CMD includes the --build option then these files should
+# not be created in the current directory, but attempting to delete
+# them is harmless.
 GDB_TEX_TMPS = gdb.aux gdb.cp* gdb.fn* gdb.ky* gdb.log gdb.pg* gdb.toc \
 	gdb.tp* gdb.vr*
 
 # GDB MANUAL: TeX dvi file
 gdb.dvi: ${GDB_DOC_FILES}
-	$(SILENCE) rm -f $(GDB_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) $(READLINE_TEXI_INCFLAG) \
-		-I ${GDBMI_DIR} -I $(srcdir) $(srcdir)/gdb.texinfo
+	$(SILENCE) rm -fr $(GDB_TEX_TMPS) gdb.t2d/dvi/
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) \
+		$(READLINE_TEXI_INCFLAG) -I ${GDBMI_DIR} -I $(srcdir) \
+		$(srcdir)/gdb.texinfo
 
 gdb.ps: gdb.dvi
 	$(ECHO_DVIPS) $(DVIPS) $(SILENT_Q_FLAG) -o $@ $?
 
 gdb.pdf: ${GDB_DOC_FILES}
-	$(SILENCE) rm -f $(GDB_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) --pdf \
+	$(SILENCE) rm -fr $(GDB_TEX_TMPS) gdb.t2d/pdf/
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) --pdf \
 		$(READLINE_TEXI_INCFLAG) -I ${GDBMI_DIR} -I $(srcdir) \
 		$(srcdir)/gdb.texinfo
 
@@ -607,41 +614,49 @@ stabs/index.html: $(STABS_DOC_FILES)
 # Clean these up before each run.  Avoids a catch 22 with not being
 # able to re-generate these files (to fix a corruption) because these
 # files contain a corruption.
+#
+# If TEXI2DVI_CMD includes the --build option then these files should
+# not be created in the current directory, but attempting to delete
+# them is harmless.
 STABS_TEX_TMPS = stabs.aux stabs.cp* stabs.fn* stabs.ky* \
 	stabs.log stabs.pg* stabs.toc stabs.tp* stabs.vr*
 
 # STABS DOCUMENTATION: TeX dvi file
 stabs.dvi : $(STABS_DOC_FILES)
-	$(SILENCE) rm -f $(STABS_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) -I $(srcdir) \
+	$(SILENCE) rm -fr $(STABS_TEX_TMPS) stabs.t2d/dvi/
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) -I $(srcdir) \
 		$(srcdir)/stabs.texinfo
 
 stabs.ps: stabs.dvi
 	$(ECHO_DVIPS) $(DVIPS) $(SILENT_Q_FLAG) -o $@ $?
 
 stabs.pdf: $(STABS_DOC_FILES)
-	$(SILENCE) rm -f $(STABS_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) --pdf -I $(srcdir) \
+	$(SILENCE) rm -fr $(STABS_TEX_TMPS) stabs.t2d/pdf/
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) --pdf -I $(srcdir) \
 		$(srcdir)/stabs.texinfo
 
 # Clean these up before each run.  Avoids a catch 22 with not being
 # able to re-generate these files (to fix a corruption) because these
 # files contain a corruption.
+#
+# If TEXI2DVI_CMD includes the --build option then these files should
+# not be created in the current directory, but attempting to delete
+# them is harmless.
 ANNOTATE_TEX_TMPS = annotate.aux annotate.cp* annotate.fn* annotate.ky* \
 	annotate.log annotate.pg* annotate.toc annotate.tp* annotate.vr*
 
 # ANNOTATE DOCUMENTATION: TeX dvi file
 annotate.dvi : $(ANNOTATE_DOC_FILES)
-	$(SILENCE) rm -f $(ANNOTATE_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) -I $(srcdir) \
+	$(SILENCE) rm -fr $(ANNOTATE_TEX_TMPS) annotate.t2d/dvi/
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) -I $(srcdir) \
 		$(srcdir)/annotate.texinfo
 
 annotate.ps: annotate.dvi
 	$(ECHO_DVIPS) $(DVIPS) $(SILENT_Q_FLAG) -o $@ $?
 
 annotate.pdf: $(ANNOTATE_DOC_FILES)
-	$(SILENCE) rm -f $(ANNOTATE_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) --pdf -I $(srcdir) \
+	$(SILENCE) rm -fr $(ANNOTATE_TEX_TMPS) annotate.t2d/pdf/
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) --pdf -I $(srcdir) \
 		$(srcdir)/annotate.texinfo
 
 annotate.info: $(ANNOTATE_DOC_FILES)
@@ -689,6 +704,7 @@ Makefile: Makefile.in $(host_makefile_frag) ../config.status
 
 mostlyclean:
 	rm -f gdb.mm gdb.ms gdb.me links2roff
+	rm -fr annotate.t2d/ stabs.t2d/ gdb.t2d/
 	rm -f $(GDB_TEX_TMPS)
 	rm -f $(STABS_TEX_TMPS)
 	rm -f $(ANNOTATE_TEX_TMPS)
-- 
2.25.4

Re: [PATCHv2 2/5] gdb/doc: allow for version.subst in the source tree

[Eli Zaretskii]

> From: Andrew Burgess <aburgess@redhat.com>
> Cc: Andrew Burgess <aburgess@redhat.com>
> Date: Fri, 31 May 2024 09:18:39 +0100
> 
> In a git checkout of the source code we don't have a version.subst
> file in the gdb/doc directory.  When building the GDB docs the
> version.subst file is generated on demand (we have a recipe for that).
> 
> However, in a release tar file we do include a copy of the
> version.subst file in the source tree, as a result the version.subst
> recipe will not be run.
> 
> If, in a release build, we force the running of any recipe that
> depends on version.subst then we run into a problem.  For example,
> slightly confusingly, if we 'touch gdb/doc/version.subst' within the
> unpacked source tree of a release, then 'make -C gdb/doc GDBvn.texi'
> in the build tree, we'll see:
> 
>   make: Entering directory '/tmp/build/build/gdb/doc'
>     GEN    GDBvn.texi
>   sed: can't read version.subst: No such file or directory
>   make: Leaving directory '/tmp/build/build/gdb/doc'
> 
> The problem is that every reference to version.subst in GDB's Makefile
> assumes that the version.subst file will always be in the build
> directory.
> 
> In this commit I replace direct references to version.subst with a
> small shell snippet which returns the path to the version.subst file
> to use.  If there is a version.subst file in the build directory then
> that file is selected, otherwise we return the path to a version.subst
> file in the source directory.

Why isn't VPATH or vpath enough to solve this nuisance?

Alternatively, how about copying version.subst from the source tree to
the build tree if it is absent from the build tree?

Thanks.

Re: [PATCHv2 2/5] gdb/doc: allow for version.subst in the source tree

[Andrew Burgess]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrew Burgess <aburgess@redhat.com>
>> Cc: Andrew Burgess <aburgess@redhat.com>
>> Date: Fri, 31 May 2024 09:18:39 +0100
>> 
>> In a git checkout of the source code we don't have a version.subst
>> file in the gdb/doc directory.  When building the GDB docs the
>> version.subst file is generated on demand (we have a recipe for that).
>> 
>> However, in a release tar file we do include a copy of the
>> version.subst file in the source tree, as a result the version.subst
>> recipe will not be run.
>> 
>> If, in a release build, we force the running of any recipe that
>> depends on version.subst then we run into a problem.  For example,
>> slightly confusingly, if we 'touch gdb/doc/version.subst' within the
>> unpacked source tree of a release, then 'make -C gdb/doc GDBvn.texi'
>> in the build tree, we'll see:
>> 
>>   make: Entering directory '/tmp/build/build/gdb/doc'
>>     GEN    GDBvn.texi
>>   sed: can't read version.subst: No such file or directory
>>   make: Leaving directory '/tmp/build/build/gdb/doc'
>> 
>> The problem is that every reference to version.subst in GDB's Makefile
>> assumes that the version.subst file will always be in the build
>> directory.
>> 
>> In this commit I replace direct references to version.subst with a
>> small shell snippet which returns the path to the version.subst file
>> to use.  If there is a version.subst file in the build directory then
>> that file is selected, otherwise we return the path to a version.subst
>> file in the source directory.
>
> Why isn't VPATH or vpath enough to solve this nuisance?

If anything VPATH is the _cause_ of this nuisance!  With the VPATH set
as it is, make will check both the build directory and the source
directory.  So when we have a rule like:

  some_target : version.subst
     some_command version.subst

make is going to look for the version.subst dependency in both
locations, however, where 'some_command' looks is entirely up to
'some_command'.  In our case for example, the 'sed' command doesn't (to
my knowledge) have a search path command line option, so a command like:

  sed q version.subst

is only going to search in the build directory.

Another file that has this problem is GDBvn.texi, however, all the
commands that consume this file support a '-I dir' command line flag to
add a search path, and so for those commands we add '-I $srcdir' so the
command will look in the source directory.

> Alternatively, how about copying version.subst from the source tree to
> the build tree if it is absent from the build tree?

That for sure would be possible.  We would need to take care to ensure
that the modification time on the file wasn't changed, e.g. if we had to
copy from one filesystem to another, as GDBvn.texi depends on
version.subst, and most of the other doc files depend on GDBvn.texi, and
as you pointed out in an earlier email, we don't want to end up
triggering a doc rebuild for a release build/install.

The other problem is copying/moving the file from the source tree is for
parallel builds.  Multiple recipes use version.subst, so each of those
would I think need to include the logic for copying the file.  And would
need to ensure that the copy was (a) atomic, and (b) didn't result in an
error if another recipe completed before us... otherwise we'd see random
errors from a parallel build.

I did wonder if we could create a hard dependency on
$(builddir)/version.subst, but I have no idea how I could make that
work, we need to copy the file from the $(srcdir) only if its up to
date, otherwise we need to rebuild the file.  And we'd still need to
solve the problem that if targets depended on $(builddir)/version.subst
which would certainly not exist in a release, then this would trigger a
complete rebuild of the docs, which violates a hard requirement.

I agree that the solution proposed here is not ideal ... but I'm not
sure anything else is better.

Thanks,
Andrew

Re: [PATCHv2 2/5] gdb/doc: allow for version.subst in the source tree

[Eli Zaretskii]

> From: Andrew Burgess <aburgess@redhat.com>
> Cc: gdb-patches@sourceware.org
> Date: Mon, 03 Jun 2024 15:22:41 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >>   make: Entering directory '/tmp/build/build/gdb/doc'
> >>     GEN    GDBvn.texi
> >>   sed: can't read version.subst: No such file or directory
> >>   make: Leaving directory '/tmp/build/build/gdb/doc'
> >> 
> >> The problem is that every reference to version.subst in GDB's Makefile
> >> assumes that the version.subst file will always be in the build
> >> directory.
> >> 
> >> In this commit I replace direct references to version.subst with a
> >> small shell snippet which returns the path to the version.subst file
> >> to use.  If there is a version.subst file in the build directory then
> >> that file is selected, otherwise we return the path to a version.subst
> >> file in the source directory.
> >
> > Why isn't VPATH or vpath enough to solve this nuisance?
> 
> If anything VPATH is the _cause_ of this nuisance!  With the VPATH set
> as it is, make will check both the build directory and the source
> directory.  So when we have a rule like:
> 
>   some_target : version.subst
>      some_command version.subst
> 
> make is going to look for the version.subst dependency in both
> locations, however, where 'some_command' looks is entirely up to
> 'some_command'.  In our case for example, the 'sed' command doesn't (to
> my knowledge) have a search path command line option, so a command like:
> 
>   sed q version.subst
> 
> is only going to search in the build directory.

My reading of the node "Recipes/Search" in the GNU Make manual is that
if you use automatic variables, such as $^, in the recipe, Make will
expand the variable into the full file name including the directory in
which it found the file via VPATH search.  Can't we use that feature
here?  IOW, instead of

  some_target : version.subst
     some_command version.subst

you should use something like

  some_target : version.subst
     some_command $<

> I agree that the solution proposed here is not ideal ... but I'm not
> sure anything else is better.

How about the above, assuming it works?

[PATCHv3 0/6] gdb/doc: parallel build fixes and improvements

[Andrew Burgess]

In v3:

  - Add new patch #2 which merges the two man page rules into one.

  - Updated patch #3 based on Eli's feedback.  Now use $< instead of a
    shell snippet to locate version.subst.

In V2:

  - Patch #1: the content of this patch is unchanged from v1, however
    the commit message is completely rewritten.  I believe the change
    I made was good, but my previous justification was completely
    wrong!

  - Patches #2 and #3: these patches are completely new in v2.

  - Patches #4 and #5: these are completely unchanged from v1 where
    they were already approved.  Further feedback is of course
    welcome, but if you want to skip looking at these, that's fine
    too.

Thanks,
Andrew

---

Andrew Burgess (6):
  gdb/doc: don't try to copy GDBvn.texi from the source tree
  gdb/doc: merge rules for building .1 and .5 man pages
  gdb/doc: allow for version.subst in the source tree
  gdb/doc: also look in srcdir when running TEXI2POD
  gdb/doc: fix parallel build of refcard related targets
  gdb/doc: fix parallel build of pdf and dvi files

 gdb/configure       |  33 ++++++++++-
 gdb/configure.ac    |  21 +++++++
 gdb/doc/Makefile.in | 135 +++++++++++++++++++++++---------------------
 3 files changed, 122 insertions(+), 67 deletions(-)


base-commit: de4edfcb949c0d04ec8b49fdfe06267f92618589
-- 
2.25.4

[PATCHv3 1/6] gdb/doc: don't try to copy GDBvn.texi from the source tree

[Andrew Burgess]

The build recipe for gdb.dvi and gdb.pdf contains instructions for
copying the GDBvn.texi file from the source tree into the build
directory if the GDBvn.texi file doesn't already exist in the build
directory.

The gdb.dvi and gdb.pdf targets also have a dependency on GDBvn.texi,
and we have a recipe for building GDBvn.texi.

What's happening here is this:

  - In a git checkout of the source tree there is no GDBvn.texi in the
    source tree, the GDBvn.texi dependency will trigger a rebuild of
    GDBvn.texi, which is then used to build gdb.dvi and/or gdb.pdf.

  - In a release tar file we do include a copy of GDBvn.texi.  This
    file will appear to be up to date, and so no copy of GDBvn.texi is
    created within the build directory.  Now when building gdb.dvi
    and/or gdb.pdf we copy (or symlink) the version of GDBvn.texi from
    the source tree into the build directory.

However, copying GDBvn.texi into the source directory is completely
unnecessary.  The gdb.dvi/gdb.pdf recipes both invoke texi2dvi and
pass '-I $(srcdir)' as an argument, this means that texi2dvi will look
in the $(srcdir) to find included files, including GDBvn.texi.

As such I believe we can remove the code that copies GDBvn.texi from
the source tree into the build tree.

I've tested with a release build; creating a release with:

  ./src-release gdb

Then in an empty directory, unpacking the resulting .tar file,
creating a parallel build directory and doing the usual configure,
make, and 'make install'.

Having done this I can then 'touch gdb/doc/*.texinfo' in the unpacked
source tree, and do 'make -C gdb/doc pdf dvi' to rebuild all the pdf
and dvi files, this works fine without having to either build or copy
GDBvn.texi into the build directory.
---
 gdb/doc/Makefile.in | 17 -----------------
 1 file changed, 17 deletions(-)

diff --git a/gdb/doc/Makefile.in b/gdb/doc/Makefile.in
index d9c13873a60..1a0791d518f 100644
--- a/gdb/doc/Makefile.in
+++ b/gdb/doc/Makefile.in
@@ -477,15 +477,6 @@ gdb-cfg.texi: ${srcdir}/${DOC_CONFIG}-cfg.texi
 	ln ${srcdir}/${DOC_CONFIG}-cfg.texi gdb-cfg.texi || \
 	cp ${srcdir}/${DOC_CONFIG}-cfg.texi gdb-cfg.texi
 
-# GDB MANUAL: texinfo source, using @set/@clear/@value/@ifset/@ifclear
-# If your texinfo or makeinfo don't support these, get a new texinfo release
-#
-# The nonsense with GDBvn.texi gets this to run with both Sun and GNU make.
-#   Note that we can *generate* GDBvn.texi, but since we distribute one in the
-#   source directory for the benefit of people who *don't* use this makefile,
-#   VPATH will often tell make not to bother building it, because the one
-#   in the srcdir is up to date.  (if not, then make should build one here).
-
 # Clean these up before each run.  Avoids a catch 22 with not being
 # able to re-generate these files (to fix a corruption) because these
 # files contain a corruption.
@@ -494,10 +485,6 @@ GDB_TEX_TMPS = gdb.aux gdb.cp* gdb.fn* gdb.ky* gdb.log gdb.pg* gdb.toc \
 
 # GDB MANUAL: TeX dvi file
 gdb.dvi: ${GDB_DOC_FILES}
-	$(SILENCE) if [ ! -f ./GDBvn.texi ]; then \
-		(test "$(LN_S)" = "ln -s" && ln -s $(srcdir)/GDBvn.texi .) || \
-		ln $(srcdir)/GDBvn.texi . || \
-		cp $(srcdir)/GDBvn.texi . ; else true; fi
 	$(SILENCE) rm -f $(GDB_TEX_TMPS)
 	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) $(READLINE_TEXI_INCFLAG) \
 		-I ${GDBMI_DIR} -I $(srcdir) $(srcdir)/gdb.texinfo
@@ -506,10 +493,6 @@ gdb.ps: gdb.dvi
 	$(ECHO_DVIPS) $(DVIPS) $(SILENT_Q_FLAG) -o $@ $?
 
 gdb.pdf: ${GDB_DOC_FILES}
-	$(SILENCE) if [ ! -f ./GDBvn.texi ]; then \
-		(test "$(LN_S)" = "ln -s" && ln -s $(srcdir)/GDBvn.texi .) || \
-		ln $(srcdir)/GDBvn.texi . || \
-		cp $(srcdir)/GDBvn.texi . ; else true; fi
 	$(SILENCE) rm -f $(GDB_TEX_TMPS)
 	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) --pdf \
 		$(READLINE_TEXI_INCFLAG) -I ${GDBMI_DIR} -I $(srcdir) \
-- 
2.25.4

[PATCHv3 2/6] gdb/doc: merge rules for building .1 and .5 man pages

[Andrew Burgess]

We have two rules, one each for building the .1 and .5 man pages.  The
only actual difference is that one rule passes --section=1 and the
other passes --section=5 (see the definitions of POD2MAN1 and POD2MAN5
respectively.

I figure by using the suffix from the target of the rule we can
combine these two rules into one.

I use:

  $(subst .,,$(suffix $@))

This gets the suffix from the target, either '.1' or '.5', and the
'subst' removes the '.' leaving '1' or '5'.

Now that I'm not using a static pattern rule for building the man
pages, the advice in the 'make' documentation is to not use $*, so
I've moved away from that to instead use $(basename $@), e.g. for
'gdbinit.5' this gives 'gdbinit', which is what we want.

There should be no difference in what is created after this change.
---
 gdb/doc/Makefile.in | 23 +++++++++--------------
 1 file changed, 9 insertions(+), 14 deletions(-)

diff --git a/gdb/doc/Makefile.in b/gdb/doc/Makefile.in
index 1a0791d518f..70d65d58ed0 100644
--- a/gdb/doc/Makefile.in
+++ b/gdb/doc/Makefile.in
@@ -178,10 +178,8 @@ MANCONF = -Dman
 TEXI2POD = perl $(srcdir)/../../etc/texi2pod.pl \
 		$(MAKEINFOFLAGS) $(MAKEINFO_EXTRA_FLAGS)
 
-POD2MAN1 = pod2man --center="GNU Development Tools" \
-		   --release="gdb-`sed q version.subst`" --section=1
-POD2MAN5 = pod2man --center="GNU Development Tools" \
-		   --release="gdb-`sed q version.subst`" --section=5
+POD2MAN = pod2man --center="GNU Development Tools" \
+		--release="gdb-`sed q version.subst`"
 
 # List of man pages generated from gdb.texi
 MAN1S = gdb.1 gdbserver.1 gcore.1 gdb-add-index.1
@@ -654,17 +652,14 @@ annotate/index.html: $(ANNOTATE_DOC_FILES)
 # pages, then the .pod files must become a dependency, this will
 # trigger an attempt to rebuild these files while building and
 # installing a release of GDB, which is something we don't want.
-$(MAN1S) : %.1 : $(GDB_DOC_FILES)
-	$(ECHO_TEXI2POD) $(TEXI2POD) $(MANCONF) -D$* < $(srcdir)/gdb.texinfo > $*.pod
-	$(ECHO_TEXI2MAN) ($(POD2MAN1) $*.pod | sed -e '/^.if n .na/d' > $@.T$$$$ && \
+$(MAN1S) $(MAN5S) : version.subst $(GDB_DOC_FILES)
+	$(ECHO_TEXI2POD) $(TEXI2POD) $(MANCONF) -D$(basename $@) \
+		< $(srcdir)/gdb.texinfo > $(basename $@).pod
+	$(ECHO_TEXI2MAN) ($(POD2MAN) --section=$(subst .,,$(suffix $@)) \
+			$(basename $@).pod | \
+		sed -e '/^.if n .na/d' > $@.T$$$$ && \
 		mv -f $@.T$$$$ $@) || (rm -f $@.T$$$$ && exit 1)
-	$(SILENCE) rm -f $*.pod
-
-$(MAN5S) : %.5 : $(GDB_DOC_FILES)
-	$(ECHO_TEXI2POD) $(TEXI2POD) $(MANCONF) -D$* < $(srcdir)/gdb.texinfo > $*.pod
-	$(ECHO_TEXI2MAN) ($(POD2MAN5) $*.pod | sed -e '/^.if n .na/d' > $@.T$$$$ && \
-		mv -f $@.T$$$$ $@) || (rm -f $@.T$$$$ && exit 1)
-	$(SILENCE) rm -f $*.pod
+	$(SILENCE) rm -f $(basename $@).pod
 
 force:
 
-- 
2.25.4

[PATCHv3 3/6] gdb/doc: allow for version.subst in the source tree

[Andrew Burgess]

In a git checkout of the source code we don't have a version.subst
file in the gdb/doc directory.  When building the GDB docs the
version.subst file is generated on demand (we have a recipe for that).

However, in a release tar file we do include a copy of the
version.subst file in the source tree, as a result the version.subst
recipe will not be run.

If, in a release build, we force the running of any recipe that
depends on version.subst then we run into a problem.  For example,
slightly confusingly, if we 'touch gdb/doc/version.subst' within the
unpacked source tree of a release, then 'make -C gdb/doc GDBvn.texi'
in the build tree, we'll see:

  make: Entering directory '/tmp/build/build/gdb/doc'
    GEN    GDBvn.texi
  sed: can't read version.subst: No such file or directory
  make: Leaving directory '/tmp/build/build/gdb/doc'

The problem is that every reference to version.subst in GDB's Makefile
assumes that the version.subst file will always be in the build
directory.

Handily version.subst is always the first dependency in every recipe
that uses that file.  As such we can replace references to
version.subst with $<, make will expand this to the location where the
dependency was found.

In the case of the man page generation, the reference to version.subst
is hidden inside POD2MAN.  It seemed a little confusing adding a use
of  $< within POD2MAN, so I've moved the use into the recipe, which I
think is clearer.

I've also added comments for the two rules that I've modified to
explain our use of $<.

After this change it is possible to rebuild the man pages even when
version.subst is located in the source tree.
---
 gdb/doc/Makefile.in | 18 ++++++++++++++----
 1 file changed, 14 insertions(+), 4 deletions(-)

diff --git a/gdb/doc/Makefile.in b/gdb/doc/Makefile.in
index 70d65d58ed0..eb800bf3848 100644
--- a/gdb/doc/Makefile.in
+++ b/gdb/doc/Makefile.in
@@ -178,8 +178,8 @@ MANCONF = -Dman
 TEXI2POD = perl $(srcdir)/../../etc/texi2pod.pl \
 		$(MAKEINFOFLAGS) $(MAKEINFO_EXTRA_FLAGS)
 
-POD2MAN = pod2man --center="GNU Development Tools" \
-		--release="gdb-`sed q version.subst`"
+
+POD2MAN = pod2man --center="GNU Development Tools"
 
 # List of man pages generated from gdb.texi
 MAN1S = gdb.1 gdbserver.1 gcore.1 gdb-add-index.1
@@ -435,9 +435,14 @@ refcard.pdf : refcard.tex $(REFEDITS)
 	$(SILENCE) rm -f sedref.log sedref.tex tmp.sed
 
 # File to record current GDB version number.
+#
+# It is important that version.subst be the first dependency so that
+# $< can be used in the recipe, this allows us to find version.subst
+# in either the source tree or the build tree as this file is
+# included, pre-built, as part of a relase.
 GDBvn.texi : version.subst
 	$(ECHO_GEN)
-	$(SILENCE) echo "@set GDBVN `sed q version.subst`" > ./GDBvn.new
+	$(SILENCE) echo "@set GDBVN `sed q $<`" > ./GDBvn.new
 	$(SILENCE) if [ -n "$(PKGVERSION)" ]; then \
 	  echo "@set VERSION_PACKAGE $(PKGVERSION)" >> ./GDBvn.new; \
 	fi
@@ -652,11 +657,16 @@ annotate/index.html: $(ANNOTATE_DOC_FILES)
 # pages, then the .pod files must become a dependency, this will
 # trigger an attempt to rebuild these files while building and
 # installing a release of GDB, which is something we don't want.
+#
+# It is important that version.subst be the first dependency so that
+# $< can be used in the recipe, this allows us to find version.subst
+# in either the source tree or the build tree as this file is
+# included, pre-built, as part of a relase.
 $(MAN1S) $(MAN5S) : version.subst $(GDB_DOC_FILES)
 	$(ECHO_TEXI2POD) $(TEXI2POD) $(MANCONF) -D$(basename $@) \
 		< $(srcdir)/gdb.texinfo > $(basename $@).pod
 	$(ECHO_TEXI2MAN) ($(POD2MAN) --section=$(subst .,,$(suffix $@)) \
-			$(basename $@).pod | \
+			--release="gdb-`sed q $<`" $(basename $@).pod | \
 		sed -e '/^.if n .na/d' > $@.T$$$$ && \
 		mv -f $@.T$$$$ $@) || (rm -f $@.T$$$$ && exit 1)
 	$(SILENCE) rm -f $(basename $@).pod
-- 
2.25.4

[PATCHv3 4/6] gdb/doc: also look in srcdir when running TEXI2POD

[Andrew Burgess]

In gdb/doc/Makefile.in the TEXI2POD variable is used to invoke
texi2pod.pl, which process the .texinfo files.  This also handles the
'include' directives within the .texinfo files.

Like the texi2dvi and texi2pdf tools, texi2pod.pl handles the -I flag
to add search directories for resolving 'include' directives within
.texinfo files.

When GDB runs TEXI2POD we include gdb-cfg.texi, which then includes
GDBvn.texi.

When building from a git checkout the gdb-cfg.texi files and
GDBvn.texi files will be created in the build directory, which is
where texi2pod.pl is invoked, so the files will be found just fine.

However, for a GDB release we ship gdb-cfg.texi and GDBvn.texi in the
source tree, along with the generated manual (.1 and .5) files.

So when building a release, what normally happens is that we spot that
the .1 and .5 man files are up to date, and don't run the recipe to
regenerate these files.

However, if we deliberately touch the *.texinfo files in a release
source tree, and then try to rebuild the man files, we'll get an error
like this:

  make: Entering directory '/tmp/release-build/build/gdb/doc'
    TEXI2POD gdb.1
  cannot find GDBvn.texi at ../../../gdb-16.0.50.20240529/gdb/doc/../../etc/texi2pod.pl line 251, <GEN0> line 16.
  make: *** [Makefile:664: gdb.1] Error 2
  make: Leaving directory '/tmp/release-build/build/gdb/doc'

The problem is that texi2pod.pl doesn't know to look in the source
tree for the GDBvn.texi file.

If we compare this to the recipe for creating (for example) gdb.dvi,
which uses texi2dvi, this recipe adds '-I $(srcdir)' to the texi2dvi
command line, which allows texi2dvi to find GDBvn.texi in the source
tree.

In this commit I add a similar -I option to the texi2pod.pl command
line.  After this, given a GDB release, it is possible to edit (or
just touch) the gdb.texinfo file and rebuild the man pages, the
GDBvn.texi will be picked up from the source tree.

If however a dependency for GDBvn.texi is changed in a release tree
then GDBvn.texi will be regenerated into the build directory and this
will be picked up in preference to the GDBvn.texi in the source tree,
just as you would want.
---
 gdb/doc/Makefile.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gdb/doc/Makefile.in b/gdb/doc/Makefile.in
index eb800bf3848..7b7b85dae10 100644
--- a/gdb/doc/Makefile.in
+++ b/gdb/doc/Makefile.in
@@ -176,7 +176,7 @@ ANNOTATE_DOC_FILES = \
 MANCONF = -Dman
 
 TEXI2POD = perl $(srcdir)/../../etc/texi2pod.pl \
-		$(MAKEINFOFLAGS) $(MAKEINFO_EXTRA_FLAGS)
+		$(MAKEINFOFLAGS) $(MAKEINFO_EXTRA_FLAGS) -I $(srcdir)
 
 
 POD2MAN = pod2man --center="GNU Development Tools"
-- 
2.25.4

[PATCHv3 5/6] gdb/doc: fix parallel build of refcard related targets

[Andrew Burgess]

There are two problems we encounter when trying to build the refcard
related target in parallel, i.e.:

  $ make -j20 -C gdb/doc/ refcard.dvi refcard.ps refcard.pdf

These problems are:

(1) The refcard.dvi and refcard.pdf targets both try and generate the
    tmp.sed and sedref.tex files.  If two make threads end up trying
    to create these files at the same time then the result is these
    files become corrupted.

    I've fixed this by creating a new rule that creates sedref.tex,
    both refcard.dvi and refcard.pdf now depend on this, and make will
    build sedref.tex just once.  The tmp.sed file is now generated as
    refcard.sed, this is generated and deleted as a temporary file
    within the sedref.tex recipe.

(2) Having created sedref.tex the recipes for refcard.dvi and
    refcard.pdf both run various LaTeX based tools with sedref.tex as
    the input file.  The problem with this is that these tools all
    rely on creating temporary files calls sedref.*.

    If the refcard.dvi and refcard.pdf rules run at the same time then
    these temporary files clash and overwrite each other causing the
    build to fail.

    We already copy the result file in order to rename it, our input
    file is sedref.tex which results in an output file named
    sedref.dvi or sedref.pdf, but we actually want refcard.dvi or
    refcard.pdf.  So within the recipe for refcard.dvi I copy the
    input file from sedref.tex to sedref_dvi.tex.  Now all the temp
    files are named sedref_dvi.* and the output is sedref_dvi.dvi, I
    then rename this new output file to refcard.dvi.

    I've done the same thing for refcard.pdf, but I copy the input
    to sedref_pdf.tex.

    In this way the temp files no longer clash, and both recipes can
    safely run in parallel.

After this commit I was able to reliably build all of the refcard
targets in parallel.  There should be no change in the final file.

Approved-By: Tom Tromey <tom@tromey.com>
---
 gdb/doc/Makefile.in | 39 ++++++++++++++++++++-------------------
 1 file changed, 20 insertions(+), 19 deletions(-)

diff --git a/gdb/doc/Makefile.in b/gdb/doc/Makefile.in
index 7b7b85dae10..d294120ad19 100644
--- a/gdb/doc/Makefile.in
+++ b/gdb/doc/Makefile.in
@@ -408,31 +408,32 @@ de-stage3: force
 	-(cd stage3 ; mv -f * ..)
 	-rmdir stage3
 
-# GDB QUICK REFERENCE (dvi output)
-refcard.dvi : refcard.tex $(REFEDITS)
-	$(SILENCE) echo > tmp.sed
+
+sedref.tex : refcard.tex $(REFEDITS)
+	$(SILENCE) echo > refcard.sed
 	$(SILENCE) for f in x $(REFEDITS) ; do \
 		test x$$f = xx && continue ; \
-		cat $(srcdir)/$$f >>tmp.sed ; \
+		cat $(srcdir)/$$f >>refcard.sed ; \
 	done
-	$(SILENCE) sed -f tmp.sed $(srcdir)/refcard.tex >sedref.tex
-	$(ECHO_TEX) $(SET_TEXINPUTS) $(TEX) sedref.tex
-	$(SILENCE) mv sedref.dvi refcard.dvi
-	$(SILENCE) rm -f sedref.log sedref.tex tmp.sed
+	$(ECHO_GEN) sed -f refcard.sed $(srcdir)/refcard.tex >$@
+	$(SILENCE) rm -f refcard.sed
+
+
+# GDB QUICK REFERENCE (dvi output)
+refcard.dvi : sedref.tex
+	$(SILENCE) rm -f sedref_dvi.*
+	$(SILENCE) cp $< sedref_dvi.tex
+	$(ECHO_TEX) $(SET_TEXINPUTS) $(TEX) sedref_dvi.tex && \
+		mv sedref_dvi.dvi $@
 
 refcard.ps : refcard.dvi
 	$(ECHO_DVIPS) $(DVIPS) $(SILENT_Q_FLAG) -t landscape -o $@ $?
 
-refcard.pdf : refcard.tex $(REFEDITS)
-	$(SILENCE) echo > tmp.sed
-	$(SILENCE) for f in x $(REFEDITS) ; do \
-		test x$$f = xx && continue ; \
-		cat $(srcdir)/$$f >>tmp.sed ; \
-	done
-	$(SILENCE) sed -f tmp.sed $(srcdir)/refcard.tex >sedref.tex
-	$(ECHO_PDFTEX) $(SET_TEXINPUTS) $(PDFTEX) sedref.tex
-	$(SILENCE) mv sedref.pdf refcard.pdf
-	$(SILENCE) rm -f sedref.log sedref.tex tmp.sed
+refcard.pdf : sedref.tex
+	$(SILENCE) rm -f sedref_pdf.*
+	$(SILENCE) cp $< sedref_pdf.tex
+	$(ECHO_PDFTEX) $(SET_TEXINPUTS) $(PDFTEX) sedref_pdf.tex && \
+		mv sedref_pdf.pdf $@
 
 # File to record current GDB version number.
 #
@@ -685,7 +686,7 @@ mostlyclean:
 	rm -f $(GDB_TEX_TMPS)
 	rm -f $(STABS_TEX_TMPS)
 	rm -f $(ANNOTATE_TEX_TMPS)
-	rm -f sedref.dvi sedref.tex tmp.sed sedref.log
+	rm -f refcard.sed sedref.tex sedref_dvi.* sedref_pdf.*
 	rm -f $(POD_FILE_TMPS)
 
 clean: mostlyclean
-- 
2.25.4

[PATCHv3 6/6] gdb/doc: fix parallel build of pdf and dvi files

[Andrew Burgess]

When building with 'make -j20 -C gdb/doc all-doc' I often see problems
caused from trying to build some dvi files in parallel with some pdf
files.  The problem files are: gdb.dvi and gdb.pdf; stabs.dvi and
stabs.pdf; and annotate.dvi and annotate.pdf.

The problem is that building these files create temporary files in the
local directory.  There's already a race here that two make threads
might try to create these files at the same time.

But it gets worse, to avoid issues where a failed build could leave
these temporary files in a corrupted state, and so prevent the next
build from succeeding, the recipe for each of these files delete all
the temporary files first, this obviously causes problems if some
other thread has already started the build and is relying on these
temporary files.

To work around this problem I propose we start using the --build flag
for texi2dvi (which is the same tool used to create the pdf files).
The --build flag allows for the temporary files to be placed into a
sub-directory, e.g. creating gdb.pdf will create the temporary files
in gdb.t2d/pdf/ and gdb.dvi uses gdb.t2d/dvi/, in this way the
temporary files will never clash, and we can easily delete the
specific sub-directory at the start of the recipe to ensure a
successful build.

Very old versions of texi2dvi don't support --build, so I've added a
configure check for this option.  If the option is not supported then
we don't use it.  This means we fall back to placing temporary files
in the local directory, which means parallel builds will remain
broken.  The --build option is definitely present in texi2dvi version
6.1, from 2016, so any version after that should be fine.

For me, with this fix in place, I can now run 'make -j20 -C gdb/doc
all-doc' without seeing any build problems.

Approved-By: Tom Tromey <tom@tromey.com>
---
 gdb/configure       | 33 +++++++++++++++++++++++++++++++--
 gdb/configure.ac    | 21 +++++++++++++++++++++
 gdb/doc/Makefile.in | 42 +++++++++++++++++++++++++++++-------------
 3 files changed, 81 insertions(+), 15 deletions(-)

diff --git a/gdb/configure b/gdb/configure
index 66a7ad8d256..8d870d50de3 100755
--- a/gdb/configure
+++ b/gdb/configure
@@ -765,6 +765,7 @@ ENABLE_BFD_64_BIT_TRUE
 subdirs
 GDB_DATADIR
 DEBUGDIR
+TEXI2DVI_EXTRA_FLAGS
 MAKEINFO_EXTRA_FLAGS
 MAKEINFOFLAGS
 MAKEINFO
@@ -11499,7 +11500,7 @@ else
   lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
   lt_status=$lt_dlunknown
   cat > conftest.$ac_ext <<_LT_EOF
-#line 11502 "configure"
+#line 11503 "configure"
 #include "confdefs.h"
 
 #if HAVE_DLFCN_H
@@ -11605,7 +11606,7 @@ else
   lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
   lt_status=$lt_dlunknown
   cat > conftest.$ac_ext <<_LT_EOF
-#line 11608 "configure"
+#line 11609 "configure"
 #include "confdefs.h"
 
 #if HAVE_DLFCN_H
@@ -24609,6 +24610,34 @@ if test x"$gdb_cv_have_makeinfo_click" = xyes; then
 fi
 
 
+## Figure out if texi2dvi supports the --build option.  Older versions
+## didn't.  If this option is not supported then parallel building in
+## the docs/ directory is probably going to be broken.  But single
+## threaded build should be fine.
+TEXI2DVI=texi2dvi
+TEXI2DVI_EXTRA_FLAGS=""
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether $TEXI2DVI supports --build" >&5
+$as_echo_n "checking whether $TEXI2DVI supports --build... " >&6; }
+if ${gdb_cv_have_texi2dvi_build_opt+:} false; then :
+  $as_echo_n "(cached) " >&6
+else
+  echo '\input texinfo' >conftest.texi
+   echo 'abc' >>conftest.texi
+   echo '@bye' >>conftest.texi
+   cp conftest.texi /home/andrew/tmp/
+   if eval "$TEXI2DVI --build=tidy conftest.texi >&5 2>&5"; then
+     gdb_cv_have_texi2dvi_build_opt=yes
+   else
+     gdb_cv_have_texi2dvi_build_opt=no
+   fi
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $gdb_cv_have_texi2dvi_build_opt" >&5
+$as_echo "$gdb_cv_have_texi2dvi_build_opt" >&6; }
+if test x"$gdb_cv_have_texi2dvi_build_opt" = xyes; then
+  TEXI2DVI_EXTRA_FLAGS="$TEXI2DVI_EXTRA_FLAGS --build=tidy"
+fi
+
+
 
 
 # Check whether --with-separate-debug-dir was given.
diff --git a/gdb/configure.ac b/gdb/configure.ac
index 62ff09cea20..3b726e5f0aa 100644
--- a/gdb/configure.ac
+++ b/gdb/configure.ac
@@ -117,6 +117,27 @@ if test x"$gdb_cv_have_makeinfo_click" = xyes; then
 fi
 AC_SUBST(MAKEINFO_EXTRA_FLAGS)
 
+## Figure out if texi2dvi supports the --build option.  Older versions
+## didn't.  If this option is not supported then parallel building in
+## the docs/ directory is probably going to be broken.  But single
+## threaded build should be fine.
+TEXI2DVI=texi2dvi
+TEXI2DVI_EXTRA_FLAGS=""
+AC_CACHE_CHECK([whether $TEXI2DVI supports --build], gdb_cv_have_texi2dvi_build_opt,
+  [echo '\input texinfo' >conftest.texi
+   echo 'abc' >>conftest.texi
+   echo '@bye' >>conftest.texi
+   cp conftest.texi /home/andrew/tmp/
+   if eval "$TEXI2DVI --build=tidy conftest.texi >&5 2>&5"; then
+     gdb_cv_have_texi2dvi_build_opt=yes
+   else
+     gdb_cv_have_texi2dvi_build_opt=no
+   fi])
+if test x"$gdb_cv_have_texi2dvi_build_opt" = xyes; then
+  TEXI2DVI_EXTRA_FLAGS="$TEXI2DVI_EXTRA_FLAGS --build=tidy"
+fi
+AC_SUBST(TEXI2DVI_EXTRA_FLAGS)
+
 GDB_AC_WITH_DIR(DEBUGDIR, separate-debug-dir,
     [look for global separate debug info in this path @<:@LIBDIR/debug@:>@],
     [${libdir}/debug])
diff --git a/gdb/doc/Makefile.in b/gdb/doc/Makefile.in
index d294120ad19..33149749982 100644
--- a/gdb/doc/Makefile.in
+++ b/gdb/doc/Makefile.in
@@ -68,6 +68,8 @@ TEXI2ROFF=texi2roff
 
 # where to find texi2dvi, ditto
 TEXI2DVI=texi2dvi
+TEXI2DVI_EXTRA_FLAGS=@TEXI2DVI_EXTRA_FLAGS@
+TEXI2DVI_CMD = $(TEXI2DVI) $(TEXI2DVI_EXTRA_FLAGS)
 
 # Package to install the docs under
 PACKAGE = @PACKAGE@
@@ -484,21 +486,26 @@ gdb-cfg.texi: ${srcdir}/${DOC_CONFIG}-cfg.texi
 # Clean these up before each run.  Avoids a catch 22 with not being
 # able to re-generate these files (to fix a corruption) because these
 # files contain a corruption.
+#
+# If TEXI2DVI_CMD includes the --build option then these files should
+# not be created in the current directory, but attempting to delete
+# them is harmless.
 GDB_TEX_TMPS = gdb.aux gdb.cp* gdb.fn* gdb.ky* gdb.log gdb.pg* gdb.toc \
 	gdb.tp* gdb.vr*
 
 # GDB MANUAL: TeX dvi file
 gdb.dvi: ${GDB_DOC_FILES}
-	$(SILENCE) rm -f $(GDB_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) $(READLINE_TEXI_INCFLAG) \
-		-I ${GDBMI_DIR} -I $(srcdir) $(srcdir)/gdb.texinfo
+	$(SILENCE) rm -fr $(GDB_TEX_TMPS) gdb.t2d/dvi/
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) \
+		$(READLINE_TEXI_INCFLAG) -I ${GDBMI_DIR} -I $(srcdir) \
+		$(srcdir)/gdb.texinfo
 
 gdb.ps: gdb.dvi
 	$(ECHO_DVIPS) $(DVIPS) $(SILENT_Q_FLAG) -o $@ $?
 
 gdb.pdf: ${GDB_DOC_FILES}
-	$(SILENCE) rm -f $(GDB_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) --pdf \
+	$(SILENCE) rm -fr $(GDB_TEX_TMPS) gdb.t2d/pdf/
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) --pdf \
 		$(READLINE_TEXI_INCFLAG) -I ${GDBMI_DIR} -I $(srcdir) \
 		$(srcdir)/gdb.texinfo
 
@@ -599,41 +606,49 @@ stabs/index.html: $(STABS_DOC_FILES)
 # Clean these up before each run.  Avoids a catch 22 with not being
 # able to re-generate these files (to fix a corruption) because these
 # files contain a corruption.
+#
+# If TEXI2DVI_CMD includes the --build option then these files should
+# not be created in the current directory, but attempting to delete
+# them is harmless.
 STABS_TEX_TMPS = stabs.aux stabs.cp* stabs.fn* stabs.ky* \
 	stabs.log stabs.pg* stabs.toc stabs.tp* stabs.vr*
 
 # STABS DOCUMENTATION: TeX dvi file
 stabs.dvi : $(STABS_DOC_FILES)
-	$(SILENCE) rm -f $(STABS_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) -I $(srcdir) \
+	$(SILENCE) rm -fr $(STABS_TEX_TMPS) stabs.t2d/dvi/
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) -I $(srcdir) \
 		$(srcdir)/stabs.texinfo
 
 stabs.ps: stabs.dvi
 	$(ECHO_DVIPS) $(DVIPS) $(SILENT_Q_FLAG) -o $@ $?
 
 stabs.pdf: $(STABS_DOC_FILES)
-	$(SILENCE) rm -f $(STABS_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) --pdf -I $(srcdir) \
+	$(SILENCE) rm -fr $(STABS_TEX_TMPS) stabs.t2d/pdf/
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) --pdf -I $(srcdir) \
 		$(srcdir)/stabs.texinfo
 
 # Clean these up before each run.  Avoids a catch 22 with not being
 # able to re-generate these files (to fix a corruption) because these
 # files contain a corruption.
+#
+# If TEXI2DVI_CMD includes the --build option then these files should
+# not be created in the current directory, but attempting to delete
+# them is harmless.
 ANNOTATE_TEX_TMPS = annotate.aux annotate.cp* annotate.fn* annotate.ky* \
 	annotate.log annotate.pg* annotate.toc annotate.tp* annotate.vr*
 
 # ANNOTATE DOCUMENTATION: TeX dvi file
 annotate.dvi : $(ANNOTATE_DOC_FILES)
-	$(SILENCE) rm -f $(ANNOTATE_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) -I $(srcdir) \
+	$(SILENCE) rm -fr $(ANNOTATE_TEX_TMPS) annotate.t2d/dvi/
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) -I $(srcdir) \
 		$(srcdir)/annotate.texinfo
 
 annotate.ps: annotate.dvi
 	$(ECHO_DVIPS) $(DVIPS) $(SILENT_Q_FLAG) -o $@ $?
 
 annotate.pdf: $(ANNOTATE_DOC_FILES)
-	$(SILENCE) rm -f $(ANNOTATE_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) --pdf -I $(srcdir) \
+	$(SILENCE) rm -fr $(ANNOTATE_TEX_TMPS) annotate.t2d/pdf/
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) --pdf -I $(srcdir) \
 		$(srcdir)/annotate.texinfo
 
 annotate.info: $(ANNOTATE_DOC_FILES)
@@ -683,6 +698,7 @@ Makefile: Makefile.in $(host_makefile_frag) ../config.status
 
 mostlyclean:
 	rm -f gdb.mm gdb.ms gdb.me links2roff
+	rm -fr annotate.t2d/ stabs.t2d/ gdb.t2d/
 	rm -f $(GDB_TEX_TMPS)
 	rm -f $(STABS_TEX_TMPS)
 	rm -f $(ANNOTATE_TEX_TMPS)
-- 
2.25.4

Re: [PATCHv2 2/5] gdb/doc: allow for version.subst in the source tree

[Andrew Burgess]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrew Burgess <aburgess@redhat.com>
>> Cc: gdb-patches@sourceware.org
>> Date: Mon, 03 Jun 2024 15:22:41 +0100
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >>   make: Entering directory '/tmp/build/build/gdb/doc'
>> >>     GEN    GDBvn.texi
>> >>   sed: can't read version.subst: No such file or directory
>> >>   make: Leaving directory '/tmp/build/build/gdb/doc'
>> >> 
>> >> The problem is that every reference to version.subst in GDB's Makefile
>> >> assumes that the version.subst file will always be in the build
>> >> directory.
>> >> 
>> >> In this commit I replace direct references to version.subst with a
>> >> small shell snippet which returns the path to the version.subst file
>> >> to use.  If there is a version.subst file in the build directory then
>> >> that file is selected, otherwise we return the path to a version.subst
>> >> file in the source directory.
>> >
>> > Why isn't VPATH or vpath enough to solve this nuisance?
>> 
>> If anything VPATH is the _cause_ of this nuisance!  With the VPATH set
>> as it is, make will check both the build directory and the source
>> directory.  So when we have a rule like:
>> 
>>   some_target : version.subst
>>      some_command version.subst
>> 
>> make is going to look for the version.subst dependency in both
>> locations, however, where 'some_command' looks is entirely up to
>> 'some_command'.  In our case for example, the 'sed' command doesn't (to
>> my knowledge) have a search path command line option, so a command like:
>> 
>>   sed q version.subst
>> 
>> is only going to search in the build directory.
>
> My reading of the node "Recipes/Search" in the GNU Make manual is that
> if you use automatic variables, such as $^, in the recipe, Make will
> expand the variable into the full file name including the directory in
> which it found the file via VPATH search.  Can't we use that feature
> here?  IOW, instead of
>
>   some_target : version.subst
>      some_command version.subst
>
> you should use something like
>
>   some_target : version.subst
>      some_command $<

Thanks.  I was completely unaware that $< (and friends) worked like
this... but I guess it makes sense when you think about it.

This is indeed a better way to solve the problem I had.

While working on this change I ended up adding a whole extra patch to
the series, so I've followed up with a V3 series.

Thanks for your patience.

Andrew

Re: [PATCHv3 0/6] gdb/doc: parallel build fixes and improvements

[Tom Tromey]

>>>>> "Andrew" == Andrew Burgess <aburgess@redhat.com> writes:

Andrew> In v3:
Andrew>   - Add new patch #2 which merges the two man page rules into one.

Andrew>   - Updated patch #3 based on Eli's feedback.  Now use $< instead of a
Andrew>     shell snippet to locate version.subst.

Looks good to me.
Approved-By: Tom Tromey <tom@tromey.com>

Tom

Re: [PATCHv3 3/6] gdb/doc: allow for version.subst in the source tree

[Pedro Alves]

Hi,

I was skimming this and noticed a couple typos.

On 2024-06-06 18:49, Andrew Burgess wrote:
> +# included, pre-built, as part of a relase.

relase -> release

> +# included, pre-built, as part of a relase.

Same.   (Appears twice in the patch.)

Re: [PATCHv3 1/6] gdb/doc: don't try to copy GDBvn.texi from the source tree

[Pedro Alves]

On 2024-06-06 18:49, Andrew Burgess wrote:

> However, copying GDBvn.texi into the source directory is completely
> unnecessary.  

I think you meant "from the source directory" here.

Re: [PATCHv3 6/6] gdb/doc: fix parallel build of pdf and dvi files

[Pedro Alves]

On 2024-06-06 18:49, Andrew Burgess wrote:
> When building with 'make -j20 -C gdb/doc all-doc' I often see problems
> caused from trying to build some dvi files in parallel with some pdf
> files.  The problem files are: gdb.dvi and gdb.pdf; stabs.dvi and
> stabs.pdf; and annotate.dvi and annotate.pdf.
> 
> The problem is that building these files create temporary files in the
> local directory.  There's already a race here that two make threads
> might try to create these files at the same time.
> 
> But it gets worse, to avoid issues where a failed build could leave
> these temporary files in a corrupted state, and so prevent the next
> build from succeeding, the recipe for each of these files delete all
> the temporary files first, this obviously causes problems if some
> other thread has already started the build and is relying on these
> temporary files.
> 
> To work around this problem I propose we start using the --build flag
> for texi2dvi (which is the same tool used to create the pdf files).
> The --build flag allows for the temporary files to be placed into a
> sub-directory, e.g. creating gdb.pdf will create the temporary files
> in gdb.t2d/pdf/ and gdb.dvi uses gdb.t2d/dvi/, in this way the
> temporary files will never clash, and we can easily delete the
> specific sub-directory at the start of the recipe to ensure a
> successful build.
> 
> Very old versions of texi2dvi don't support --build, so I've added a
> configure check for this option.  If the option is not supported then
> we don't use it.  This means we fall back to placing temporary files
> in the local directory, which means parallel builds will remain
> broken.  The --build option is definitely present in texi2dvi version
> 6.1, from 2016, so any version after that should be fine.
> 

If we care about old versions enough to support them like this, then
couldn't we instead make the subdirectories ourselves and build there,
instead of using --build?  That way, we would only have one code path,
and it would work the same in new or old texi2dvi?

I mean, make the rule do something like:

  mkdir -p pdf && cd pdf && texi2dvi ...

Re: [PATCH 2/4] gdb/doc: don't try to copy GDBvn.texi from the source tree

[Pedro Alves]

On 2024-05-13 17:12, Tom Tromey wrote:
>>>>>> "Andrew" == Andrew Burgess <aburgess@redhat.com> writes:
> 
> Andrew> That's right, we also have a rule that builds GDBvn.texi, and
> Andrew> GDBvn.texi is in the dependency list for both gdb.dvi and gdb.pdf.
> 
> Andrew> On top of this, GDBvn.texi was removed from the source tree in 2000,
> Andrew> in this commit:
> 
> Andrew>   commit 1171f8d754f71e35fd2f6c4e82b00f3c685e515f
> Andrew>   Date:   Tue Feb 8 03:37:34 2000 +0000
> 
> Andrew>       2000-02-07  Jason Molenda  (jsm@bugshack.cygnus.com)
> 
> Andrew> I think we can assume that everyone is happy generating GDBvn.texi, so
> Andrew> lets remove the redundant copying code.
> 
> I am not sure about this change.
> 
> I downloaded gdb-14.2.tar.xz and I do see gdb/doc/GDBvn.texi in the
> source tree there.  I think this is done because we also ship the .info
> files, so that users won't have to have makeinfo in order to build.
> 
> (I'm not sure how well that plan really works since there used to be
> open bugs about not building when makeinfo is missing ... not sure of
> the current state.)

Sometimes filesystem timestamps will end up messed up and building a
release will want to rebuild the docs.  But you can always work around
that with "make MAKEINFO=true".

Re: [PATCHv3 6/6] gdb/doc: fix parallel build of pdf and dvi files

[Andrew Burgess]

Pedro Alves <pedro@palves.net> writes:

> On 2024-06-06 18:49, Andrew Burgess wrote:
>> When building with 'make -j20 -C gdb/doc all-doc' I often see problems
>> caused from trying to build some dvi files in parallel with some pdf
>> files.  The problem files are: gdb.dvi and gdb.pdf; stabs.dvi and
>> stabs.pdf; and annotate.dvi and annotate.pdf.
>> 
>> The problem is that building these files create temporary files in the
>> local directory.  There's already a race here that two make threads
>> might try to create these files at the same time.
>> 
>> But it gets worse, to avoid issues where a failed build could leave
>> these temporary files in a corrupted state, and so prevent the next
>> build from succeeding, the recipe for each of these files delete all
>> the temporary files first, this obviously causes problems if some
>> other thread has already started the build and is relying on these
>> temporary files.
>> 
>> To work around this problem I propose we start using the --build flag
>> for texi2dvi (which is the same tool used to create the pdf files).
>> The --build flag allows for the temporary files to be placed into a
>> sub-directory, e.g. creating gdb.pdf will create the temporary files
>> in gdb.t2d/pdf/ and gdb.dvi uses gdb.t2d/dvi/, in this way the
>> temporary files will never clash, and we can easily delete the
>> specific sub-directory at the start of the recipe to ensure a
>> successful build.
>> 
>> Very old versions of texi2dvi don't support --build, so I've added a
>> configure check for this option.  If the option is not supported then
>> we don't use it.  This means we fall back to placing temporary files
>> in the local directory, which means parallel builds will remain
>> broken.  The --build option is definitely present in texi2dvi version
>> 6.1, from 2016, so any version after that should be fine.
>> 
>
> If we care about old versions enough to support them like this, then
> couldn't we instead make the subdirectories ourselves and build there,
> instead of using --build?  That way, we would only have one code path,
> and it would work the same in new or old texi2dvi?
>
> I mean, make the rule do something like:
>
>   mkdir -p pdf && cd pdf && texi2dvi ...

I took a look at this approach and the problem is fixing all the include
paths passed to the texi2dvi command, it all gets a bit messy as we need
to add '../' to the start of every include path.

I'm happy to drop the configure part of this patch, looking at the
texinfo NEWS file the --build=tidy option was added in release 4.9 which
came out in June 2007.

So if we're willing to make texinfo 4.9 a (doc) build requirement then
I'll update the patch.

Thanks,
Andrew

Re: [PATCHv3 6/6] gdb/doc: fix parallel build of pdf and dvi files

[Eli Zaretskii]

> From: Andrew Burgess <aburgess@redhat.com>
> Cc: Tom Tromey <tom@tromey.com>
> Date: Sat, 15 Jun 2024 10:44:13 +0100
> 
> > If we care about old versions enough to support them like this, then
> > couldn't we instead make the subdirectories ourselves and build there,
> > instead of using --build?  That way, we would only have one code path,
> > and it would work the same in new or old texi2dvi?
> >
> > I mean, make the rule do something like:
> >
> >   mkdir -p pdf && cd pdf && texi2dvi ...
> 
> I took a look at this approach and the problem is fixing all the include
> paths passed to the texi2dvi command, it all gets a bit messy as we need
> to add '../' to the start of every include path.
> 
> I'm happy to drop the configure part of this patch, looking at the
> texinfo NEWS file the --build=tidy option was added in release 4.9 which
> came out in June 2007.
> 
> So if we're willing to make texinfo 4.9 a (doc) build requirement then
> I'll update the patch.

Does the manual even build with such an old version of Texinfo?  I
very much doubt that, as there were many additions to the language
since then.

But if 4.9 can still be used, I don't oppose to making the patch
support that.

Re: [PATCHv3 6/6] gdb/doc: fix parallel build of pdf and dvi files

[Andrew Burgess]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrew Burgess <aburgess@redhat.com>
>> Cc: Tom Tromey <tom@tromey.com>
>> Date: Sat, 15 Jun 2024 10:44:13 +0100
>> 
>> > If we care about old versions enough to support them like this, then
>> > couldn't we instead make the subdirectories ourselves and build there,
>> > instead of using --build?  That way, we would only have one code path,
>> > and it would work the same in new or old texi2dvi?
>> >
>> > I mean, make the rule do something like:
>> >
>> >   mkdir -p pdf && cd pdf && texi2dvi ...
>> 
>> I took a look at this approach and the problem is fixing all the include
>> paths passed to the texi2dvi command, it all gets a bit messy as we need
>> to add '../' to the start of every include path.
>> 
>> I'm happy to drop the configure part of this patch, looking at the
>> texinfo NEWS file the --build=tidy option was added in release 4.9 which
>> came out in June 2007.
>> 
>> So if we're willing to make texinfo 4.9 a (doc) build requirement then
>> I'll update the patch.
>
> Does the manual even build with such an old version of Texinfo?  I
> very much doubt that, as there were many additions to the language
> since then.

You're right.  I tried with texinfo 4.13 (I had that kicking around
already) and the docs build fails with errors from the .texi files, so I
guess we already require a version > 4.13.

I'll update the patch and post a new version.

Thanks,
Andrew

Re: [PATCHv3 6/6] gdb/doc: fix parallel build of pdf and dvi files

[Andrew Burgess]

Here's an updated version of just patch #6.  In this update:

  - Removed all the configure nonsense, we already require a version of
    texinfo greater than 4.9, so I just assume the required options are
    available.

  - Now also make use of the --build-dir option (also added in 4.9) to
    better control the creation of the temporary directories, this makes
    it easier to cleanup (see mostlyclean recipe).

Let me know what you think.

Thanks,
Andrew

---

commit e063bdc6f213f80da0de28a7f84cd31f240a6716
Author: Andrew Burgess <aburgess@redhat.com>
Date:   Wed May 8 19:12:57 2024 +0100

    gdb/doc: fix parallel build of pdf and dvi files
    
    When building with 'make -j20 -C gdb/doc all-doc' I often see problems
    caused from trying to build some dvi files in parallel with some pdf
    files.  The problem files are: gdb.dvi and gdb.pdf; stabs.dvi and
    stabs.pdf; and annotate.dvi and annotate.pdf.
    
    The problem is that building these files create temporary files in the
    local directory.  There's already a race here that two make threads
    might try to create these files at the same time.
    
    But it gets worse, to avoid issues where a failed build could leave
    these temporary files in a corrupted state, and so prevent the next
    build from succeeding, the recipe for each of these files deletes all
    the temporary files first, this obviously causes problems if some
    other thread has already started the build and is relying on these
    temporary files.
    
    To work around this problem I propose we start using the --build and
    --build-dir options for texi2dvi (which is the same tool used to
    create the pdf files).  These options were added in texinfo 4.9 which
    was released in June 2007.  We already require using a version of
    texinfo after 4.9 (I tried to build with 4.13 and the doc build failed
    as some of the texinfo constructs were not understood), so this patch
    has not changed the minimum required version at all.
    
    The --build flag allows the temporary files to be placed into a
    sub-directory, and the --build-dir option allows us to control the
    name of that sub-directory.
    
    What we do is create a unique sub-directory for each target that
    invokes texi2dvi, all of the unique sub-directories are created within
    a single directory texi2dvi_tmpdir, and so after a complete doc build,
    we are left with a build tree like this:
    
      build/gdb/doc/
      '-- texi2dvi_tmpdir/
          |-- annotate_dvi/
          |-- annotate_pdf/
          |-- gdb_dvi/
          |-- gdb_pdf/
          |-- stabs_dvi/
          '-- stabs_pdf/
    
    I've left out all the individual files that live within these
    directories for simplicity.
    
    To avoid corrupted temporary files preventing a future build to
    complete, each recipe deletes its associated sub-directory from within
    texi2dvi_tmpdir/ before it attempts a build, this ensures a fresh
    start each time.
    
    And the mostlyclean target deletes texi2dvi_tmpdir/ and all its
    sub-directories, ensuring that everything is cleaned up.
    
    For me, with this fix in place, I can now run 'make -j20 -C gdb/doc
    all-doc' without seeing any build problems.

diff --git a/gdb/doc/Makefile.in b/gdb/doc/Makefile.in
index cfd3b3597fb..cf10868ed05 100644
--- a/gdb/doc/Makefile.in
+++ b/gdb/doc/Makefile.in
@@ -66,8 +66,13 @@ MAKEHTMLFLAGS =
 # where to find texi2roff, ditto
 TEXI2ROFF=texi2roff
 
-# where to find texi2dvi, ditto
+# Where to find texi2dvi.  The use of --build and --build-dir options
+# mean that at least texinfo 4.9 is required.
 TEXI2DVI=texi2dvi
+TEXI2DVI_TMPDIR=texi2dvi_tmpdir
+TEXI2DVI_CMD = rm -fr $(TEXI2DVI_TMPDIR)/$(subst .,_,$@) \
+		&& $(TEXI2DVI) --build=tidy \
+			       --build-dir=$(TEXI2DVI_TMPDIR)/$(subst .,_,$@)
 
 # Package to install the docs under
 PACKAGE = @PACKAGE@
@@ -481,24 +486,17 @@ gdb-cfg.texi: ${srcdir}/${DOC_CONFIG}-cfg.texi
 	ln ${srcdir}/${DOC_CONFIG}-cfg.texi gdb-cfg.texi || \
 	cp ${srcdir}/${DOC_CONFIG}-cfg.texi gdb-cfg.texi
 
-# Clean these up before each run.  Avoids a catch 22 with not being
-# able to re-generate these files (to fix a corruption) because these
-# files contain a corruption.
-GDB_TEX_TMPS = gdb.aux gdb.cp* gdb.fn* gdb.ky* gdb.log gdb.pg* gdb.toc \
-	gdb.tp* gdb.vr*
-
 # GDB MANUAL: TeX dvi file
 gdb.dvi: ${GDB_DOC_FILES}
-	$(SILENCE) rm -f $(GDB_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) $(READLINE_TEXI_INCFLAG) \
-		-I ${GDBMI_DIR} -I $(srcdir) $(srcdir)/gdb.texinfo
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) \
+		$(READLINE_TEXI_INCFLAG) -I ${GDBMI_DIR} -I $(srcdir) \
+		$(srcdir)/gdb.texinfo
 
 gdb.ps: gdb.dvi
 	$(ECHO_DVIPS) $(DVIPS) $(SILENT_Q_FLAG) -o $@ $?
 
 gdb.pdf: ${GDB_DOC_FILES}
-	$(SILENCE) rm -f $(GDB_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) --pdf \
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) --pdf \
 		$(READLINE_TEXI_INCFLAG) -I ${GDBMI_DIR} -I $(srcdir) \
 		$(srcdir)/gdb.texinfo
 
@@ -596,44 +594,28 @@ stabs/index.html: $(STABS_DOC_FILES)
 		-I $(srcdir) \
 		$(srcdir)/stabs.texinfo
 
-# Clean these up before each run.  Avoids a catch 22 with not being
-# able to re-generate these files (to fix a corruption) because these
-# files contain a corruption.
-STABS_TEX_TMPS = stabs.aux stabs.cp* stabs.fn* stabs.ky* \
-	stabs.log stabs.pg* stabs.toc stabs.tp* stabs.vr*
-
 # STABS DOCUMENTATION: TeX dvi file
 stabs.dvi : $(STABS_DOC_FILES)
-	$(SILENCE) rm -f $(STABS_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) -I $(srcdir) \
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) -I $(srcdir) \
 		$(srcdir)/stabs.texinfo
 
 stabs.ps: stabs.dvi
 	$(ECHO_DVIPS) $(DVIPS) $(SILENT_Q_FLAG) -o $@ $?
 
 stabs.pdf: $(STABS_DOC_FILES)
-	$(SILENCE) rm -f $(STABS_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) --pdf -I $(srcdir) \
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) --pdf -I $(srcdir) \
 		$(srcdir)/stabs.texinfo
 
-# Clean these up before each run.  Avoids a catch 22 with not being
-# able to re-generate these files (to fix a corruption) because these
-# files contain a corruption.
-ANNOTATE_TEX_TMPS = annotate.aux annotate.cp* annotate.fn* annotate.ky* \
-	annotate.log annotate.pg* annotate.toc annotate.tp* annotate.vr*
-
 # ANNOTATE DOCUMENTATION: TeX dvi file
 annotate.dvi : $(ANNOTATE_DOC_FILES)
-	$(SILENCE) rm -f $(ANNOTATE_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) -I $(srcdir) \
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) -I $(srcdir) \
 		$(srcdir)/annotate.texinfo
 
 annotate.ps: annotate.dvi
 	$(ECHO_DVIPS) $(DVIPS) $(SILENT_Q_FLAG) -o $@ $?
 
 annotate.pdf: $(ANNOTATE_DOC_FILES)
-	$(SILENCE) rm -f $(ANNOTATE_TEX_TMPS)
-	$(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) --pdf -I $(srcdir) \
+	$(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) --pdf -I $(srcdir) \
 		$(srcdir)/annotate.texinfo
 
 annotate.info: $(ANNOTATE_DOC_FILES)
@@ -683,9 +665,7 @@ Makefile: Makefile.in $(host_makefile_frag) ../config.status
 
 mostlyclean:
 	rm -f gdb.mm gdb.ms gdb.me links2roff
-	rm -f $(GDB_TEX_TMPS)
-	rm -f $(STABS_TEX_TMPS)
-	rm -f $(ANNOTATE_TEX_TMPS)
+	rm -fr $(TEXI2DVI_TMPDIR)
 	rm -f refcard.sed sedref.tex sedref_dvi.* sedref_pdf.*
 	rm -f $(POD_FILE_TMPS)
 

Re: [PATCHv3 6/6] gdb/doc: fix parallel build of pdf and dvi files

[Pedro Alves]

On 2024-06-15 16:04, Andrew Burgess wrote:
> 
> Here's an updated version of just patch #6.  In this update:
> 
>   - Removed all the configure nonsense, we already require a version of
>     texinfo greater than 4.9, so I just assume the required options are
>     available.
> 
>   - Now also make use of the --build-dir option (also added in 4.9) to
>     better control the creation of the temporary directories, this makes
>     it easier to cleanup (see mostlyclean recipe).
> 
> Let me know what you think.

Just ran into this myself, which reminded me to reply...  :-D

LGTM.

Approved-By: Pedro Alves <pedro@palves.net>

Re: [PATCHv3 6/6] gdb/doc: fix parallel build of pdf and dvi files

[Andrew Burgess]

Pedro Alves <pedro@palves.net> writes:

> On 2024-06-15 16:04, Andrew Burgess wrote:
>> 
>> Here's an updated version of just patch #6.  In this update:
>> 
>>   - Removed all the configure nonsense, we already require a version of
>>     texinfo greater than 4.9, so I just assume the required options are
>>     available.
>> 
>>   - Now also make use of the --build-dir option (also added in 4.9) to
>>     better control the creation of the temporary directories, this makes
>>     it easier to cleanup (see mostlyclean recipe).
>> 
>> Let me know what you think.
>
> Just ran into this myself, which reminded me to reply...  :-D
>
> LGTM.
>
> Approved-By: Pedro Alves <pedro@palves.net>

I've now pushed this series.

Thanks,
Andrew