From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTPS id 20FE5387084B for ; Sat, 15 Jun 2024 15:04:57 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 20FE5387084B Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 20FE5387084B Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1718463899; cv=none; b=xXfOMvovFNUuniZYHJIqXFiI3HCAPno9NaPz5puMpP5Suo1Yl+nzmurcZVNsrHNgsZAKKXG2Jv0BntwEyKSIXz1BNLEKN1ayhTFXLbw1+HC68yNsZs/h8E4NM6OYcZleQ0ufFXihMYeInQnmQQlxcLFSSH4U0Kc+CiN58GqqK0g= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1718463899; c=relaxed/simple; bh=nxcT51A4+jF/xd+SlaaGyhnEIvyThGTKU4OLCMUHzLk=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=Yy4a6y6Ncud7ZjtU/TcyAUEPuac9f6CbtjWm5NdSHC8POOk/TGppmMh8dz53sIiGxGd8t3EN6MDccpDoBAxVPqSlE6ErRrk0DRdN0UA280EV3xMeNLvu3XGsDGpqQuQXi/St4DFEauyGqtzwVRJBM+KzTw0F48aNKYIR4e6Dwds= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1718463896; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=OyP2UARoKLKChAH/tbQ18W4wbzZ/KpQa659HoO3STgA=; b=HNoxTs7/KDBJQ9ZasyflYrVZz0ixeW/ED+3SOG5whnXSV7a8lUCGrykZM3wOAhYsqgNOwq 9HeWphbnYpjnJxQsd0M5DFCDSg5Dhj1EP8cr+3w7q0HrLmo4jEFeiI4wvRRLBGVmlrLZU0 Ucf/M6opI/07c6LA5UOPNFsg9rjs9es= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-401-hXoI9L4PM6uOpBHw5Vs2-g-1; Sat, 15 Jun 2024 11:04:55 -0400 X-MC-Unique: hXoI9L4PM6uOpBHw5Vs2-g-1 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-35f09791466so1462571f8f.2 for ; Sat, 15 Jun 2024 08:04:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718463893; x=1719068693; h=mime-version:message-id:date:references:in-reply-to:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=OyP2UARoKLKChAH/tbQ18W4wbzZ/KpQa659HoO3STgA=; b=hpjqlCKerkYE3RvbN3HWAFgEDYD042rEssWQaj+302NuqWhI+ufqq5oMy4jb78ufpz mxVRDVnDzBWEw/cyN9HbJ/8aRZ7LS6miAEiUAyQ5ySJh+wXIsj5rYarzQk2bHJLg9VqK C65SPlrsJvMCdGWpaqogrqlcOKgYesFR89peRjKh4U5g91NR7e3zF8JhKYRSWCLUE7I2 HqObvmr8kA2TMaX1nMuPaUIf2CB5Sj+GYpyn9wOW0ig11JzhFg0XaqK5CQG8zW4+Af/d NWldMzJvDT3S5smlzogqmeqRzKL3f2u8Cf03Zg05vEF4u4c7AjEu8Pzw7Hb1RMNeJUvq 3xZg== X-Forwarded-Encrypted: i=1; AJvYcCVTMZGfGFyBFPRf1XpBSGFTqk6kJ3wJibD3szGx/mlwKNfZMF/WjrCVuHQldRCS35QxNPAwbKcH5kusPYLuZq+iaRv4hqiNU2TNBA== X-Gm-Message-State: AOJu0Yy8W3RaFK7BTEuhMgSWlGKy9MTPb3pihMkSpj/wwMQTmIl/LD4L aOai2LQZLHpqGiONPfqZes2IBQRDPaua6EBCkHqepDdvNfNw8jGGpZlAe2pCliy1mNRNiptOUq3 h4xNIC55HvFkob2HknSwFDy1SkRApMk/9Psj7Lhu+8eQcPO7t/eUUo7xLxrFhpQzrLS4= X-Received: by 2002:a05:6000:bcd:b0:35f:26e7:f978 with SMTP id ffacd0b85a97d-3607a75c3b2mr3848653f8f.37.1718463893266; Sat, 15 Jun 2024 08:04:53 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF++3CQ0Yqz00h+K6pyjSLvZnJORwpIQcMyc8SdCy6DZNi2o+iZD7JBM02YvBF5/Yfdch8ojw== X-Received: by 2002:a05:6000:bcd:b0:35f:26e7:f978 with SMTP id ffacd0b85a97d-3607a75c3b2mr3848637f8f.37.1718463892636; Sat, 15 Jun 2024 08:04:52 -0700 (PDT) Received: from localhost ([31.111.84.186]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3607509c89dsm7362320f8f.25.2024.06.15.08.04.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 15 Jun 2024 08:04:52 -0700 (PDT) From: Andrew Burgess To: Eli Zaretskii Cc: pedro@palves.net, gdb-patches@sourceware.org, tom@tromey.com Subject: Re: [PATCHv3 6/6] gdb/doc: fix parallel build of pdf and dvi files In-Reply-To: <86sexetm5b.fsf@gnu.org> References: <412817c6d07836aa2ba2026ac9047fb283c481ea.1717696065.git.aburgess@redhat.com> <871q4y8rrm.fsf@redhat.com> <86sexetm5b.fsf@gnu.org> Date: Sat, 15 Jun 2024 16:04:51 +0100 Message-ID: <87jziq6ycs.fsf@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain X-Spam-Status: No, score=-11.8 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Here's an updated version of just patch #6. In this update: - Removed all the configure nonsense, we already require a version of texinfo greater than 4.9, so I just assume the required options are available. - Now also make use of the --build-dir option (also added in 4.9) to better control the creation of the temporary directories, this makes it easier to cleanup (see mostlyclean recipe). Let me know what you think. Thanks, Andrew --- commit e063bdc6f213f80da0de28a7f84cd31f240a6716 Author: Andrew Burgess Date: Wed May 8 19:12:57 2024 +0100 gdb/doc: fix parallel build of pdf and dvi files When building with 'make -j20 -C gdb/doc all-doc' I often see problems caused from trying to build some dvi files in parallel with some pdf files. The problem files are: gdb.dvi and gdb.pdf; stabs.dvi and stabs.pdf; and annotate.dvi and annotate.pdf. The problem is that building these files create temporary files in the local directory. There's already a race here that two make threads might try to create these files at the same time. But it gets worse, to avoid issues where a failed build could leave these temporary files in a corrupted state, and so prevent the next build from succeeding, the recipe for each of these files deletes all the temporary files first, this obviously causes problems if some other thread has already started the build and is relying on these temporary files. To work around this problem I propose we start using the --build and --build-dir options for texi2dvi (which is the same tool used to create the pdf files). These options were added in texinfo 4.9 which was released in June 2007. We already require using a version of texinfo after 4.9 (I tried to build with 4.13 and the doc build failed as some of the texinfo constructs were not understood), so this patch has not changed the minimum required version at all. The --build flag allows the temporary files to be placed into a sub-directory, and the --build-dir option allows us to control the name of that sub-directory. What we do is create a unique sub-directory for each target that invokes texi2dvi, all of the unique sub-directories are created within a single directory texi2dvi_tmpdir, and so after a complete doc build, we are left with a build tree like this: build/gdb/doc/ '-- texi2dvi_tmpdir/ |-- annotate_dvi/ |-- annotate_pdf/ |-- gdb_dvi/ |-- gdb_pdf/ |-- stabs_dvi/ '-- stabs_pdf/ I've left out all the individual files that live within these directories for simplicity. To avoid corrupted temporary files preventing a future build to complete, each recipe deletes its associated sub-directory from within texi2dvi_tmpdir/ before it attempts a build, this ensures a fresh start each time. And the mostlyclean target deletes texi2dvi_tmpdir/ and all its sub-directories, ensuring that everything is cleaned up. For me, with this fix in place, I can now run 'make -j20 -C gdb/doc all-doc' without seeing any build problems. diff --git a/gdb/doc/Makefile.in b/gdb/doc/Makefile.in index cfd3b3597fb..cf10868ed05 100644 --- a/gdb/doc/Makefile.in +++ b/gdb/doc/Makefile.in @@ -66,8 +66,13 @@ MAKEHTMLFLAGS = # where to find texi2roff, ditto TEXI2ROFF=texi2roff -# where to find texi2dvi, ditto +# Where to find texi2dvi. The use of --build and --build-dir options +# mean that at least texinfo 4.9 is required. TEXI2DVI=texi2dvi +TEXI2DVI_TMPDIR=texi2dvi_tmpdir +TEXI2DVI_CMD = rm -fr $(TEXI2DVI_TMPDIR)/$(subst .,_,$@) \ + && $(TEXI2DVI) --build=tidy \ + --build-dir=$(TEXI2DVI_TMPDIR)/$(subst .,_,$@) # Package to install the docs under PACKAGE = @PACKAGE@ @@ -481,24 +486,17 @@ gdb-cfg.texi: ${srcdir}/${DOC_CONFIG}-cfg.texi ln ${srcdir}/${DOC_CONFIG}-cfg.texi gdb-cfg.texi || \ cp ${srcdir}/${DOC_CONFIG}-cfg.texi gdb-cfg.texi -# Clean these up before each run. Avoids a catch 22 with not being -# able to re-generate these files (to fix a corruption) because these -# files contain a corruption. -GDB_TEX_TMPS = gdb.aux gdb.cp* gdb.fn* gdb.ky* gdb.log gdb.pg* gdb.toc \ - gdb.tp* gdb.vr* - # GDB MANUAL: TeX dvi file gdb.dvi: ${GDB_DOC_FILES} - $(SILENCE) rm -f $(GDB_TEX_TMPS) - $(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) $(READLINE_TEXI_INCFLAG) \ - -I ${GDBMI_DIR} -I $(srcdir) $(srcdir)/gdb.texinfo + $(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) \ + $(READLINE_TEXI_INCFLAG) -I ${GDBMI_DIR} -I $(srcdir) \ + $(srcdir)/gdb.texinfo gdb.ps: gdb.dvi $(ECHO_DVIPS) $(DVIPS) $(SILENT_Q_FLAG) -o $@ $? gdb.pdf: ${GDB_DOC_FILES} - $(SILENCE) rm -f $(GDB_TEX_TMPS) - $(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) --pdf \ + $(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) --pdf \ $(READLINE_TEXI_INCFLAG) -I ${GDBMI_DIR} -I $(srcdir) \ $(srcdir)/gdb.texinfo @@ -596,44 +594,28 @@ stabs/index.html: $(STABS_DOC_FILES) -I $(srcdir) \ $(srcdir)/stabs.texinfo -# Clean these up before each run. Avoids a catch 22 with not being -# able to re-generate these files (to fix a corruption) because these -# files contain a corruption. -STABS_TEX_TMPS = stabs.aux stabs.cp* stabs.fn* stabs.ky* \ - stabs.log stabs.pg* stabs.toc stabs.tp* stabs.vr* - # STABS DOCUMENTATION: TeX dvi file stabs.dvi : $(STABS_DOC_FILES) - $(SILENCE) rm -f $(STABS_TEX_TMPS) - $(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) -I $(srcdir) \ + $(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) -I $(srcdir) \ $(srcdir)/stabs.texinfo stabs.ps: stabs.dvi $(ECHO_DVIPS) $(DVIPS) $(SILENT_Q_FLAG) -o $@ $? stabs.pdf: $(STABS_DOC_FILES) - $(SILENCE) rm -f $(STABS_TEX_TMPS) - $(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) --pdf -I $(srcdir) \ + $(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) --pdf -I $(srcdir) \ $(srcdir)/stabs.texinfo -# Clean these up before each run. Avoids a catch 22 with not being -# able to re-generate these files (to fix a corruption) because these -# files contain a corruption. -ANNOTATE_TEX_TMPS = annotate.aux annotate.cp* annotate.fn* annotate.ky* \ - annotate.log annotate.pg* annotate.toc annotate.tp* annotate.vr* - # ANNOTATE DOCUMENTATION: TeX dvi file annotate.dvi : $(ANNOTATE_DOC_FILES) - $(SILENCE) rm -f $(ANNOTATE_TEX_TMPS) - $(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) -I $(srcdir) \ + $(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) -I $(srcdir) \ $(srcdir)/annotate.texinfo annotate.ps: annotate.dvi $(ECHO_DVIPS) $(DVIPS) $(SILENT_Q_FLAG) -o $@ $? annotate.pdf: $(ANNOTATE_DOC_FILES) - $(SILENCE) rm -f $(ANNOTATE_TEX_TMPS) - $(ECHO_TEXI2DVI) $(TEXI2DVI) $(SILENT_Q_FLAG) --pdf -I $(srcdir) \ + $(ECHO_TEXI2DVI) $(TEXI2DVI_CMD) $(SILENT_Q_FLAG) --pdf -I $(srcdir) \ $(srcdir)/annotate.texinfo annotate.info: $(ANNOTATE_DOC_FILES) @@ -683,9 +665,7 @@ Makefile: Makefile.in $(host_makefile_frag) ../config.status mostlyclean: rm -f gdb.mm gdb.ms gdb.me links2roff - rm -f $(GDB_TEX_TMPS) - rm -f $(STABS_TEX_TMPS) - rm -f $(ANNOTATE_TEX_TMPS) + rm -fr $(TEXI2DVI_TMPDIR) rm -f refcard.sed sedref.tex sedref_dvi.* sedref_pdf.* rm -f $(POD_FILE_TMPS)