From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi1-x233.google.com (mail-oi1-x233.google.com [IPv6:2607:f8b0:4864:20::233]) by sourceware.org (Postfix) with ESMTPS id CDBAC3858C41 for ; Fri, 15 Mar 2024 14:13:14 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org CDBAC3858C41 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=gwmail.gwu.edu Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gwmail.gwu.edu ARC-Filter: OpenARC Filter v1.0.0 sourceware.org CDBAC3858C41 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::233 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710511996; cv=none; b=qm4+jzAG3xBJx3irRT8UFRube+iEi51hKsgg2Ckf5v9R8mnp7RTMiUil4G1OaHa0sgbMzDy1KlCof5xeLDffFo8PHF+EZKdp39I0FJJ7XQufmF98JpAPJClIryPIziELgFtcpVIw9f/WaKVwHrMv9dPodbUaTh66Y2R5B63g9k0= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710511996; c=relaxed/simple; bh=dQVV30Pk3l+r5WKQrKbNXFdBIPGIBuCn8eZxUBXEHSM=; h=DKIM-Signature:MIME-Version:From:Date:Message-ID:Subject:To; b=bisyzPtWpaANMKruZQD4dkgkPdqh5RkQYuewaMitHayyesoIinXmiKr9vyxcW4M2hTzDla2BWnY1r19VKaV4frj8fW2L9nbpX9csNLPs+Hmowj69NltFmBDTuocZYcJ390ae9bIwul1ypzspUaTgseD+wEYVQkHs1NWZ4uMhf10= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-oi1-x233.google.com with SMTP id 5614622812f47-3c1ea5e54d3so1149336b6e.0 for ; Fri, 15 Mar 2024 07:13:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gwmail.gwu.edu; s=google; t=1710511994; x=1711116794; darn=gcc.gnu.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=LJwsp+ST9nGtoPziifW8xDopyfpRwpSOvhBXuPX5ac0=; b=FrLJZFEMTL5QMWU/xTVA1XV72ba9zTUTVjCjWrU7foH2ZhkoKC9m9lbSWZOPPdE+yw YVUePNng06MFuOyeLaEKo2n54eJh7M0u73NdMlmRf2nqemYguCW6tL0slEsdSH1ehlj2 ipty7NkX6p95VbXtfd/PXGo6fIhg8s0fPhBSVk512oQTmixdaOM9SxjRQZU7TS05KQub Fxn1K+pK7pYles1iMjZ3AgW0NtE2nHKUNbR8qL6FT4aA0TyfbAMyUYe4/SGsrAsMt/BA 4QR6JRn1NJNZKbss0Os00QMFAD3ZH4x7iLIIJIh26HZkw2VeRj1hgTzmV4isF0ZSPGR1 VJFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710511994; x=1711116794; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LJwsp+ST9nGtoPziifW8xDopyfpRwpSOvhBXuPX5ac0=; b=duFbT7c4MIz98BGjgZ+oN2YptQg70GMALjRBxhfi/oddCXTLnV7baCaevHgfkTLa9n 5hPEMgpRMkZJspt5QNlxBKdiyND9HO7YaAwxBEUV3o7RgksuW7zrzpb4eFnKAfzRPOAx QwNbMjpUtk4Z4ttRFqoTpbsHCfROFZ2P+kQ0Seyp+SFtc6EQhhOyLa85tUN9QkNsQuwB 5GEBAkrGtHe+eUqwee+ioJwgGL8yfrp+AGoiEHnCnIC7pxt5XhEVPaQIBIpI+wqL+6lJ skMlb0Tb8gq4bgSK4cEIcVEbBioQxOufzVQvsAXLlu48hw8IlduMGi0KvnF7LLDO3KTP sB7Q== X-Forwarded-Encrypted: i=1; AJvYcCU2Tryvs4G2h+5HQ8LI0Yq2eD5ICwN1NZOUypW6zuD2HpP8EFU6BfmIBq22O+wmfCseqoZuvjXahToqZyiRmJc= X-Gm-Message-State: AOJu0Yx4YVaUyA7YtTrEe6CHgcReRf62DgOnbS0AXXvy+h2AXXAZIycR NExaZLUbqqEeRQVOBBh0UN7JQCKcuJ23cPMoQErD2d/dMzCsvpZ+1OMbq0G8+lURjLrm4UzZNpc 3QT09jcVNFnvxrTpXFhGH/Tes8VhXw+j7tANa X-Google-Smtp-Source: AGHT+IGPf+JL7Ywwul7JNhGxGhHd2+0fagNKdaBUpMywleDoxgi27Phd2z5c+VDFPX1r2oonPdfwFTQaPCoCXONO2Vo= X-Received: by 2002:a05:6808:23c6:b0:3c2:5c1a:d721 with SMTP id bq6-20020a05680823c600b003c25c1ad721mr5986911oib.17.1710511993822; Fri, 15 Mar 2024 07:13:13 -0700 (PDT) MIME-Version: 1.0 References: <20240313080237.1143034-1-christophe.lyon@linaro.org> <1eb529f2-3842-4090-a8e2-f713a28f2394@simark.ca> In-Reply-To: From: Eric Gallager Date: Fri, 15 Mar 2024 10:13:02 -0400 Message-ID: Subject: Re: [RFC] add regenerate Makefile target To: Christophe Lyon Cc: Simon Marchi , binutils@sourceware.org, gdb@sourceware.org, gcc@gcc.gnu.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,JMQ_SPF_NEUTRAL,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Fri, Mar 15, 2024 at 4:53=E2=80=AFAM Christophe Lyon via Gcc wrote: > > On Thu, 14 Mar 2024 at 19:10, Simon Marchi wrote: > > > > > > > > On 2024-03-13 04:02, Christophe Lyon via Gdb wrote: > > > Hi! > > > > > > After recent discussions on IRC and on the lists about maintainer-mod= e > > > and various problems with auto-generated source files, I've written > > > this small prototype. > > > > > > Based on those discussions, I assumed that people generally want to > > > update autotools files using a script similar to autoregen.py, which > > > takes care of running aclocal, autoheader, automake and autoconf as > > > appropriate. > > > > > > What is currently missing is a "simple" way of regenerating other > > > files, which happens normally with --enable-maintainer-mode (which is > > > reportedly broken). This patch as a "regenerate" Makefile target > > > which can be called to update those files, provided > > > --enable-maintainer-mode is used. > > > > > > I tried this approach with the following workflow for binutils/gdb: > > > - run autoregen.py in srcdir > > > - cd builddir > > > - configure --enable-maintainer-mode > > > - make all-bfd all-libiberty regenerate -j1 > > > - for gdb: make all -C gdb/data-directory -j1 > > > - make all -jXXX > > > > > > Making 'all' in bfd and libiberty is needed by some XXX-gen host > > > programs in opcodes. > > > > > > The advantage (for instance for CI) is that we can regenerate files a= t > > > -j1, thus avoiding the existing race conditions, and build the rest > > > with -j XXX. > > > > > > Among drawbacks: > > > - most sub-components use Makefile.am, but gdb does not: this may mak= e > > > maintenance more complex (different rules for different projects) > > > - maintaining such ad-hoc "regenerate" rules would require special > > > attention from maintainers/reviewers > > > - dependency on -all-bfd and all-libiberty is probably not fully > > > intuitive, but should not be a problem if the "regenerate" rules > > > are used after a full build for instance > > > > > > Of course Makefile.def/Makefile.tpl would need further cleanup as I > > > didn't try to take gcc into account is this patch. > > > > > > Thoughts? > > > > My first thought it: why is it a Makefile target, instead of some scrip= t > > on the side (like autoregen.sh). It would be nice / useful to be > > able to it without configuring / building anything. For instance, the > > autoregen buildbot job could run it without configuring anything. > > Ideally, the buildbot wouldn't maintain its own autoregen.py file on th= e > > side, it would just use whatever is in the repo. > > Firstly because of what you mention later: some regeneration steps > require building host tools first, like the XXX-gen in opcodes. > > Since the existing Makefiles already contain the rules to autoregen > all these files, it seemed natural to me to reuse them, to avoid > reinventing the wheel with the risk of introducing new bugs. > > This involves changes in places where I've never looked at before, so > I'd rather reuse as much existing support as possible. > > For instance, there are the generators in opcodes/, but also things in > sim/, bfd/, updates to the docs and potfiles. In gcc, there's also > something "unusual" in fixincludes/ and libgfortran/ > > In fact, I considered also including 'configure', 'Makefile.in', > etc... in the 'regenerate' target, it does not seem natural to me to > invoke a script on the side, where you have to replicate the behaviour > of existing Makefiles, possibly getting out-of-sync when someone > forgets to update either Makefile or autoregen.py. What is currently > missing is a way to easily regenerate files without having to run a > full 'make all' (which currently takes care of calling autoconf & > friends to update configure/Makefile.in). > > But yeah, having to configure before being able to regenerate files is > a bit awkward too :-) > > > > > > Looking at the rule to re-generate copying.c in gdb for instance: > > > > # Make copying.c from COPYING > > $(srcdir)/copying.c: @MAINTAINER_MODE_TRUE@ $(srcdir)/../COPYING3 $= (srcdir)/copying.awk > > awk -f $(srcdir)/copying.awk \ > > < $(srcdir)/../COPYING3 > $(srcdir)/copying.tmp > > mv $(srcdir)/copying.tmp $(srcdir)/copying.c > > > > There is nothing in this code that requires having configured the sourc= e > > tree. This code could for instance be moved to some > > generate-copying-c.sh script. generate-copying-c.sh could be called by > > an hypothetical autoregen.sh script, as well as the copying.c Makefile > > target, if we want to continue supporting the maintainer mode. > Wouldn't it be more obscure than now? Currently such build rules are > all in the relevant Makefile. You'd have to open several scripts to > discover what's involved with updating copying.c > Yeah I agree that it's good to keep all build rules in the Makefile; if there's a possibility of something changing, things that depend upon it need to know, and the best way to express those dependencies is in the Makefile. > > > > Much like your regenerate targets, an autoregen.sh script in a given > > directory would be responsible to re-generate all the files in this > > directory that are generated and checked in git. It would also be > > responsible to call any autoregen.sh file in subdirectories. > Makefiles already have all that in place :-) > Except if you consider that you'd want to ignore timestamps and always > regenerate things? > > > > There's just the issue of files that are generated using tools that are > > compiled. When experimenting with maintainer mode the other day, I > > stumbled on the opcodes/i386-gen, for instance. I don't have a good > > solution to that, except to rewrite these tools in a scripting language > > like Python. > > So for opcodes, it currently means rewriting such programs for i386, > aarch64, ia64 and luckily msp430/rl78/rx share the same opc2c > generator. Also there are the files generated by cgen, too, which no one seems to know how to regenerate, either. And then in bfd there's that chew program in the doc subdir. And then in the binutils subdirectory proper there's that sysinfo tool for generating sysroff.[ch]. > Not sure how to find volunteers? > > Christophe > > > > > Simon