From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com [IPv6:2a00:1450:4864:20::52b]) by sourceware.org (Postfix) with ESMTPS id 48CA03858CD1 for ; Mon, 18 Mar 2024 16:14:08 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 48CA03858CD1 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=linaro.org ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 48CA03858CD1 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2a00:1450:4864:20::52b ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710778450; cv=none; b=XVsgGxvlkDlZFlQ3XnN/4TgQtbBKBDcTldxMqJfkwqf9WZuCzM43l0e7VTbdpRWwAARL7I9JaWqRvvgQKye3aewKfOylY4d9pTXoXYlAkt+1Fuffozm0Hn+l0fCRttPV83JYbWFdOE76+faG6swiTj0n0DJFR3Q+5lLLFfXQi3I= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710778450; c=relaxed/simple; bh=r7/UmgRlzkHb1kEN8/ST8mrkgikbCe4wxdxzcXx6rqc=; h=DKIM-Signature:MIME-Version:From:Date:Message-ID:Subject:To; b=q454KyShO/PJeWaBrx7Dcmf1MFYo7HTTSZaVDvbcytFroINun7sdhqZoncI4x1Ecf52ZuTXFWRcfgj/PBQyEHiaqNL+VOPWPz8ttU2hMMX4Dp/hFJDYnxxcA4tTDNysVATZVBQ7WIMhvK7ZkTZvdRC1hyGy5zu9HAe12L5G3rbg= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-ed1-x52b.google.com with SMTP id 4fb4d7f45d1cf-56845954ffeso6124994a12.2 for ; Mon, 18 Mar 2024 09:14:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1710778447; x=1711383247; darn=sourceware.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=twTpbzyQS2/ou4/UG3w8JQeU1dQHZ+BRG8lvHotbXpQ=; b=f13v8TDCKFkPejyHCDq0K6DifUNxeSGinc45yhgQjej5aMgyLXTPzeV6ho7qVACt6v HqgG7hiJaXtui1hqtv3AJebNchT88o4hoDO0QowHAl+2oiSeb5HZMg3dOiAI2I7YshOD KVBi+7HXy16rJrrzkPmFLmd7MyoPaTgLLMzWUvSkbfbnNQvpOmGniGcup4I1PP0I2dUr rNjhsQg1UD+0WzkTiwHWmyWWCuDmQ0J+/SkCSbWwNZYGEZkCKWQyvLJE81FTP6vg2R+L 9FYMOVFTxjlhVwKEIYiQwvqsm0jt1ia0c46TaIvMkakce+D6b0FVqJwR6dnRDpDJtwTb ms8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710778447; x=1711383247; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=twTpbzyQS2/ou4/UG3w8JQeU1dQHZ+BRG8lvHotbXpQ=; b=Bs0lm+0apxcg4UbyGKIPl2csTrPt67wn2YT732GNlL/B1FggC/gP+nNA8r+iH44DCK oV44lEUrvVOKR6J6sOELONxwGreXT/HmlmCebILsbzIh3JNQsGN1oM3v9V0M5YkhqeRk x0n7T/hb8SzJupbcK8AklsWxYpIagTzdxnKv+2ivyOqU22k9ur9IHuQLOOT4n1wSsWFB 2JCAw2kGgIlTXl4qIUf5kfv7CxvR1Jx3EJUFFoa3XZchzraMNSmAxbxWP7PFOlssNXq2 zZukTJ2EVEjFAnjtSp3TIkgFeC4kUVV47QM7KkmMCxfybJkr+Od7muaX5P4g70lmsCN1 wF7A== X-Forwarded-Encrypted: i=1; AJvYcCXHwiPoMRal334L8dnwscXUrTv3wEp2GXBCwsX6lJWEtBgyQ3bp5t6+eCJOBya5URjao/pHWdnr/r835BlHoEmSFic= X-Gm-Message-State: AOJu0YybaNAuVHzSOsz89cL58mCPflv9EY6xn9xc1U8Mg1aKjZgMGwrt mYs+fUQDiU4IhTOcsm5peZfO8pGwAY8ukstaJb80G5O0U5vN16/qLNr9iCQ1wJ3TYtO0b79LRhK 31RZuyntDu+WG65L0PttfQQYa9sS+7Qz4L7hjmg== X-Google-Smtp-Source: AGHT+IHyWbheeTGzFYznqm1EqOsR1oiaMF80+BxDYXNMhuLM0/CjATJUzIqiWYM9M7FuH61CDBtfN1ndsy3ntLXSbk4= X-Received: by 2002:a05:6402:1f81:b0:565:9c76:73c3 with SMTP id c1-20020a0564021f8100b005659c7673c3mr9937376edc.29.1710778446809; Mon, 18 Mar 2024 09:14:06 -0700 (PDT) MIME-Version: 1.0 References: <20240313080237.1143034-1-christophe.lyon@linaro.org> <1eb529f2-3842-4090-a8e2-f713a28f2394@simark.ca> In-Reply-To: From: Christophe Lyon Date: Mon, 18 Mar 2024 17:13:56 +0100 Message-ID: Subject: Re: [RFC] add regenerate Makefile target To: Eric Gallager Cc: Simon Marchi , binutils@sourceware.org, gdb@sourceware.org, gcc@gcc.gnu.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Fri, 15 Mar 2024 at 15:13, Eric Gallager wrote: > > On Fri, Mar 15, 2024 at 4:53=E2=80=AFAM Christophe Lyon via Gcc wrote: > > > > On Thu, 14 Mar 2024 at 19:10, Simon Marchi wrote: > > > > > > > > > > > > On 2024-03-13 04:02, Christophe Lyon via Gdb wrote: > > > > Hi! > > > > > > > > After recent discussions on IRC and on the lists about maintainer-m= ode > > > > and various problems with auto-generated source files, I've written > > > > this small prototype. > > > > > > > > Based on those discussions, I assumed that people generally want to > > > > update autotools files using a script similar to autoregen.py, whic= h > > > > takes care of running aclocal, autoheader, automake and autoconf as > > > > appropriate. > > > > > > > > What is currently missing is a "simple" way of regenerating other > > > > files, which happens normally with --enable-maintainer-mode (which = is > > > > reportedly broken). This patch as a "regenerate" Makefile target > > > > which can be called to update those files, provided > > > > --enable-maintainer-mode is used. > > > > > > > > I tried this approach with the following workflow for binutils/gdb: > > > > - run autoregen.py in srcdir > > > > - cd builddir > > > > - configure --enable-maintainer-mode > > > > - make all-bfd all-libiberty regenerate -j1 > > > > - for gdb: make all -C gdb/data-directory -j1 > > > > - make all -jXXX > > > > > > > > Making 'all' in bfd and libiberty is needed by some XXX-gen host > > > > programs in opcodes. > > > > > > > > The advantage (for instance for CI) is that we can regenerate files= at > > > > -j1, thus avoiding the existing race conditions, and build the rest > > > > with -j XXX. > > > > > > > > Among drawbacks: > > > > - most sub-components use Makefile.am, but gdb does not: this may m= ake > > > > maintenance more complex (different rules for different projects) > > > > - maintaining such ad-hoc "regenerate" rules would require special > > > > attention from maintainers/reviewers > > > > - dependency on -all-bfd and all-libiberty is probably not fully > > > > intuitive, but should not be a problem if the "regenerate" rules > > > > are used after a full build for instance > > > > > > > > Of course Makefile.def/Makefile.tpl would need further cleanup as I > > > > didn't try to take gcc into account is this patch. > > > > > > > > Thoughts? > > > > > > My first thought it: why is it a Makefile target, instead of some scr= ipt > > > on the side (like autoregen.sh). It would be nice / useful to be > > > able to it without configuring / building anything. For instance, th= e > > > autoregen buildbot job could run it without configuring anything. > > > Ideally, the buildbot wouldn't maintain its own autoregen.py file on = the > > > side, it would just use whatever is in the repo. > > > > Firstly because of what you mention later: some regeneration steps > > require building host tools first, like the XXX-gen in opcodes. > > > > Since the existing Makefiles already contain the rules to autoregen > > all these files, it seemed natural to me to reuse them, to avoid > > reinventing the wheel with the risk of introducing new bugs. > > > > This involves changes in places where I've never looked at before, so > > I'd rather reuse as much existing support as possible. > > > > For instance, there are the generators in opcodes/, but also things in > > sim/, bfd/, updates to the docs and potfiles. In gcc, there's also > > something "unusual" in fixincludes/ and libgfortran/ > > > > In fact, I considered also including 'configure', 'Makefile.in', > > etc... in the 'regenerate' target, it does not seem natural to me to > > invoke a script on the side, where you have to replicate the behaviour > > of existing Makefiles, possibly getting out-of-sync when someone > > forgets to update either Makefile or autoregen.py. What is currently > > missing is a way to easily regenerate files without having to run a > > full 'make all' (which currently takes care of calling autoconf & > > friends to update configure/Makefile.in). > > > > But yeah, having to configure before being able to regenerate files is > > a bit awkward too :-) > > > > > > > > > > Looking at the rule to re-generate copying.c in gdb for instance: > > > > > > # Make copying.c from COPYING > > > $(srcdir)/copying.c: @MAINTAINER_MODE_TRUE@ $(srcdir)/../COPYING3= $(srcdir)/copying.awk > > > awk -f $(srcdir)/copying.awk \ > > > < $(srcdir)/../COPYING3 > $(srcdir)/copying.tmp > > > mv $(srcdir)/copying.tmp $(srcdir)/copying.c > > > > > > There is nothing in this code that requires having configured the sou= rce > > > tree. This code could for instance be moved to some > > > generate-copying-c.sh script. generate-copying-c.sh could be called = by > > > an hypothetical autoregen.sh script, as well as the copying.c Makefil= e > > > target, if we want to continue supporting the maintainer mode. > > Wouldn't it be more obscure than now? Currently such build rules are > > all in the relevant Makefile. You'd have to open several scripts to > > discover what's involved with updating copying.c > > > > Yeah I agree that it's good to keep all build rules in the Makefile; > if there's a possibility of something changing, things that depend > upon it need to know, and the best way to express those dependencies > is in the Makefile. > > > > > > > Much like your regenerate targets, an autoregen.sh script in a given > > > directory would be responsible to re-generate all the files in this > > > directory that are generated and checked in git. It would also be > > > responsible to call any autoregen.sh file in subdirectories. > > Makefiles already have all that in place :-) > > Except if you consider that you'd want to ignore timestamps and always > > regenerate things? > > > > > > > There's just the issue of files that are generated using tools that a= re > > > compiled. When experimenting with maintainer mode the other day, I > > > stumbled on the opcodes/i386-gen, for instance. I don't have a good > > > solution to that, except to rewrite these tools in a scripting langua= ge > > > like Python. > > > > So for opcodes, it currently means rewriting such programs for i386, > > aarch64, ia64 and luckily msp430/rl78/rx share the same opc2c > > generator. > > Also there are the files generated by cgen, too, which no one seems to I had heard about cgen ages ago but I had forgotten about them. And your message made me discover configure's option --enable-cgen-maint :-= ) > know how to regenerate, either. And then in bfd there's that chew > program in the doc subdir. And then in the binutils subdirectory > proper there's that sysinfo tool for generating sysroff.[ch]. > > > Not sure how to find volunteers? > > > > Christophe > > > > > > > > Simon