From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com [IPv6:2a00:1450:4864:20::530]) by sourceware.org (Postfix) with ESMTPS id DC3DF3858417 for ; Mon, 8 Apr 2024 09:22:13 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org DC3DF3858417 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=linaro.org ARC-Filter: OpenARC Filter v1.0.0 sourceware.org DC3DF3858417 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2a00:1450:4864:20::530 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1712568136; cv=none; b=h7masO3uPowPmmA9GIHnCR2h0Ac9BmP2vnr5xpwgF5v03cZv/ycfaL6Vw8juBqSHDiuiDh01Efi00d8COnomO+ZegpBkHTZcZiWqbkj6dvZM1sdpoYomRYGUm21+H+CWKbO2bxG+U4FO5SbzFoJk7D4yzwQlVqE0lrCsJExR0jo= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1712568136; c=relaxed/simple; bh=HrQG+KjI6mzlTzvoyXWM06a2UfsKGDC16r4fxdH7HqE=; h=DKIM-Signature:MIME-Version:From:Date:Message-ID:Subject:To; b=EKwx1eMkU2FidDwm7pQxLvN4IG/pB91dbPHNL46CbhuROc6vtSFcjuwUt2VG+L+ctW3X9QybPxxMgHsTAUMnQrFHqUXerekuV64hTbNMamQ5OsO0Y0aLiEKS5sOicbBXnBJoSxxGkk685y24DeJW85r6x1s/yaOHTkDeNkxfKBE= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-ed1-x530.google.com with SMTP id 4fb4d7f45d1cf-56e56ee8d5cso1101292a12.2 for ; Mon, 08 Apr 2024 02:22:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1712568131; x=1713172931; darn=gcc.gnu.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=bFglIBPlIIalJ9fN4tcbRTDA5Zz9LmLRAdZ81/1aTvs=; b=bTXQ3u0BChJTV85T2SXAHKx1rGAXmIisfESC0xHwseOX3O7WQtsBUI/LGsEfGTydMe OkBD2xY/yD3lUkgN8ZQUlNpRUiThMWJBgwp/5zAOinBeNl6himV0D/HL3kDIATk0T+xe mZK4HVjRFzWWLGzWfADHQ3kh9OHi9G2wW1WSeeIqrr/69ZOheHcK7S5CT0OnizCMUJq9 gOXStCkIBvoY5Z6oSiV+arnIc7B6I2LZhl3M31XCVLNgNYb0+i7egAIEvYss4Cz7LlET x5DjuEEyCKWzNmgCt0WuyYoBTdekOBEs5rmivPbLmosqgG93Gj/+uSECf1qkQzCXsLD5 OSwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712568131; x=1713172931; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=bFglIBPlIIalJ9fN4tcbRTDA5Zz9LmLRAdZ81/1aTvs=; b=OP/SG3iRXVQsv2Hpn+4A0OyNzr+kuCPbvpxwibIVaqvCFDPp+6RKctPxRV56CarZE8 0FZE//H1xC7DR1YB1GX8sZuQRIx4g/mPs8SLvSiEg6S+vIFjYBP/Dp7aFm4arMQn9Lc3 D07YvAOohHem6eM0q7XKrdvWwTmV8y4zT832Y7Q1JeX8lohDW0/4ojZaeJnGEiQ2TmSP mrN8EiLq/YNrKurHqrDNPmkJuyUPMS8NJrS/ZPKpqNz0gEnjKOAT7iNJrST916p49DAy PkOE/fwDWhz1uDHzpomrcSUd8fIXkFcKSstBNYtmV13yJKEUjliA0IRu6ZmdW3+lcFQW 2lqw== X-Forwarded-Encrypted: i=1; AJvYcCV6HK4c7kmriUZB1o7sCDgDbraqWjZ/9m9HarRQsRqO3ZOr1lp5N4F3UXFu00E+bX5VxkhESPHoVdJe+zKACzs= X-Gm-Message-State: AOJu0YyurH6zQwTAHBw86m27bHK2I0mkO86/eCyZGbD+/s7QYZfb/2YQ ydmn3iOvY8hJzypV/rGqeZjXu2dsJsA1fK4jFO2fiunuRsvuuN/OF0xrGyAbexaynqp6iRCI2xP Xrjd7xs6433k1iUyRGgXslwSrSr8k8BLBu3YiuA== X-Google-Smtp-Source: AGHT+IFmF2Iy0aIpsN2LJsYPwoeVXzghbeyxEYVrfMkvMDViw/uMDwy9ugilsV9u0gc3om10/RP4x4e+zMztAhksfgk= X-Received: by 2002:a17:906:e088:b0:a51:827d:c99b with SMTP id gh8-20020a170906e08800b00a51827dc99bmr5534915ejb.14.1712568131398; Mon, 08 Apr 2024 02:22:11 -0700 (PDT) MIME-Version: 1.0 References: <20240313080237.1143034-1-christophe.lyon@linaro.org> <1eb529f2-3842-4090-a8e2-f713a28f2394@simark.ca> <33f72ef7-f990-437d-8fc4-dba08d7db24b@simark.ca> In-Reply-To: From: Christophe Lyon Date: Mon, 8 Apr 2024 11:22:04 +0200 Message-ID: Subject: Re: [RFC] add regenerate Makefile target To: Simon Marchi Cc: binutils@sourceware.org, gdb@sourceware.org, gcc@gcc.gnu.org Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-4.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Hi, On Mon, 25 Mar 2024 at 15:19, Christophe Lyon wrote: > > On Thu, 21 Mar 2024 at 15:32, Christophe Lyon > wrote: > > > > On Wed, 20 Mar 2024 at 16:34, Simon Marchi wrote: > > > > > > On 3/18/24 13:25, Christophe Lyon wrote: > > > > Well the rule to regenerate Makefile.in (eg in in opcodes/) is a bit > > > > more complex > > > > than just calling automake. IIUC it calls automake --foreign it any of > > > > *.m4 file from $(am__configure_deps) that is newer than Makefile.in > > > > (with an early exit in the loop), does nothing if Makefile.am or > > > > doc/local.mk are newer than Makefile.in, and then calls 'automake > > > > --foreign Makefile' > > > > > > The rules looks complex because they've been generated by automake, this > > > Makefile.in is not written by hand. And I guess automake has put > > > `--foreign` there because foreign is used in Makefile.am: > > Yes, I know :-) > > > > > > > > AUTOMAKE_OPTIONS = foreign no-dist > > > > > > But a simple call so `automake -f` (or `autoreconf -f`) just works, as > > > automake picks up the foreign option from AUTOMAKE_OPTIONS, so a human > > > or an external script who wants to regenerate things would probably just > > > use that. > > > > Indeed. I guess my concern is: if some change happens to > > Makefile.am/Makefile.in which would imply that 'autoreconf -f' would > > not work, how do we make sure autoregen.py (or whatever script) is > > updated accordingly? Or maybe whatever change is made to > > Makefile.am/Makefile.in, 'autoreconf -f' is supposed to handle it > > without additional flag? > > > I think I've just noticed a variant of this: if you look at > opcodes/Makefile.in, you can see that aclocal.m4 depends on > configure.ac (among others). So if configure.ac is updated, a > maintainer-mode rule in Makefile.in will call aclocal and regenerate > aclocal.m4. > > However, autoregen.py calls aclocal only if configure.ac contains > AC_CONFIG_MACRO_DIRS, which is not the case here. > > That's probably a bug in opcode/configure.ac, but still the current > Makefile.in machinery would update aclocal.m4 as needed when > autoregen.py will not. > > I haven't audited all configure.ac but there are probably other > occurrences of this. > Another discrepancy I've just noticed: if you look at libsframe/Makefile.am, you can see that ACLOCAL_AMFLAGS = -I .. -I ../config -I ../bfd, so if you run autoreconf -f, it will invoke aclocal with these flags (the same is performed by the aclocal.m4 regeneration rule in the Makefile), but autoregen.py won't run aclocal because configure.ac does not define AC_CONFIG_MACRO_DIRS, and even if it did, it would only use -I../config I guess the same applies for several other subdirs. So in general how do we make sure autoregen.py uses the right flags? Or what prevents us from just using autoreconf -f? If that does not work because configure.ac/Makeline.am and others have bugs, maybe we should fix those bugs instead? which makes me think about Eric's reply: > `autoreconf -f` works fine in individual subdirectories, the problem > is that the top-level configure.ac doesn't use the AC_CONFIG_SUBDIRS > macro to specify its subdirectories, but rather uses its own > hand-rolled method of specifying subdirectories that autoreconf > doesn't know about. This means that autoreconf won't automatically > recurse into all the necessary subdirectories by itself automatically, > and instead has to be run manually in each subdirectory separately. It's not clear to me if that "problem" is a bug, or a design decision we must take into account when writing tools to help regeneration? > Also the various subdirectories are inconsistent about whether they > have a rule for running it (autoreconf) from the Makefile or not, should that be considered a bug, and fixed? > which usually comes down to whether the subdirectory uses automake for > its Makefile or not (the top-level Makefile doesn't; it uses its own > weird autogen-based regeneration method instead, which means that it > misses out on all the built-in rules that automake would implicitly > generate, including ones related to build system regeneration). Thanks, Christophe > Christophe > > > > > > > > The bot I want to put in place would regenerate things as they are > > > > supposed to be, then build and run the testsuite to make sure that > > > > what is supposed to be committed would work (if the committer > > > > regenerates everything correctly) > > > > > > For your job, would it be fine to just force-regenerate everything and > > > ignore timestamps (just like the buildbot's autoregen job wants to do)? > > > It would waste a few cycles, but it would be much simpler. > > > > > Yes, that would achieve the purpose: be able to handle as many patches > > as possible in precommit-CI. > > And as described earlier, for binutils this currently means: > > autoregen > > confgure --enable-maintainer-mode > > make all (with a low -j value otherwise we have random build failures) > > and my proposal to workaround the problem with -j is to do > > make all-bfd all-libiberty regenerate -j1 > > make all -j XXX > > > > Another possibility would be a policy change in how patches are > > submitted, to require that they contain all the autogenerated files. > > > > > > > Simon