Re: Help needed with maintainer-mode

["Richard Earnshaw (lists)" <Richard.Earnshaw@arm.com>]

On 04/03/2024 14:46, Christophe Lyon via Gcc wrote:
> On Mon, 4 Mar 2024 at 12:25, Jonathan Wakely <jwakely.gcc@gmail.com> wrote:
>>
>> On Mon, 4 Mar 2024 at 10:44, Christophe Lyon via Gcc <gcc@gcc.gnu.org> wrote:
>>>
>>> Hi!
>>>
>>> On Mon, 4 Mar 2024 at 10:36, Thomas Schwinge <tschwinge@baylibre.com> wrote:
>>>>
>>>> Hi!
>>>>
>>>> On 2024-03-04T00:30:05+0000, Sam James <sam@gentoo.org> wrote:
>>>>> Mark Wielaard <mark@klomp.org> writes:
>>>>>> On Fri, Mar 01, 2024 at 05:32:15PM +0100, Christophe Lyon wrote:
>>>>>>> [...], I read
>>>>>>> https://gcc.gnu.org/wiki/Regenerating_GCC_Configuration
>>>>>>> which basically says "run autoreconf in every dir where there is a
>>>>>>> configure script"
>>>>>>> but this is not exactly what autoregen.py is doing. IIRC it is based
>>>>>>> on a script from Martin Liska, do you know/remember why it follows a
>>>>>>> different process?
>>>>>>
>>>>>> CCing Sam and Arsen who helped refine the autoregen.py script, who
>>>>>> might remember more details. We wanted a script that worked for both
>>>>>> gcc and binutils-gdb. And as far as I know autoreconf simply didn't
>>>>>> work in all directories. We also needed to skip some directories that
>>>>>> did contain a configure script, but that were imported (gotools,
>>>>>> readline, minizip).
>>>>>
>>>>> What we really need to do is, for a start, land tschwinge/aoliva's patches [0]
>>>>> for AC_CONFIG_SUBDIRS.
>>>>
>>>> Let me allocate some time this week to get that one completed.
>>>>
>>>>> Right now, the current situation is janky and it's nowhere near
>>>>> idiomatic autotools usage. It is not a comfortable experience
>>>>> interacting with it even as someone who is familiar and happy with using
>>>>> autotools otherwise.
>>>>>
>>>>> I didn't yet play with maintainer-mode myself but I also didn't see much
>>>>> point given I knew of more fundamental problems like this.
>>>>>
>>>>> [0] https://inbox.sourceware.org/gcc-patches/oril72c4yh.fsf@lxoliva.fsfla.org/
>>>>
>>>
>>> Thanks for the background. I didn't follow that discussion at that time :-)
>>>
>>> So... I was confused because I noticed many warnings when doing a simple
>>> find . -name configure |while read f; do echo $f;d=$(dirname $f) &&
>>> autoreconf -f $d && echo $d; done
>>> as suggested by https://gcc.gnu.org/wiki/Regenerating_GCC_Configuration
>>>
>>> Then I tried with autoregen.py, and saw the same.... and now just
>>> checked Sourceware's bot logs and saw the same numerous warnings at
>>> least in GCC (didn't check binutils yet). Looks like this is
>>> "expected" ....
>>>
>>> I started looking at auto-regenerating these files in our CI a couple
>>> of weeks ago, after we received several "complaints" from contributors
>>> saying that our precommit CI was useless / bothering since it didn't
>>> regenerate files, leading to false alarms.
>>> But now I'm wondering how such contributors regenerate the files
>>> impacted by their patches before committing, they probably just
>>> regenerate things in their subdir of interest, not noticing the whole
>>> picture :-(
>>>
>>> As a first step, we can probably use autoregen.py too, and declare
>>> maintainer-mode broken. However, I do notice that besides the rules
>>> about regenerating configure/Makefile.in/..., maintainer-mode is also
>>> used to update some files.
>>> In gcc:
>>> fixincludes: fixincl.x
>>> libffi: doc/version.texi
>>> libgfortran: some stuff :-)
>>> libiberty: functions.texi
>>
>> My recently proposed patch adds the first of those to gcc_update, the
>> other should be done too.
>> https://gcc.gnu.org/pipermail/gcc-patches/2024-March/647027.html
>>
> 
> This script touches files such that they appear more recent than their
> dependencies,
> so IIUC even if one uses --enable-maintainer-mode, it will have no effect.
> For auto* files, this is "fine" as we can run autoreconf or
> autoregen.py before starting configure+build, but what about other
> files?
> For instance, if we have to test a patch which implies changes to
> fixincludes/fixincl.x, how should we proceed?
> 1- git checkout (with possibly "wrong" timestamps)
> 2- apply patch-to-test
> 3- contrib/gcc_update -t
> 4- configure --enable-maintainer-mode

If you ran

git reset --hard master // restore state to 'master'
contrib/gcc_update // pull latest code

then anything coming from upstream will be touched automatically.  You really don't want to re-touch the files after patching unless you're sure they've all been patched correctly, it will break if there's anything regenerated that's missing.

R.

> 
> I guess --enable-maintainer-mode would be largely (if not completely)
> disabled by step 3 above?
> And we should probably swap steps 2 and 3, so that the effects of
> patch-to-test are handled by make?
> 
> Thanks,
> 
> Christophe
> 
>>
>>
>>>
>>> in binutils/bfd:
>>> gdb/sim
>>> bfd/po/SRC-POTFILES.in
>>> bfd/po/BLD-POTFILES.in
>>> bfd/bfd-in2.h
>>> bfd/libbfd.h
>>> bfd/libcoff.h
>>> binutils/po/POTFILES.in
>>> gas/po/POTFILES.in
>>> opcodes/i386*.h
>>> gdb/copying.c
>>> gdb/data-directory/*.xml
>>> gold/po/POTFILES.in
>>> gprof/po/POTFILES.in
>>> gfprofng/doc/version.texi
>>> ld/po/SRC-POTFILES.in
>>> ld/po/BLD-POTFILES.in
>>> ld: ldgram/ldlex... and all emulation sources
>>> libiberty/functions.texi
>>> opcodes/po/POTFILES.in
>>> opcodes/aarch64-{asm,dis,opc}-2.c
>>> opcodes/ia64 msp430 rl78 rx z8k decoders
>>>
>>> How are these files "normally" expected to be updated? Do people just
>>> manually uncomment the corresponding maintainer lines in the Makefiles
>>> and update manually?   In particular we hit issues several times with
>>> files under opcodes, that we don't regenerate currently. Maybe we
>>> could build binutils in maintainer-mode at -j1 but, well....
>>>
>>> README-maintainer-mode in binutils/gdb only mentions a problem with
>>> 'make distclean' and maintainer mode
>>> binutils/README-how-to-make-a-release indicates to use
>>> --enable-maintainer-mode, and the sample 'make' invocations do not
>>> include any -j flag, is that an indication that only -j1 is supposed
>>> to work?
>>> Similarly, the src-release.sh script does not use -j.
>>>
>>> Thanks,
>>>
>>> Christophe
>>>
>>>>
>>>> Grüße
>>>>  Thomas