Re: Help needed with maintainer-mode

[Richard Earnshaw <Richard.Earnshaw@arm.com>]

On 04/03/2024 15:36, Richard Earnshaw (lists) wrote:
> On 04/03/2024 14:46, Christophe Lyon via Gcc wrote:
>> On Mon, 4 Mar 2024 at 12:25, Jonathan Wakely <jwakely.gcc@gmail.com> wrote:
>>>
>>> On Mon, 4 Mar 2024 at 10:44, Christophe Lyon via Gcc <gcc@gcc.gnu.org> wrote:
>>>>
>>>> Hi!
>>>>
>>>> On Mon, 4 Mar 2024 at 10:36, Thomas Schwinge <tschwinge@baylibre.com> wrote:
>>>>>
>>>>> Hi!
>>>>>
>>>>> On 2024-03-04T00:30:05+0000, Sam James <sam@gentoo.org> wrote:
>>>>>> Mark Wielaard <mark@klomp.org> writes:
>>>>>>> On Fri, Mar 01, 2024 at 05:32:15PM +0100, Christophe Lyon wrote:
>>>>>>>> [...], I read
>>>>>>>> https://gcc.gnu.org/wiki/Regenerating_GCC_Configuration <https://gcc.gnu.org/wiki/Regenerating_GCC_Configuration>
>>>>>>>> which basically says "run autoreconf in every dir where there is a
>>>>>>>> configure script"
>>>>>>>> but this is not exactly what autoregen.py is doing. IIRC it is based
>>>>>>>> on a script from Martin Liska, do you know/remember why it follows a
>>>>>>>> different process?
>>>>>>>
>>>>>>> CCing Sam and Arsen who helped refine the autoregen.py script, who
>>>>>>> might remember more details. We wanted a script that worked for both
>>>>>>> gcc and binutils-gdb. And as far as I know autoreconf simply didn't
>>>>>>> work in all directories. We also needed to skip some directories that
>>>>>>> did contain a configure script, but that were imported (gotools,
>>>>>>> readline, minizip).
>>>>>>
>>>>>> What we really need to do is, for a start, land tschwinge/aoliva's patches [0]
>>>>>> for AC_CONFIG_SUBDIRS.
>>>>>
>>>>> Let me allocate some time this week to get that one completed.
>>>>>
>>>>>> Right now, the current situation is janky and it's nowhere near
>>>>>> idiomatic autotools usage. It is not a comfortable experience
>>>>>> interacting with it even as someone who is familiar and happy with using
>>>>>> autotools otherwise.
>>>>>>
>>>>>> I didn't yet play with maintainer-mode myself but I also didn't see much
>>>>>> point given I knew of more fundamental problems like this.
>>>>>>
>>>>>> [0] https://inbox.sourceware.org/gcc-patches/oril72c4yh.fsf@lxoliva.fsfla.org/ <https://inbox.sourceware.org/gcc-patches/oril72c4yh.fsf@lxoliva.fsfla.org/>
>>>>>
>>>>
>>>> Thanks for the background. I didn't follow that discussion at that time :-)
>>>>
>>>> So... I was confused because I noticed many warnings when doing a simple
>>>> find . -name configure |while read f; do echo $f;d=$(dirname $f) &&
>>>> autoreconf -f $d && echo $d; done
>>>> as suggested by https://gcc.gnu.org/wiki/Regenerating_GCC_Configuration <https://gcc.gnu.org/wiki/Regenerating_GCC_Configuration>
>>>>
>>>> Then I tried with autoregen.py, and saw the same.... and now just
>>>> checked Sourceware's bot logs and saw the same numerous warnings at
>>>> least in GCC (didn't check binutils yet). Looks like this is
>>>> "expected" ....
>>>>
>>>> I started looking at auto-regenerating these files in our CI a couple
>>>> of weeks ago, after we received several "complaints" from contributors
>>>> saying that our precommit CI was useless / bothering since it didn't
>>>> regenerate files, leading to false alarms.
>>>> But now I'm wondering how such contributors regenerate the files
>>>> impacted by their patches before committing, they probably just
>>>> regenerate things in their subdir of interest, not noticing the whole
>>>> picture :-(
>>>>
>>>> As a first step, we can probably use autoregen.py too, and declare
>>>> maintainer-mode broken. However, I do notice that besides the rules
>>>> about regenerating configure/Makefile.in/..., maintainer-mode is also
>>>> used to update some files.
>>>> In gcc:
>>>> fixincludes: fixincl.x
>>>> libffi: doc/version.texi
>>>> libgfortran: some stuff :-)
>>>> libiberty: functions.texi
>>>
>>> My recently proposed patch adds the first of those to gcc_update, the
>>> other should be done too.
>>> https://gcc.gnu.org/pipermail/gcc-patches/2024-March/647027.html <https://gcc.gnu.org/pipermail/gcc-patches/2024-March/647027.html>
>>>
>> 
>> This script touches files such that they appear more recent than their
>> dependencies,
>> so IIUC even if one uses --enable-maintainer-mode, it will have no effect.
>> For auto* files, this is "fine" as we can run autoreconf or
>> autoregen.py before starting configure+build, but what about other
>> files?
>> For instance, if we have to test a patch which implies changes to
>> fixincludes/fixincl.x, how should we proceed?
>> 1- git checkout (with possibly "wrong" timestamps)
>> 2- apply patch-to-test
>> 3- contrib/gcc_update -t
>> 4- configure --enable-maintainer-mode
> 
> If you ran
> 
> git reset --hard master // restore state to 'master'
> contrib/gcc_update // pull latest code
> 
> then anything coming from upstream will be touched automatically.  You really don't want to re-touch the files after patching unless you're sure they've all been patched correctly, it will break if there's anything regenerated that's missing.
> 
> R.

Alternatively, if you did 

git reset --hard master // restore state to 'master'
contrib/gcc_update // pull latest code
patch
contrib/gcc_update -t

and the second command touched files that weren't listed in the patches, then you'd know that there was something missing from the patch set.  You'd probably still need to go back to the pristine trunk version though, to get the state into something that could be maintainer-mode regenerated.

R.

> 
>> 
>> I guess --enable-maintainer-mode would be largely (if not completely)
>> disabled by step 3 above?
>> And we should probably swap steps 2 and 3, so that the effects of
>> patch-to-test are handled by make?
>> 
>> Thanks,
>> 
>> Christophe
>> 
>>>
>>>
>>>>
>>>> in binutils/bfd:
>>>> gdb/sim
>>>> bfd/po/SRC-POTFILES.in
>>>> bfd/po/BLD-POTFILES.in
>>>> bfd/bfd-in2.h
>>>> bfd/libbfd.h
>>>> bfd/libcoff.h
>>>> binutils/po/POTFILES.in
>>>> gas/po/POTFILES.in
>>>> opcodes/i386*.h
>>>> gdb/copying.c
>>>> gdb/data-directory/*.xml
>>>> gold/po/POTFILES.in
>>>> gprof/po/POTFILES.in
>>>> gfprofng/doc/version.texi
>>>> ld/po/SRC-POTFILES.in
>>>> ld/po/BLD-POTFILES.in
>>>> ld: ldgram/ldlex... and all emulation sources
>>>> libiberty/functions.texi
>>>> opcodes/po/POTFILES.in
>>>> opcodes/aarch64-{asm,dis,opc}-2.c
>>>> opcodes/ia64 msp430 rl78 rx z8k decoders
>>>>
>>>> How are these files "normally" expected to be updated? Do people just
>>>> manually uncomment the corresponding maintainer lines in the Makefiles
>>>> and update manually?   In particular we hit issues several times with
>>>> files under opcodes, that we don't regenerate currently. Maybe we
>>>> could build binutils in maintainer-mode at -j1 but, well....
>>>>
>>>> README-maintainer-mode in binutils/gdb only mentions a problem with
>>>> 'make distclean' and maintainer mode
>>>> binutils/README-how-to-make-a-release indicates to use
>>>> --enable-maintainer-mode, and the sample 'make' invocations do not
>>>> include any -j flag, is that an indication that only -j1 is supposed
>>>> to work?
>>>> Similarly, the src-release.sh script does not use -j.
>>>>
>>>> Thanks,
>>>>
>>>> Christophe
>>>>
>>>>>
>>>>> Grüße
>>>>>  Thomas
>