Re: transition git-hooks on sourceware.org to Python 3.x

[Mark Wielaard <mark@klomp.org>]

Hi,

I added Ivo to the CC who setup the valgrind hooks. If he isn't
available I'll take care of them.

Cheers,

Mark

On Sun, Dec 12, 2021 at 03:32:47PM +0400, Joel Brobecker via Overseers wrote:
> Hello,
> 
> The current version of the git-hooks being used on sourceware
> is a version which uses Python 2.7. I have since then migrated
> to Python 3, where the minimum version is currently Python 3.8.
> That version has been running on AdaCore's server for a few weeks,
> now, so I think it's time to think about transitioning our repos
> on sourceware.
> 
> The main improvement, besides the transition away from Python 2.x,
> comes from Python 3.x itself and the handling of strings vs bytes,
> so charsets are handled a bit better.
> 
> The one small difficulty comes from making sure a recent version
> of Python 3.x gets used. Ideally, we should use the same Python
> I'm using on sourceware to run the git-hooks testsuite to make sure
> there aren't any compatibility issues.
> 
> For this, what I'd like to propose is that we use a set of wrapper
> scripts which provide the same hooks as the git-hooks, and which
> only do the following: add the correct Python to the PATH, and
> then call the real hook script in git-hooks.
> 
> I've created a small prototype and did some testing with it,
> it seems to work well. In a nutshell, we have:
> 
>   <wrappers root>/
>       - hooks/           <-- directory to use as hooks in Git bare repos
>       - git-hooks/       <-- clone of https://github.com/adacore/git-hooks
>       - python3-for-git-hooks/  <-- Python 3.x venv to be used by the git-hooks
> 
> To transition a repository to updated hooks, we simply replace
> the hooks symbolic link to point to <wrappers root>/hooks.
> 
> My proptotype is at...
> 
>     /home/gdbadmin/brobecke/py3-git-hooks/git-hooks-wrapper
> 
> ... for those who are curious and can log onto sourceware.org.
> There is a README.md file that gives a bit more information
> (I'll attach to this message as well)
> 
> If agreed with the approach, I think we'll want to decide where
> we want this to live. Maybe /sourceware/projects/src-home/?
> The main issue with this location is that I worry about shared
> access and in particular maintenance over time. I'd like to be
> able to make changes there if there is a need to fix something.
> I'd like also to be able to continue updating the git-hooks version
> being used as I continue to push fixes and enhancements.
> 
> I can take care of the git-hooks-wrapper setup once the location
> is settled.
> 
> In terms of the projects themselves, I've identified the following
> repositories as using the git-hooks:
> 
>   - binutils-gdb.git/hooks
>   - gcc.git/hooks
>   - glibc.git/hooks
>   - newlib-cygwin.git/hooks
>   - valgrind.git/hooks
> 
> This is based on searching for repositories on sourcware.org
> whose hooks dir is a symlink to the git-hooks. Not sure if there are
> others.
> 
> I've Cc'ed people I know have asked me about the git-hooks for
> GCC, glibc and newlib. Not sure about valgrind.
> 
> What we could do is give the binutils & GDB maintainers a quick
> heads up and then transition the binutils-gdb repository. If all
> goes well, I think we can transition the remaining ones.
> 
> Thoughts?
> 
> PS: If a project needs to stay on Python 2.7 for the time being,
>     it is easy to accomodate it with the current scheme. But they
>     will need to use a dedicated branch, as the latest no longer
>     supports Python 2.x.
> -- 
> Joel

> Git-Hooks Wrapper
> =================
> 
> This repository provices a hooks/ directory which contain the same set of
> scripts as the AdaCore git-hooks (https://github.com/adacore/git-hooks).
> Those scripts simply act as wrappers allowing us to set the environment
> up prior to calling the real git-hooks hooks. In particular, our wrappers
> allow us to make sure we use a controlled version of Python to run the hooks.
> 
> Wrapper Setup Overview
> ----------------------
> 
> The wrappers expect the following structure:
> 
>     <wrappers root>/
>         - hooks/           <-- directory to use as hooks in Git bare repos
>         - git-hooks/       <-- clone of https://github.com/adacore/git-hooks
>         - python3-for-git-hooks/  <-- Python 3.x venv to be used by the git-hooks
> 
> The wrappers provides wrapper scripts which will do the following:
> 
>   - set the environment up to put the python3-for-git-hooks virtual env
>     in its PATH;
> 
>   - Call the corresponding hook in git-hooks/hooks/.
> 
> 
> Git-Hooks Setup
> ---------------
> 
> The wrappers expect a clone of the git-hooks to be located at
> the root of this repository, named git-hooks.
> 
> 
> Python Setup
> ------------
> 
> The wrappers expect a Python 3 Virtual Enviroment (venv) to be set up
> at the root of this repository, named python3-for-git-hooks.
> 
> The venv should be set up as follow:
> 
>    # Create the venv, and activate it...
> 
>    $ python3 -m venv python3-for-git-hooks
>    $ . ./python3-for-git-hooks/bin/activate
> 
>    # Make sure we use the latest version of pip
> 
>    $ python3 -m pip install pip --upgrade
> 
>    # Install all dependencies requried by the git-hooks.
>    #
>    # Note that the exact list of dependencies may change
>    # over time. If unsure, run the git-hooks testsuite
>    # as follow:
>    #
>    #      $ cd path/to/git-hooks/testsuite
>    #      $ ./run-validation-tests
>    #
>    # The testsuite will abort with an error if any dependency
>    # is missing.
> 
>    $ python3 -m pip install black e3-core flake8 pre-commit pytest pytest-cov pytest-xdist
> 
> Check the git-hooks' README.md file for the required minimum version
> of Python that must be used.
> 
> Activating the Git-Hooks Wrapper
> --------------------------------
> 
> To use these wrappers in a Git repository, replace the hooks/ directory
> inside that Git repository by a symbolic link to the hooks/ directory
> inside the git-hooks-wrapper