transition git-hooks on sourceware.org to Python 3.x

[Joel Brobecker <brobecker@adacore.com>]

[-- Attachment #1: Type: text/plain, Size: 3142 bytes --]

Hello,

The current version of the git-hooks being used on sourceware
is a version which uses Python 2.7. I have since then migrated
to Python 3, where the minimum version is currently Python 3.8.
That version has been running on AdaCore's server for a few weeks,
now, so I think it's time to think about transitioning our repos
on sourceware.

The main improvement, besides the transition away from Python 2.x,
comes from Python 3.x itself and the handling of strings vs bytes,
so charsets are handled a bit better.

The one small difficulty comes from making sure a recent version
of Python 3.x gets used. Ideally, we should use the same Python
I'm using on sourceware to run the git-hooks testsuite to make sure
there aren't any compatibility issues.

For this, what I'd like to propose is that we use a set of wrapper
scripts which provide the same hooks as the git-hooks, and which
only do the following: add the correct Python to the PATH, and
then call the real hook script in git-hooks.

I've created a small prototype and did some testing with it,
it seems to work well. In a nutshell, we have:

  <wrappers root>/
      - hooks/           <-- directory to use as hooks in Git bare repos
      - git-hooks/       <-- clone of https://github.com/adacore/git-hooks
      - python3-for-git-hooks/  <-- Python 3.x venv to be used by the git-hooks

To transition a repository to updated hooks, we simply replace
the hooks symbolic link to point to <wrappers root>/hooks.

My proptotype is at...

    /home/gdbadmin/brobecke/py3-git-hooks/git-hooks-wrapper

... for those who are curious and can log onto sourceware.org.
There is a README.md file that gives a bit more information
(I'll attach to this message as well)

If agreed with the approach, I think we'll want to decide where
we want this to live. Maybe /sourceware/projects/src-home/?
The main issue with this location is that I worry about shared
access and in particular maintenance over time. I'd like to be
able to make changes there if there is a need to fix something.
I'd like also to be able to continue updating the git-hooks version
being used as I continue to push fixes and enhancements.

I can take care of the git-hooks-wrapper setup once the location
is settled.

In terms of the projects themselves, I've identified the following
repositories as using the git-hooks:

  - binutils-gdb.git/hooks
  - gcc.git/hooks
  - glibc.git/hooks
  - newlib-cygwin.git/hooks
  - valgrind.git/hooks

This is based on searching for repositories on sourcware.org
whose hooks dir is a symlink to the git-hooks. Not sure if there are
others.

I've Cc'ed people I know have asked me about the git-hooks for
GCC, glibc and newlib. Not sure about valgrind.

What we could do is give the binutils & GDB maintainers a quick
heads up and then transition the binutils-gdb repository. If all
goes well, I think we can transition the remaining ones.

Thoughts?

PS: If a project needs to stay on Python 2.7 for the time being,
    it is easy to accomodate it with the current scheme. But they
    will need to use a dedicated branch, as the latest no longer
    supports Python 2.x.
-- 
Joel

[-- Attachment #2: README.md --]
[-- Type: text/markdown, Size: 2320 bytes --]

Git-Hooks Wrapper
=================

This repository provices a hooks/ directory which contain the same set of
scripts as the AdaCore git-hooks (https://github.com/adacore/git-hooks).
Those scripts simply act as wrappers allowing us to set the environment
up prior to calling the real git-hooks hooks. In particular, our wrappers
allow us to make sure we use a controlled version of Python to run the hooks.

Wrapper Setup Overview
----------------------

The wrappers expect the following structure:

    <wrappers root>/
        - hooks/           <-- directory to use as hooks in Git bare repos
        - git-hooks/       <-- clone of https://github.com/adacore/git-hooks
        - python3-for-git-hooks/  <-- Python 3.x venv to be used by the git-hooks

The wrappers provides wrapper scripts which will do the following:

  - set the environment up to put the python3-for-git-hooks virtual env
    in its PATH;

  - Call the corresponding hook in git-hooks/hooks/.


Git-Hooks Setup
---------------

The wrappers expect a clone of the git-hooks to be located at
the root of this repository, named git-hooks.


Python Setup
------------

The wrappers expect a Python 3 Virtual Enviroment (venv) to be set up
at the root of this repository, named python3-for-git-hooks.

The venv should be set up as follow:

   # Create the venv, and activate it...

   $ python3 -m venv python3-for-git-hooks
   $ . ./python3-for-git-hooks/bin/activate

   # Make sure we use the latest version of pip

   $ python3 -m pip install pip --upgrade

   # Install all dependencies requried by the git-hooks.
   #
   # Note that the exact list of dependencies may change
   # over time. If unsure, run the git-hooks testsuite
   # as follow:
   #
   #      $ cd path/to/git-hooks/testsuite
   #      $ ./run-validation-tests
   #
   # The testsuite will abort with an error if any dependency
   # is missing.

   $ python3 -m pip install black e3-core flake8 pre-commit pytest pytest-cov pytest-xdist

Check the git-hooks' README.md file for the required minimum version
of Python that must be used.

Activating the Git-Hooks Wrapper
--------------------------------

To use these wrappers in a Git repository, replace the hooks/ directory
inside that Git repository by a symbolic link to the hooks/ directory
inside the git-hooks-wrapper