public inbox for
 help / color / mirror / Atom feed
From: Joel Brobecker <>
Cc: Joseph Myers <>,
	Carlos O'Donell <>,
	Corinna Vinschen <>,
	Joel Brobecker <>
Subject: transition git-hooks on to Python 3.x
Date: Sun, 12 Dec 2021 15:32:47 +0400	[thread overview]
Message-ID: <> (raw)

[-- Attachment #1: Type: text/plain, Size: 3142 bytes --]


The current version of the git-hooks being used on sourceware
is a version which uses Python 2.7. I have since then migrated
to Python 3, where the minimum version is currently Python 3.8.
That version has been running on AdaCore's server for a few weeks,
now, so I think it's time to think about transitioning our repos
on sourceware.

The main improvement, besides the transition away from Python 2.x,
comes from Python 3.x itself and the handling of strings vs bytes,
so charsets are handled a bit better.

The one small difficulty comes from making sure a recent version
of Python 3.x gets used. Ideally, we should use the same Python
I'm using on sourceware to run the git-hooks testsuite to make sure
there aren't any compatibility issues.

For this, what I'd like to propose is that we use a set of wrapper
scripts which provide the same hooks as the git-hooks, and which
only do the following: add the correct Python to the PATH, and
then call the real hook script in git-hooks.

I've created a small prototype and did some testing with it,
it seems to work well. In a nutshell, we have:

  <wrappers root>/
      - hooks/           <-- directory to use as hooks in Git bare repos
      - git-hooks/       <-- clone of
      - python3-for-git-hooks/  <-- Python 3.x venv to be used by the git-hooks

To transition a repository to updated hooks, we simply replace
the hooks symbolic link to point to <wrappers root>/hooks.

My proptotype is at...


... for those who are curious and can log onto
There is a file that gives a bit more information
(I'll attach to this message as well)

If agreed with the approach, I think we'll want to decide where
we want this to live. Maybe /sourceware/projects/src-home/?
The main issue with this location is that I worry about shared
access and in particular maintenance over time. I'd like to be
able to make changes there if there is a need to fix something.
I'd like also to be able to continue updating the git-hooks version
being used as I continue to push fixes and enhancements.

I can take care of the git-hooks-wrapper setup once the location
is settled.

In terms of the projects themselves, I've identified the following
repositories as using the git-hooks:

  - binutils-gdb.git/hooks
  - gcc.git/hooks
  - glibc.git/hooks
  - newlib-cygwin.git/hooks
  - valgrind.git/hooks

This is based on searching for repositories on
whose hooks dir is a symlink to the git-hooks. Not sure if there are

I've Cc'ed people I know have asked me about the git-hooks for
GCC, glibc and newlib. Not sure about valgrind.

What we could do is give the binutils & GDB maintainers a quick
heads up and then transition the binutils-gdb repository. If all
goes well, I think we can transition the remaining ones.


PS: If a project needs to stay on Python 2.7 for the time being,
    it is easy to accomodate it with the current scheme. But they
    will need to use a dedicated branch, as the latest no longer
    supports Python 2.x.

[-- Attachment #2: --]
[-- Type: text/markdown, Size: 2320 bytes --]

Git-Hooks Wrapper

This repository provices a hooks/ directory which contain the same set of
scripts as the AdaCore git-hooks (
Those scripts simply act as wrappers allowing us to set the environment
up prior to calling the real git-hooks hooks. In particular, our wrappers
allow us to make sure we use a controlled version of Python to run the hooks.

Wrapper Setup Overview

The wrappers expect the following structure:

    <wrappers root>/
        - hooks/           <-- directory to use as hooks in Git bare repos
        - git-hooks/       <-- clone of
        - python3-for-git-hooks/  <-- Python 3.x venv to be used by the git-hooks

The wrappers provides wrapper scripts which will do the following:

  - set the environment up to put the python3-for-git-hooks virtual env
    in its PATH;

  - Call the corresponding hook in git-hooks/hooks/.

Git-Hooks Setup

The wrappers expect a clone of the git-hooks to be located at
the root of this repository, named git-hooks.

Python Setup

The wrappers expect a Python 3 Virtual Enviroment (venv) to be set up
at the root of this repository, named python3-for-git-hooks.

The venv should be set up as follow:

   # Create the venv, and activate it...

   $ python3 -m venv python3-for-git-hooks
   $ . ./python3-for-git-hooks/bin/activate

   # Make sure we use the latest version of pip

   $ python3 -m pip install pip --upgrade

   # Install all dependencies requried by the git-hooks.
   # Note that the exact list of dependencies may change
   # over time. If unsure, run the git-hooks testsuite
   # as follow:
   #      $ cd path/to/git-hooks/testsuite
   #      $ ./run-validation-tests
   # The testsuite will abort with an error if any dependency
   # is missing.

   $ python3 -m pip install black e3-core flake8 pre-commit pytest pytest-cov pytest-xdist

Check the git-hooks' file for the required minimum version
of Python that must be used.

Activating the Git-Hooks Wrapper

To use these wrappers in a Git repository, replace the hooks/ directory
inside that Git repository by a symbolic link to the hooks/ directory
inside the git-hooks-wrapper

             reply	other threads:[~2021-12-12 11:32 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-12-12 11:32 Joel Brobecker [this message]
2021-12-13  9:23 ` Mark Wielaard
2021-12-15  9:30   ` Ivo Raisr
2021-12-16  2:16     ` Joel Brobecker
2021-12-13 11:35 ` Corinna Vinschen
2021-12-13 11:47   ` Joel Brobecker
2021-12-13 22:20 ` Joseph Myers
2021-12-14  3:39   ` Joel Brobecker
2021-12-17  2:46 ` Joel Brobecker
2021-12-19  4:45   ` Joel Brobecker
2021-12-20  5:10     ` Joel Brobecker
2022-01-16 10:06     ` Joel Brobecker
2022-01-17 10:02       ` Corinna Vinschen
2022-01-20  5:27         ` Joel Brobecker
2022-01-20 10:04           ` Corinna Vinschen
2022-01-20 13:54             ` Joel Brobecker

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).