From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gnu.wildebeest.org (gnu.wildebeest.org [45.83.234.184]) by sourceware.org (Postfix) with ESMTPS id 38BEE385840C for ; Mon, 13 Dec 2021 09:23:05 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 38BEE385840C Received: from reform (deer0x12.wildebeest.org [172.31.17.148]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by gnu.wildebeest.org (Postfix) with ESMTPSA id 5956330002FC; Mon, 13 Dec 2021 10:23:03 +0100 (CET) Received: by reform (Postfix, from userid 1000) id 1E35C2E82279; Mon, 13 Dec 2021 10:23:02 +0100 (CET) Date: Mon, 13 Dec 2021 10:23:02 +0100 From: Mark Wielaard To: Overseers mailing list Cc: Joel Brobecker , Carlos O'Donell , Corinna Vinschen , Joseph Myers , Ivo Raisr Subject: Re: transition git-hooks on sourceware.org to Python 3.x Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-4.0 required=5.0 tests=BAYES_00, KAM_DMARC_STATUS, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: overseers@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Overseers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Dec 2021 09:23:07 -0000 Hi, I added Ivo to the CC who setup the valgrind hooks. If he isn't available I'll take care of them. Cheers, Mark On Sun, Dec 12, 2021 at 03:32:47PM +0400, Joel Brobecker via Overseers wrote: > Hello, > > The current version of the git-hooks being used on sourceware > is a version which uses Python 2.7. I have since then migrated > to Python 3, where the minimum version is currently Python 3.8. > That version has been running on AdaCore's server for a few weeks, > now, so I think it's time to think about transitioning our repos > on sourceware. > > The main improvement, besides the transition away from Python 2.x, > comes from Python 3.x itself and the handling of strings vs bytes, > so charsets are handled a bit better. > > The one small difficulty comes from making sure a recent version > of Python 3.x gets used. Ideally, we should use the same Python > I'm using on sourceware to run the git-hooks testsuite to make sure > there aren't any compatibility issues. > > For this, what I'd like to propose is that we use a set of wrapper > scripts which provide the same hooks as the git-hooks, and which > only do the following: add the correct Python to the PATH, and > then call the real hook script in git-hooks. > > I've created a small prototype and did some testing with it, > it seems to work well. In a nutshell, we have: > > / > - hooks/ <-- directory to use as hooks in Git bare repos > - git-hooks/ <-- clone of https://github.com/adacore/git-hooks > - python3-for-git-hooks/ <-- Python 3.x venv to be used by the git-hooks > > To transition a repository to updated hooks, we simply replace > the hooks symbolic link to point to /hooks. > > My proptotype is at... > > /home/gdbadmin/brobecke/py3-git-hooks/git-hooks-wrapper > > ... for those who are curious and can log onto sourceware.org. > There is a README.md file that gives a bit more information > (I'll attach to this message as well) > > If agreed with the approach, I think we'll want to decide where > we want this to live. Maybe /sourceware/projects/src-home/? > The main issue with this location is that I worry about shared > access and in particular maintenance over time. I'd like to be > able to make changes there if there is a need to fix something. > I'd like also to be able to continue updating the git-hooks version > being used as I continue to push fixes and enhancements. > > I can take care of the git-hooks-wrapper setup once the location > is settled. > > In terms of the projects themselves, I've identified the following > repositories as using the git-hooks: > > - binutils-gdb.git/hooks > - gcc.git/hooks > - glibc.git/hooks > - newlib-cygwin.git/hooks > - valgrind.git/hooks > > This is based on searching for repositories on sourcware.org > whose hooks dir is a symlink to the git-hooks. Not sure if there are > others. > > I've Cc'ed people I know have asked me about the git-hooks for > GCC, glibc and newlib. Not sure about valgrind. > > What we could do is give the binutils & GDB maintainers a quick > heads up and then transition the binutils-gdb repository. If all > goes well, I think we can transition the remaining ones. > > Thoughts? > > PS: If a project needs to stay on Python 2.7 for the time being, > it is easy to accomodate it with the current scheme. But they > will need to use a dedicated branch, as the latest no longer > supports Python 2.x. > -- > Joel > Git-Hooks Wrapper > ================= > > This repository provices a hooks/ directory which contain the same set of > scripts as the AdaCore git-hooks (https://github.com/adacore/git-hooks). > Those scripts simply act as wrappers allowing us to set the environment > up prior to calling the real git-hooks hooks. In particular, our wrappers > allow us to make sure we use a controlled version of Python to run the hooks. > > Wrapper Setup Overview > ---------------------- > > The wrappers expect the following structure: > > / > - hooks/ <-- directory to use as hooks in Git bare repos > - git-hooks/ <-- clone of https://github.com/adacore/git-hooks > - python3-for-git-hooks/ <-- Python 3.x venv to be used by the git-hooks > > The wrappers provides wrapper scripts which will do the following: > > - set the environment up to put the python3-for-git-hooks virtual env > in its PATH; > > - Call the corresponding hook in git-hooks/hooks/. > > > Git-Hooks Setup > --------------- > > The wrappers expect a clone of the git-hooks to be located at > the root of this repository, named git-hooks. > > > Python Setup > ------------ > > The wrappers expect a Python 3 Virtual Enviroment (venv) to be set up > at the root of this repository, named python3-for-git-hooks. > > The venv should be set up as follow: > > # Create the venv, and activate it... > > $ python3 -m venv python3-for-git-hooks > $ . ./python3-for-git-hooks/bin/activate > > # Make sure we use the latest version of pip > > $ python3 -m pip install pip --upgrade > > # Install all dependencies requried by the git-hooks. > # > # Note that the exact list of dependencies may change > # over time. If unsure, run the git-hooks testsuite > # as follow: > # > # $ cd path/to/git-hooks/testsuite > # $ ./run-validation-tests > # > # The testsuite will abort with an error if any dependency > # is missing. > > $ python3 -m pip install black e3-core flake8 pre-commit pytest pytest-cov pytest-xdist > > Check the git-hooks' README.md file for the required minimum version > of Python that must be used. > > Activating the Git-Hooks Wrapper > -------------------------------- > > To use these wrappers in a Git repository, replace the hooks/ directory > inside that Git repository by a symbolic link to the hooks/ directory > inside the git-hooks-wrapper