From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by sourceware.org (Postfix) with ESMTPS id A9297385840C for ; Sun, 12 Dec 2021 11:32:52 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org A9297385840C Received: by mail-wm1-x329.google.com with SMTP id p18so10008784wmq.5 for ; Sun, 12 Dec 2021 03:32:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:mime-version :content-disposition; bh=Rgqs0BYyl1f8OI3+KowmdXgCeKE5PS5gk+NMpMscuHk=; b=W40mfXETz5vkHsPKuQ3gWHzJFVg2AFnLH7AVII7xEvrGu1jhINEGZtHoWWyNW3jcEK s/hjAGrhn9GS3MIP0TdYve4lCnO4Wf+Z/gWu9eQ0SBlFsmnEVIQeEIbLTTV5Rz7T3MFm plYkb1nIKq3v6I11vF6JzFqwKPtKYcG0gP8tLfYGxvMrxkj/d4JzLq2K644HOE1dqdbE TOFy+DRuj6nZrurIUQrZHyB8TOKC4MRxnh+RZTaJzE5QQN90oARYiUgxAhglSuW2KpnC ehnbPQK5XBKvzwx7jmQ6v3XYVFkIiPpy88dSWffNqrL49wZvrerKEcJAQNm9U7nSwvRX oDMQ== X-Gm-Message-State: AOAM533blKTWEBW74MbEluwio6pLO7PtlyDGHYLro+soSbnHvk9FjtUb VujpDjB/yP7BV+zgk6jOhAeL X-Google-Smtp-Source: ABdhPJz8vsLcP1XHYCRV5dkp3Q5aghQzu0n19+2CsysA6bTPPe7DwQAxSYidz10si8w/UDUr9EfQIg== X-Received: by 2002:a05:600c:1548:: with SMTP id f8mr29962607wmg.67.1639308771248; Sun, 12 Dec 2021 03:32:51 -0800 (PST) Received: from takamaka.home ([2a01:cb22:1d5:1100:c406:c160:9fa0:815c]) by smtp.gmail.com with ESMTPSA id d9sm7281902wre.52.2021.12.12.03.32.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 12 Dec 2021 03:32:50 -0800 (PST) Received: by takamaka.home (Postfix, from userid 1000) id 883D5A49EE; Sun, 12 Dec 2021 15:32:47 +0400 (+04) Date: Sun, 12 Dec 2021 15:32:47 +0400 From: Joel Brobecker To: overseers@sourceware.org Cc: Joseph Myers , Carlos O'Donell , Corinna Vinschen , Joel Brobecker Subject: transition git-hooks on sourceware.org to Python 3.x Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="rdFYWrn+gjM7bqCY" Content-Disposition: inline X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: overseers@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Overseers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Dec 2021 11:32:55 -0000 --rdFYWrn+gjM7bqCY Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hello, The current version of the git-hooks being used on sourceware is a version which uses Python 2.7. I have since then migrated to Python 3, where the minimum version is currently Python 3.8. That version has been running on AdaCore's server for a few weeks, now, so I think it's time to think about transitioning our repos on sourceware. The main improvement, besides the transition away from Python 2.x, comes from Python 3.x itself and the handling of strings vs bytes, so charsets are handled a bit better. The one small difficulty comes from making sure a recent version of Python 3.x gets used. Ideally, we should use the same Python I'm using on sourceware to run the git-hooks testsuite to make sure there aren't any compatibility issues. For this, what I'd like to propose is that we use a set of wrapper scripts which provide the same hooks as the git-hooks, and which only do the following: add the correct Python to the PATH, and then call the real hook script in git-hooks. I've created a small prototype and did some testing with it, it seems to work well. In a nutshell, we have: / - hooks/ <-- directory to use as hooks in Git bare repos - git-hooks/ <-- clone of https://github.com/adacore/git-hooks - python3-for-git-hooks/ <-- Python 3.x venv to be used by the git-hooks To transition a repository to updated hooks, we simply replace the hooks symbolic link to point to /hooks. My proptotype is at... /home/gdbadmin/brobecke/py3-git-hooks/git-hooks-wrapper ... for those who are curious and can log onto sourceware.org. There is a README.md file that gives a bit more information (I'll attach to this message as well) If agreed with the approach, I think we'll want to decide where we want this to live. Maybe /sourceware/projects/src-home/? The main issue with this location is that I worry about shared access and in particular maintenance over time. I'd like to be able to make changes there if there is a need to fix something. I'd like also to be able to continue updating the git-hooks version being used as I continue to push fixes and enhancements. I can take care of the git-hooks-wrapper setup once the location is settled. In terms of the projects themselves, I've identified the following repositories as using the git-hooks: - binutils-gdb.git/hooks - gcc.git/hooks - glibc.git/hooks - newlib-cygwin.git/hooks - valgrind.git/hooks This is based on searching for repositories on sourcware.org whose hooks dir is a symlink to the git-hooks. Not sure if there are others. I've Cc'ed people I know have asked me about the git-hooks for GCC, glibc and newlib. Not sure about valgrind. What we could do is give the binutils & GDB maintainers a quick heads up and then transition the binutils-gdb repository. If all goes well, I think we can transition the remaining ones. Thoughts? PS: If a project needs to stay on Python 2.7 for the time being, it is easy to accomodate it with the current scheme. But they will need to use a dedicated branch, as the latest no longer supports Python 2.x. -- Joel --rdFYWrn+gjM7bqCY Content-Type: text/markdown; charset=us-ascii Content-Disposition: attachment; filename="README.md" Git-Hooks Wrapper ================= This repository provices a hooks/ directory which contain the same set of scripts as the AdaCore git-hooks (https://github.com/adacore/git-hooks). Those scripts simply act as wrappers allowing us to set the environment up prior to calling the real git-hooks hooks. In particular, our wrappers allow us to make sure we use a controlled version of Python to run the hooks. Wrapper Setup Overview ---------------------- The wrappers expect the following structure: / - hooks/ <-- directory to use as hooks in Git bare repos - git-hooks/ <-- clone of https://github.com/adacore/git-hooks - python3-for-git-hooks/ <-- Python 3.x venv to be used by the git-hooks The wrappers provides wrapper scripts which will do the following: - set the environment up to put the python3-for-git-hooks virtual env in its PATH; - Call the corresponding hook in git-hooks/hooks/. Git-Hooks Setup --------------- The wrappers expect a clone of the git-hooks to be located at the root of this repository, named git-hooks. Python Setup ------------ The wrappers expect a Python 3 Virtual Enviroment (venv) to be set up at the root of this repository, named python3-for-git-hooks. The venv should be set up as follow: # Create the venv, and activate it... $ python3 -m venv python3-for-git-hooks $ . ./python3-for-git-hooks/bin/activate # Make sure we use the latest version of pip $ python3 -m pip install pip --upgrade # Install all dependencies requried by the git-hooks. # # Note that the exact list of dependencies may change # over time. If unsure, run the git-hooks testsuite # as follow: # # $ cd path/to/git-hooks/testsuite # $ ./run-validation-tests # # The testsuite will abort with an error if any dependency # is missing. $ python3 -m pip install black e3-core flake8 pre-commit pytest pytest-cov pytest-xdist Check the git-hooks' README.md file for the required minimum version of Python that must be used. Activating the Git-Hooks Wrapper -------------------------------- To use these wrappers in a Git repository, replace the hooks/ directory inside that Git repository by a symbolic link to the hooks/ directory inside the git-hooks-wrapper --rdFYWrn+gjM7bqCY--