From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ed1-f49.google.com (mail-ed1-f49.google.com [209.85.208.49]) by sourceware.org (Postfix) with ESMTPS id BF2FB385802A for ; Wed, 15 Dec 2021 09:31:12 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org BF2FB385802A Received: by mail-ed1-f49.google.com with SMTP id t5so72260832edd.0 for ; Wed, 15 Dec 2021 01:31:12 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=2iVvWDtBJtUltjA/fw44bSmPSRuj7mmUW7iPwLe1zyg=; b=AbGeDXbDIzuNVTh3FGNB4BJT5mflPX5rJUQgLZ0wRdIxD5YsEn7svz7o3BIa1N1nzu TcZnTQZR5RxDmugQDaJ7PecD7cKZKb5vix3Rvf+fkjc9Va+xXKug6oJDt2og1mccAXtn Kv+XIgrxrGZ3BqCe8EcMKoLywxFFtt/F3AOaaGhAhJDnHQF0KgVo40vS6vqa9SiymiyL p8qrwEnl75D+cydTMZWc+C40wXoSpH82rbDU31sHLjg6ex2XcL8wZAO79+j9taMcr87L kxYoH9ZTJ+rRM0clqsZLQO6q901xWs6dsAdI3zZ2Rm6kBZ2+A0gqnCJoO91wi4I4E/5D 9qYQ== X-Gm-Message-State: AOAM532R8ikgQmZ5UqtmaXV2KYuq0Y/1M8pwLFEoWAv2gtxSKD5XqcHB GdFs+D19v9RamT5XuYnQieMX5Vdxx6IDnQ== X-Google-Smtp-Source: ABdhPJywHmksQrl+bnSdmkX7998ozcLLb6Y6tttniNa5hh40Y52nrHDpDNTXSvb6LtPPVE/E6oaDGA== X-Received: by 2002:a50:d606:: with SMTP id x6mr13886546edi.124.1639560671500; Wed, 15 Dec 2021 01:31:11 -0800 (PST) Received: from mail-ed1-f53.google.com (mail-ed1-f53.google.com. [209.85.208.53]) by smtp.gmail.com with ESMTPSA id hd14sm473887ejc.195.2021.12.15.01.31.11 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 15 Dec 2021 01:31:11 -0800 (PST) Received: by mail-ed1-f53.google.com with SMTP id z9so17549932edb.5 for ; Wed, 15 Dec 2021 01:31:11 -0800 (PST) X-Received: by 2002:a17:907:60cd:: with SMTP id hv13mr9648868ejc.712.1639560670800; Wed, 15 Dec 2021 01:31:10 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Ivo Raisr Date: Wed, 15 Dec 2021 10:30:59 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: transition git-hooks on sourceware.org to Python 3.x To: Joel Brobecker Cc: Overseers mailing list , "Carlos O'Donell" , Corinna Vinschen , Joseph Myers , Mark Wielaard X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, HTML_MESSAGE, KAM_DMARC_STATUS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: overseers@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Overseers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Dec 2021 09:31:17 -0000 Dear, Thank you for the heads up. Indeed, I did set up the git-hooks some time ago for the Valgrind repository. But I don't remember the exact details; I assume it was a very simple setup. Is there any action required from my side at this point? Kind regards, Ivo Raisr Il giorno lun 13 dic 2021 alle ore 10:23 Mark Wielaard ha scritto: > Hi, > > I added Ivo to the CC who setup the valgrind hooks. If he isn't > available I'll take care of them. > > Cheers, > > Mark > > On Sun, Dec 12, 2021 at 03:32:47PM +0400, Joel Brobecker via Overseers > wrote: > > Hello, > > > > The current version of the git-hooks being used on sourceware > > is a version which uses Python 2.7. I have since then migrated > > to Python 3, where the minimum version is currently Python 3.8. > > That version has been running on AdaCore's server for a few weeks, > > now, so I think it's time to think about transitioning our repos > > on sourceware. > > > > The main improvement, besides the transition away from Python 2.x, > > comes from Python 3.x itself and the handling of strings vs bytes, > > so charsets are handled a bit better. > > > > The one small difficulty comes from making sure a recent version > > of Python 3.x gets used. Ideally, we should use the same Python > > I'm using on sourceware to run the git-hooks testsuite to make sure > > there aren't any compatibility issues. > > > > For this, what I'd like to propose is that we use a set of wrapper > > scripts which provide the same hooks as the git-hooks, and which > > only do the following: add the correct Python to the PATH, and > > then call the real hook script in git-hooks. > > > > I've created a small prototype and did some testing with it, > > it seems to work well. In a nutshell, we have: > > > > / > > - hooks/ <-- directory to use as hooks in Git bare repos > > - git-hooks/ <-- clone of > https://github.com/adacore/git-hooks > > - python3-for-git-hooks/ <-- Python 3.x venv to be used by the > git-hooks > > > > To transition a repository to updated hooks, we simply replace > > the hooks symbolic link to point to /hooks. > > > > My proptotype is at... > > > > /home/gdbadmin/brobecke/py3-git-hooks/git-hooks-wrapper > > > > ... for those who are curious and can log onto sourceware.org. > > There is a README.md file that gives a bit more information > > (I'll attach to this message as well) > > > > If agreed with the approach, I think we'll want to decide where > > we want this to live. Maybe /sourceware/projects/src-home/? > > The main issue with this location is that I worry about shared > > access and in particular maintenance over time. I'd like to be > > able to make changes there if there is a need to fix something. > > I'd like also to be able to continue updating the git-hooks version > > being used as I continue to push fixes and enhancements. > > > > I can take care of the git-hooks-wrapper setup once the location > > is settled. > > > > In terms of the projects themselves, I've identified the following > > repositories as using the git-hooks: > > > > - binutils-gdb.git/hooks > > - gcc.git/hooks > > - glibc.git/hooks > > - newlib-cygwin.git/hooks > > - valgrind.git/hooks > > > > This is based on searching for repositories on sourcware.org > > whose hooks dir is a symlink to the git-hooks. Not sure if there are > > others. > > > > I've Cc'ed people I know have asked me about the git-hooks for > > GCC, glibc and newlib. Not sure about valgrind. > > > > What we could do is give the binutils & GDB maintainers a quick > > heads up and then transition the binutils-gdb repository. If all > > goes well, I think we can transition the remaining ones. > > > > Thoughts? > > > > PS: If a project needs to stay on Python 2.7 for the time being, > > it is easy to accomodate it with the current scheme. But they > > will need to use a dedicated branch, as the latest no longer > > supports Python 2.x. > > -- > > Joel > > > Git-Hooks Wrapper > > ================= > > > > This repository provices a hooks/ directory which contain the same set of > > scripts as the AdaCore git-hooks (https://github.com/adacore/git-hooks). > > Those scripts simply act as wrappers allowing us to set the environment > > up prior to calling the real git-hooks hooks. In particular, our wrappers > > allow us to make sure we use a controlled version of Python to run the > hooks. > > > > Wrapper Setup Overview > > ---------------------- > > > > The wrappers expect the following structure: > > > > / > > - hooks/ <-- directory to use as hooks in Git bare > repos > > - git-hooks/ <-- clone of > https://github.com/adacore/git-hooks > > - python3-for-git-hooks/ <-- Python 3.x venv to be used by the > git-hooks > > > > The wrappers provides wrapper scripts which will do the following: > > > > - set the environment up to put the python3-for-git-hooks virtual env > > in its PATH; > > > > - Call the corresponding hook in git-hooks/hooks/. > > > > > > Git-Hooks Setup > > --------------- > > > > The wrappers expect a clone of the git-hooks to be located at > > the root of this repository, named git-hooks. > > > > > > Python Setup > > ------------ > > > > The wrappers expect a Python 3 Virtual Enviroment (venv) to be set up > > at the root of this repository, named python3-for-git-hooks. > > > > The venv should be set up as follow: > > > > # Create the venv, and activate it... > > > > $ python3 -m venv python3-for-git-hooks > > $ . ./python3-for-git-hooks/bin/activate > > > > # Make sure we use the latest version of pip > > > > $ python3 -m pip install pip --upgrade > > > > # Install all dependencies requried by the git-hooks. > > # > > # Note that the exact list of dependencies may change > > # over time. If unsure, run the git-hooks testsuite > > # as follow: > > # > > # $ cd path/to/git-hooks/testsuite > > # $ ./run-validation-tests > > # > > # The testsuite will abort with an error if any dependency > > # is missing. > > > > $ python3 -m pip install black e3-core flake8 pre-commit pytest > pytest-cov pytest-xdist > > > > Check the git-hooks' README.md file for the required minimum version > > of Python that must be used. > > > > Activating the Git-Hooks Wrapper > > -------------------------------- > > > > To use these wrappers in a Git repository, replace the hooks/ directory > > inside that Git repository by a symbolic link to the hooks/ directory > > inside the git-hooks-wrapper > >