From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 28631 invoked by alias); 25 Dec 2019 12:07:51 -0000 Mailing-List: contact gcc-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-owner@gcc.gnu.org Received: (qmail 28621 invoked by uid 89); 25 Dec 2019 12:07:50 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-1.3 required=5.0 tests=AWL,BAYES_00,SPF_HELO_PASS,SPF_PASS autolearn=ham version=3.3.1 spammy=Mark, D*thyrsus.com, esrthyrsuscom, esr@thyrsus.com X-HELO: snark.thyrsus.com Received: from thyrsus.com (HELO snark.thyrsus.com) (71.162.243.5) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 25 Dec 2019 12:07:48 +0000 Received: by snark.thyrsus.com (Postfix, from userid 1000) id 45EE94704B4B; Wed, 25 Dec 2019 07:07:47 -0500 (EST) Date: Wed, 25 Dec 2019 12:07:00 -0000 From: "Eric S. Raymond" To: Alexandre Oliva Cc: Jeff Law , Segher Boessenkool , Joseph Myers , Mark Wielaard , Maxim Kuvyrkov , "Richard Earnshaw (lists)" , gcc@gcc.gnu.org Subject: Re: Proposal for the transition timetable for the move to GIT Message-ID: <20191225120747.GA96669@thyrsus.com> Reply-To: esr@thyrsus.com References: <20191216133632.GC3152@gate.crashing.org> <20191216135451.GA3142@thyrsus.com> <20191216140514.GD3152@gate.crashing.org> <20191216153649.GE3152@gate.crashing.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-IsSubscribed: yes X-SW-Source: 2019-12/txt/msg00402.txt.bz2 Alexandre Oliva : > I know very little about reposurgeon, but I'm concerned that, should we > make the conversion with it, and later identify e.g. missed branches, we > might be unable to make such an incremental recovery. Can anyone > alleviate my concerns and let me know we could indeed make such an > incremental recovery of a branch missed in the initial conversion, in > such a way that its commit history would be shared with that of the > already-converted branch it branched from? Reposurgeon has a reparent command. If you have determined that a branch is detached or has an incorrect attachment point, patching the metadata of the root node to fix that is very easy. > Now, would it be too much of a burden to insist that the commit graphs > out of both conversions be isomorphic, and maybe mappings between the > commit ids (if they can't be made identical to begin with, that is) be > generated and shared, so that the results of both conversions can be > efficiently and mechanically compared (disregarding expected > differences) not only in terms of branch and tag names and commit > graphs, but also tree contents, commit messages and any other metadata? > Has anything like this been done yet? On the GCC repository, no. There are very serious practical problems with full verification of git against SVN stemming mainly from the fact that Subversion checkout on a respository of this size is extremely slow. IIRC Joseph at one point estimated a check time on the order of months due to that overhead alone. If you're talking about a commit-by-commit comparison between two conversions that assumes one or te other is correct, that is theoretically possible and - because git retrieval is much faster - could theoretically be done in a reasonable amount of time. But there is a lot of devil in the practical details. The reposurgeon suite once included a tool for such comparisons. Last year this happened: commit b8a609925ba70a6b68f9eda1d748eb667ad2fa59 Author: Eric S. Raymond Date: Fri Aug 24 12:40:46 2018 -0400 Retire repodiffer. Its only use case was checks against git-svn... ...which we now know to make such bad conversions that on larger than trivial repos the differ would be prohibitively noisy. Maxim's scripts probably make a better conversion than bare git-svn, because he uses git-svn only for linear basic blocks and thereby avoids its worst failure modes. In theory I could dust off repodiffer and apply it. That's in theory. In practice, on a repository this size I am not greatly optimistic about getting a result that could be interpreted by a Mark I brain. The reasons go beyond git-svn's brain damage to the same ontological-mismatch problems that make SVN-to-git conversion a headache in general. You might think at least there'd be a 1:1 correspondence between commits in the two conversions, but that's not going to be true for a couple of different reasons. 1. Split commits. Reposurgeon decomposes these into pieces one per git branch. I don't know what Maxim's scripts do. I think Joseph turned up that there are over a thousand of these in the GCC history. 2. There are three classes of commits in Subversion that don't really fit the git data model, (1) directory creation/deletion commits, (2) directory copy commits, (3) property changes with no associated blob. For each of these exceptional commits a converter to Git has a choice of dropping the commit, turning it into some sort of annotated tag, or leaving it in place as a zero-op commit (anomalous but not forbidden in the git model). It is pretty much guaranteed that different converters will make different choices about these, which will make for huge amounts of noise in your attempt at a diff. Checking for DAG isomorphism: again, theoretically possible, practically pretty daunting. It could be worse - general graph isomorphism is not even known to be polynomial-time - but in this case we can label corresponding commits with matching legacy IDs, which should make possible an isomorphism check in linear time with a trivial algorithm. Well, except for split commits. That one would be solvable, albeit painful. The real problem here would be mergeinfo links. It's not even obvious what "correct" mapping of mergeinfo links is, in general, due to the mismatch between Subversion's cherry-pick-based merge model and git's branch merging. Again, different converters will make different choices. Reconciling them would be not fun. There is another world of hurt lurking in "(disregarding expected differences)". How do you know what differences to expect? How are you going to specify them? What will interpret that spec? There is more months of work here - nasty, wearing toil, with no guarantee of a result with a decent signal-to-noise ratio. Even though I'm quite literally the best-qualified person on earth to do it, I flinch at the thought. -- Eric S. Raymond