Re: Commit messages and the move to git

["Richard Earnshaw (lists)" <Richard.Earnshaw@arm.com>]

On 19/11/2019 19:47, Richard Earnshaw (lists) wrote:
> On 19/11/2019 19:32, Eric S. Raymond wrote:
>> Richard Earnshaw (lists) <Richard.Earnshaw@arm.com>:
>>> I was looking at the reposurgeon code last night, and I think I can see what
>>> the problem *might* be, but I haven't had time to produce a testcase.
>>>
>>> Some of our commits have mergeinfo that looks a bit like this:
>>>
>>> 202022-202023,202026,202028-202029,202036,202039-202041,202043-202044,202048-202049,202051-202056,202058-202061,202064-202065,202068-202071,202077,202079-202082,202084,202086-202088,202092-202104,202106-202113,202115-202119,202121,202124-202134,202139,202142-202146,202148-202150,202153-202154,202158-202159,202163-202165,202168,202172,202174,202179-202180,202184-202192,202195,202197,202202-202208,202225-202230,202232-202233,202237-202239,202242,202244-202245,202247,202250-202251,202258-202264,202266,202269,202271-202275,202279,202281-202282,202284,202286,202289-202292,202296-202299,202301-202302,202305,202309,202311-202323,202327-202335,202337,202339,202343-202346,202350,202352,202356-202357,202359-202360,202363-202371,202373-202374,202377,202379-202382,202384,202389,202391-202395,202398-202407,202409,202411,202416-202418,202421
>>>
>>> which is a massive long list with a number of holes in it.
>>>
>>> But I suspect the holes are really commits to other branches and that in the
>>> above describes a linear chain along one branch.  If so, rather than
>>> producing links to each subgroup (and perhaps dropping single non-list
>>> elements, the description can be mapped back to a contiguous sequence of
>>> commits down a branch and thus should really resolve to a single child being
>>> used for the merge source.  At present, I think for the above we're seeing a
>>> child reference created for each subrange in that list.
>>
>> I have no doubt you are correct. Detecting such interrupted ranges ia
>> foing to be...  interesting.
>>
>>> Incidentally, the mergeinfo pass on the gcc repo is currently taking about 8
>>> hours on my machine, that's 80-90% of the entire conversion time.  But it
>>> might be related to the above.
>>
>> You must be running the old Python code, there was on O(n**2) in that
>> phase that has since been fixed. Try the Go code from
>> https://gitlab.com/esr/reposurgeon; it is *much* faster.
>>
> 
> Nope, that was from running the go version from yesterday.  This one, to
> be precise:  1ab3c514c6cd5e1a5d6b68a8224df299751ca637
> 
> This pass used to be very fast a couple of weeks back, but something
> went in recently that's caused a major slowdown.
> 
> Oh, and I've been having problems with the ChangeLogs command as well.
> It used to run fine on my machine (128G), but now it's started blowing
> memory and taking my X server down.
> 
> R.
> 
> R.
> 


Here's the stats output:

# Statistics on read and processing times
timing
        commits:        276738  (from 278380)
        parsing:        2.85%   14m22.861991058s
       cleaning:        0.32%   1m37.653100823s
       filemaps:        0.37%   1m52.851558995s
        commits:        4.40%   22m15.380157228s
     rootcommit:        0.00%   8.779µs
       branches:        0.04%   12.710113776s
        parents:        0.00%   121.73484ms
           root:        0.00%   267.997µs
    branchlinks:        0.00%   10.58361ms
      mergeinfo:        91.67%  7h43m15.416510183s
       branches:        0.00%   11.616µs
         dejunk:        0.04%   10.672889443s
      polishing:        0.04%   11.249533399s
      tagifying:        0.03%   10.528735532s
    tagcleaning:        0.03%   9.880052536s
     debubbling:        0.00%   1.384357053s
    renumbering:        0.20%   59.718288526s
          total:        9/sec   8h25m20.439895394s