contrib: New remotes structure for vendor and personal refs

contrib: New remotes structure for vendor and personal refs

[Richard Earnshaw (lists)]

[-- Attachment #1: Type: text/plain, Size: 2300 bytes --]

The initial structure for vendor and personal branches makes use of the 
default remote (normally origin) for the upstream repository). 
Unfortunately, this causes some confusion, especially for personal 
branches because a push will not push to the correct upstream location. 
This can be 'fixed' by adding a push refspec for the remote, but that 
has the unfortunate consequence of breaking the push.default behaviour 
for git push, and it becomes too easy to accidentally commit something 
unintended to the main parts of the repository.

To work around this, this patch changes the configuration to use 
separate 'remotes' for these additional refs, with one remote for the 
personal space and another remote for each vendor's space.  The personal 
space is called after the user's preferred branch-space prefix (default 
'me'), the vendor spaces are called vendor-<vendor-name>.

As far as possible, I've made the script automatically restructure any 
existing fetch or push lines that earlier versions of the scripts may 
have created - the gcc-git-customization.sh script will convert all 
vendor refs that it can find, so it is not necessary to re-add any 
vendors you've already added.

You might, however, want to run
   git remote prune <origin>
after running to clean up any stale upstream-refs that might still be in 
your local repo, and then
   git fetch vendor-<vendor>
or
   git fetch <me>
to re-populate the remotes/ structures.

Also, for any branch you already have that tracks a personal or vendor 
branch upstream, you might need to run
   git config branch.<name>.remote <new-remote>

so that merges and pushes go to the right place (I haven't attempted to 
automate this last part).

Please be aware that if you have multiple personal branches set up, then

   git push <me>

will still consider all of them for pushing.  If you only want to push 
one branch, then either write
   git push <me> HEAD
or
   git push <me> <me>/branch
as appropriate.

And don't forget '-n' (--dry-run) to see what would be done if this were 
not a dry run.

	* gcc-git-customization.sh: New personal and vendor layout.
	Convert any existing fetch/push rules to new layout.
	* git-fetch-vendor.sh: New vendor layout.

I'll wait a bit before pushing this, just in case someone spots an issue.

R.

[-- Attachment #2: custom3.patch --]
[-- Type: text/x-patch, Size: 5420 bytes --]

diff --git a/contrib/gcc-git-customization.sh b/contrib/gcc-git-customization.sh
index 26f4389bcc8..b62397314c6 100755
--- a/contrib/gcc-git-customization.sh
+++ b/contrib/gcc-git-customization.sh
@@ -81,6 +81,13 @@ then
     upstream="origin"
 fi
 ask "Local name for upstream repository" "origin" upstream
+
+v=$(git config --get-all "remote.${upstream}.fetch")
+if [ "x$v" = "x" ]
+then
+    echo "Remote $upstream does not seem to exist as a remote"
+    exit 1
+fi
 git config "gcc-config.upstream" "$upstream"
 
 remote_id=$(git config --get "gcc-config.user")
@@ -113,22 +120,38 @@ echo "(local branches starting <prefix>/ can be pushed directly to your"
 ask "personal area on the gcc server)" $old_pfx new_pfx
 git config "gcc-config.userpfx" "$new_pfx"
 
-echo "Setting up tracking for personal namespace $remote_id in remotes/$upstream/${new_pfx}"
-git config --replace-all "remote.${upstream}.fetch" "+refs/users/${remote_id}/heads/*:refs/remotes/${upstream}/${new_pfx}/*" ":refs/remotes/${upstream}/${old_pfx}/"
-git config --replace-all "remote.${upstream}.fetch" "+refs/users/${remote_id}/tags/*:refs/tags/${new_pfx}/*" ":refs/tags/${old_pfx}/"
+# Scan the existing settings to see if there are any we need to rewrite.
+vendors=$(git config --get-all "remote.${upstream}.fetch" "refs/vendors/" | sed -r "s:.*refs/vendors/([^/]+)/.*:\1:" | sort | uniq)
+url=$(git config --get "remote.${upstream}.url")
+pushurl=$(git config --get "remote.${upstream}.pushurl")
+for v in $vendors
+do
+    echo "Migrating vendor $v to new remote vendor-$v"
+    git config --unset-all "remote.${upstream}.fetch" "refs/vendors/$v/"
+    git config --unset-all "remote.${upstream}.push" "refs/vendors/$v/"
+    git config "remote.vendor-${v}.url" "${url}"
+    if [ "x$pushurl" != "x" ]
+    then
+	git config "remote.vendor-${v}.pushurl" "${pushurl}"
+    fi
+    git config --add "remote.vendor-${v}.fetch" "+refs/vendors/$v/heads/*:refs/remotes/vendor-${v}/*"
+    git config --add "remote.vendor-${v}.fetch" "+refs/vendors/$v/tags/*:refs/tags/vendor-${v}/*"
+done
+
+echo "Setting up tracking for personal namespace $remote_id in remotes/${new_pfx}"
+git config "remote.${new_pfx}.url" "${url}"
+if [ "x$pushurl" != "x" ]
+then
+    git config "remote.${new_pfx}.pushurl" "${pushurl}"
+fi
+git config --replace-all "remote.${new_pfx}.fetch" "+refs/users/${remote_id}/heads/*:refs/remotes/${new_pfx}/*" ":refs/remotes/${old_pfx}/"
+git config --replace-all "remote.${new_pfx}.fetch" "+refs/users/${remote_id}/tags/*:refs/tags/${new_pfx}/*" ":refs/tags/${old_pfx}/"
+git config --replace-all "remote.${new_pfx}.push" "refs/heads/${new_pfx}/*:refs/users/${remote_id}/heads/*" ":refs/users/${remote_id}"
 
-push_rule=$(git config --get "remote.${upstream}.push")
-if [ "x$push_rule" != "x" ]
+if [ "$old_pfx" != "$new_pfx" -a "$old_pfx" != "${upstream}" ]
 then
-    echo "***********************************************"
-    echo "                  Warning"
-    echo "***********************************************"
-    echo
-    echo "Old versions of this script used to add custom push"
-    echo "rules to simplify pushing to personal branches."
-    echo "Your configuration contains such rules, but we no-longer"
-    echo "recommend doing this."
-    echo
-    echo "To delete these rules run:"
-    echo "  git config --unset-all \"remote.${upstream}.push\""
+    git config --remove-section "remote.${old_pfx}"
 fi
+
+git config --unset-all "remote.${upstream}.fetch" "refs/users/${remote_id}/"
+git config --unset-all "remote.${upstream}.push" "refs/users/${remote_id}/"
diff --git a/contrib/git-fetch-vendor.sh b/contrib/git-fetch-vendor.sh
index d2d3ed56ad7..75b0f110927 100755
--- a/contrib/git-fetch-vendor.sh
+++ b/contrib/git-fetch-vendor.sh
@@ -1,20 +1,30 @@
 #!/bin/sh
 
+upstream=`git config --get "gcc-config.upstream"`
+if [ x"$upstream" = x ]
+then
+    echo "Config gcc-config.upstream not set, run contrib/gcc-git-customization"
+    exit 1
+fi
+
 if [ $# != 1 ]
 then
     echo "Usage: $0 <vendor>"
+    echo "The following vendors are already known:"
+    git ls-remote ${upstream} "*/vendors/*" | sed -r "s:.*/vendors/([^/]+)/.*:\1:"|sort|uniq
     exit 1
 fi
 
 vendor=$1
-upstream=`git config --get "gcc-config.upstream"`
-if [ x"$upstream" = x ]
+
+echo "setting up git to fetch vendor ${vendor} to remotes/vendor-${vendor}"
+url=$(git config --get "remote.${upstream}.url")
+pushurl=$(git config --get "remote.${upstream}.pushurl")
+git config "remote.vendor-${vendor}.url" "${url}"
+if [ "x$pushurl" != "x" ]
 then
-    echo "Config gcc-config.upstream not set, run contrib/gcc-git-customization"
-    exit 1
+    git config "remote.vendor-${vendor}.pushurl" "${pushurl}"
 fi
-
-echo "setting up git to fetch vendor ${vendor} to remotes/${upstream}/${vendor}"
-git config --replace-all "remote.${upstream}.fetch" "+refs/vendors/${vendor}/heads/*:refs/remotes/${upstream}/${vendor}/*" ":refs/remotes/${upstream}/${vendor}/"
-git config --replace-all "remote.${upstream}.fetch" "+refs/vendors/${vendor}/tags/*:refs/tags/${vendor}/*" ":refs/tags/${vendor}/"
-git fetch
+git config --replace-all "remote.vendor-${vendor}.fetch" "+refs/vendors/${vendor}/heads/*:refs/remotes/vendor-${vendor}/*" ":refs/remotes/vendor-${vendor}/"
+git config --replace-all "remote.vendor-${vendor}.fetch" "+refs/vendors/${vendor}/tags/*:refs/tags/${vendor}/*" ":refs/tags/${vendor}/"
+git fetch vendor-${vendor}

[v2] contrib: New remotes structure for vendor and personal refs

[Richard Earnshaw (lists)]

[-- Attachment #1: Type: text/plain, Size: 2767 bytes --]

The initial structure for vendor and personal branches makes use of the 
default remote (normally origin) for the upstream repository). 
Unfortunately, this causes some confusion, especially for personal 
branches because a push will not push to the correct upstream location. 
This can be 'fixed' by adding a push refspec for the remote, but that 
has the unfortunate consequence of breaking the push.default behaviour 
for git push, and it becomes too easy to accidentally commit something 
unintended to the main parts of the repository.

To work around this, this patch changes the configuration to use 
separate 'remotes' for these additional refs, with one remote for the 
personal space and another remote for each vendor's space.  The personal 
space is called after the user's preferred branch-space prefix (default 
'me'), the vendor spaces are called vendors/<vendor-name>.

As far as possible, I've made the script automatically restructure any 
existing fetch or push lines that earlier versions of the scripts may 
have created - the gcc-git-customization.sh script will convert all 
vendor refs that it can find, so it is not necessary to re-add any 
vendors you've already added.

You might, however, want to run
   git remote prune <origin>
after running to clean up any stale upstream-refs that might still be in 
your local repo, and then
   git fetch vendors/<vendor>
or
   git fetch <me>
to re-populate the remotes/ structures.

Also, for any branch you already have that tracks a personal or vendor 
branch upstream, you might need to run
   git config branch.<name>.remote <new-remote>

so that merges and pushes go to the right place (I haven't attempted to 
automate this last part).

For vendors, the new structure means that

   git checkout -b <vendor>/<branch> remotes/vendors/<vendor>/<branch>

will correctly set up a remote tracking branch.

Please be aware that if you have multiple personal branches set up, then

   git push <me>

will still consider all of them for pushing.  If you only want to push 
one branch, then either write
   git push <me> HEAD
or
   git push <me> <me>/branch
as appropriate.

And don't forget '-n' (--dry-run) to see what would be done if this were 
not a dry run.

Finally, now that the vendors spaces are isolated from each other and 
from the other spaces, I've added an option "--enable-push" to 
git-fetch-vendor.sh.  If passed, then a "push" spec will be added for 
that vendor to enable pushing to the upstream.  If you re-run the script 
for the same vendor without the option, the push spec will be removed.

	* gcc-git-customization.sh: New personal and vendor layout.
	Convert any existing fetch/push rules to new layout.
	* git-fetch-vendor.sh: New vendor layout.  Add --enable-push
	option.

[-- Attachment #2: custom3.patch --]
[-- Type: text/x-patch, Size: 6408 bytes --]

diff --git a/contrib/gcc-git-customization.sh b/contrib/gcc-git-customization.sh
index 26f4389bcc8..c16db747503 100755
--- a/contrib/gcc-git-customization.sh
+++ b/contrib/gcc-git-customization.sh
@@ -81,6 +81,13 @@ then
     upstream="origin"
 fi
 ask "Local name for upstream repository" "origin" upstream
+
+v=$(git config --get-all "remote.${upstream}.fetch")
+if [ "x$v" = "x" ]
+then
+    echo "Remote $upstream does not seem to exist as a remote"
+    exit 1
+fi
 git config "gcc-config.upstream" "$upstream"
 
 remote_id=$(git config --get "gcc-config.user")
@@ -100,6 +107,7 @@ then
 	fi
     fi
 fi
+
 ask "Account name on gcc.gnu.org (for your personal branches area)" $remote_id remote_id
 git config "gcc-config.user" "$remote_id"
 
@@ -108,27 +116,44 @@ if [ "x$old_pfx" = "x" ]
 then
     old_pfx="me"
 fi
+echo
 echo "Local branch prefix for personal branches you want to share"
 echo "(local branches starting <prefix>/ can be pushed directly to your"
 ask "personal area on the gcc server)" $old_pfx new_pfx
 git config "gcc-config.userpfx" "$new_pfx"
 
-echo "Setting up tracking for personal namespace $remote_id in remotes/$upstream/${new_pfx}"
-git config --replace-all "remote.${upstream}.fetch" "+refs/users/${remote_id}/heads/*:refs/remotes/${upstream}/${new_pfx}/*" ":refs/remotes/${upstream}/${old_pfx}/"
-git config --replace-all "remote.${upstream}.fetch" "+refs/users/${remote_id}/tags/*:refs/tags/${new_pfx}/*" ":refs/tags/${old_pfx}/"
+# Scan the existing settings to see if there are any we need to rewrite.
+vendors=$(git config --get-all "remote.${upstream}.fetch" "refs/vendors/" | sed -r "s:.*refs/vendors/([^/]+)/.*:\1:" | sort | uniq)
+url=$(git config --get "remote.${upstream}.url")
+pushurl=$(git config --get "remote.${upstream}.pushurl")
+for v in $vendors
+do
+    echo "Migrating vendor $v to new remote vendors/$v"
+    git config --unset-all "remote.${upstream}.fetch" "refs/vendors/$v/"
+    git config --unset-all "remote.${upstream}.push" "refs/vendors/$v/"
+    git config "remote.vendors/${v}.url" "${url}"
+    if [ "x$pushurl" != "x" ]
+    then
+	git config "remote.vendors/${v}.pushurl" "${pushurl}"
+    fi
+    git config --add "remote.vendors/${v}.fetch" "+refs/vendors/$v/heads/*:refs/remotes/vendors/${v}/*"
+    git config --add "remote.vendors/${v}.fetch" "+refs/vendors/$v/tags/*:refs/tags/vendors/${v}/*"
+done
 
-push_rule=$(git config --get "remote.${upstream}.push")
-if [ "x$push_rule" != "x" ]
+echo "Setting up tracking for personal namespace $remote_id in remotes/${new_pfx}"
+git config "remote.${new_pfx}.url" "${url}"
+if [ "x$pushurl" != "x" ]
 then
-    echo "***********************************************"
-    echo "                  Warning"
-    echo "***********************************************"
-    echo
-    echo "Old versions of this script used to add custom push"
-    echo "rules to simplify pushing to personal branches."
-    echo "Your configuration contains such rules, but we no-longer"
-    echo "recommend doing this."
-    echo
-    echo "To delete these rules run:"
-    echo "  git config --unset-all \"remote.${upstream}.push\""
+    git config "remote.${new_pfx}.pushurl" "${pushurl}"
 fi
+git config --replace-all "remote.${new_pfx}.fetch" "+refs/users/${remote_id}/heads/*:refs/remotes/${new_pfx}/*" ":refs/remotes/${old_pfx}/"
+git config --replace-all "remote.${new_pfx}.fetch" "+refs/users/${remote_id}/tags/*:refs/tags/${new_pfx}/*" ":refs/tags/${old_pfx}/"
+git config --replace-all "remote.${new_pfx}.push" "refs/heads/${new_pfx}/*:refs/users/${remote_id}/heads/*" ":refs/users/${remote_id}"
+
+if [ "$old_pfx" != "$new_pfx" -a "$old_pfx" != "${upstream}" ]
+then
+    git config --remove-section "remote.${old_pfx}"
+fi
+
+git config --unset-all "remote.${upstream}.fetch" "refs/users/${remote_id}/"
+git config --unset-all "remote.${upstream}.push" "refs/users/${remote_id}/"
diff --git a/contrib/git-fetch-vendor.sh b/contrib/git-fetch-vendor.sh
index d2d3ed56ad7..15303629b5c 100755
--- a/contrib/git-fetch-vendor.sh
+++ b/contrib/git-fetch-vendor.sh
@@ -1,12 +1,16 @@
 #!/bin/sh
 
-if [ $# != 1 ]
-then
-    echo "Usage: $0 <vendor>"
+usage ()
+{
+    echo "Usage: $0 [--enable-push] <vendor>"
+    echo "The following vendors are already known:"
+    git ls-remote ${upstream} "*/vendors/*" | sed -r "s:.*/vendors/([^/]+)/.*:\1:"|sort|uniq
     exit 1
-fi
+}
+
+# Should we insert a "push" refspec to enable pushing to the vendor branch?
+enable_push=no
 
-vendor=$1
 upstream=`git config --get "gcc-config.upstream"`
 if [ x"$upstream" = x ]
 then
@@ -14,7 +18,49 @@ then
     exit 1
 fi
 
-echo "setting up git to fetch vendor ${vendor} to remotes/${upstream}/${vendor}"
-git config --replace-all "remote.${upstream}.fetch" "+refs/vendors/${vendor}/heads/*:refs/remotes/${upstream}/${vendor}/*" ":refs/remotes/${upstream}/${vendor}/"
-git config --replace-all "remote.${upstream}.fetch" "+refs/vendors/${vendor}/tags/*:refs/tags/${vendor}/*" ":refs/tags/${vendor}/"
-git fetch
+case $# in
+    1)
+	# vendor names never start with -, so catch this in case user wrote something like --help.
+	case "$1" in
+	    -*)
+		usage
+		;;
+	    *)
+		vendor=$1
+		;;
+	esac
+	;;
+    2)
+	vendor=$2
+	if [ "$1" = "--enable-push" ]
+	then
+	    enable_push=yes
+	else
+	    usage
+	fi
+	;;
+    *)
+	usage
+	;;
+esac
+
+
+echo "setting up git to fetch vendor ${vendor} to remotes/vendors/${vendor}"
+url=$(git config --get "remote.${upstream}.url")
+pushurl=$(git config --get "remote.${upstream}.pushurl")
+git config "remote.vendors/${vendor}.url" "${url}"
+if [ "x$pushurl" != "x" ]
+then
+    git config "remote.vendors/${vendor}.pushurl" "${pushurl}"
+fi
+git config --replace-all "remote.vendors/${vendor}.fetch" "+refs/vendors/${vendor}/heads/*:refs/remotes/vendors/${vendor}/*" "refs/vendors/${vendor}/heads"
+git config --replace-all "remote.vendors/${vendor}.fetch" "+refs/vendors/${vendor}/tags/*:refs/tags/vendors/${vendor}/*" "refs/vendors/${vendor}/tags"
+if [ "$enable_push" = "yes" ]
+then
+    echo "Warning: take care when pushing that you only push the changes you intend."
+    echo "E.g. use \"git push vendors/${vendor} HEAD\" to push the current branch"
+    git config --replace-all "remote.vendors/${vendor}.push" "refs/heads/${vendor}/*:refs/vendors/${vendor}/heads/*"
+else
+    git config --unset-all "remote.vendors/${vendor}.push"
+fi
+git fetch vendors/${vendor}

Re: [v2] contrib: New remotes structure for vendor and personal refs

[Richard Earnshaw (lists)]

On 17/01/2020 11:21, Richard Earnshaw (lists) wrote:
> The initial structure for vendor and personal branches makes use of the 
> default remote (normally origin) for the upstream repository). 
> Unfortunately, this causes some confusion, especially for personal 
> branches because a push will not push to the correct upstream location. 
> This can be 'fixed' by adding a push refspec for the remote, but that 
> has the unfortunate consequence of breaking the push.default behaviour 
> for git push, and it becomes too easy to accidentally commit something 
> unintended to the main parts of the repository.
> 
> To work around this, this patch changes the configuration to use 
> separate 'remotes' for these additional refs, with one remote for the 
> personal space and another remote for each vendor's space.  The personal 
> space is called after the user's preferred branch-space prefix (default 
> 'me'), the vendor spaces are called vendors/<vendor-name>.
> 
> As far as possible, I've made the script automatically restructure any 
> existing fetch or push lines that earlier versions of the scripts may 
> have created - the gcc-git-customization.sh script will convert all 
> vendor refs that it can find, so it is not necessary to re-add any 
> vendors you've already added.
> 
> You might, however, want to run
>    git remote prune <origin>
> after running to clean up any stale upstream-refs that might still be in 
> your local repo, and then
>    git fetch vendors/<vendor>
> or
>    git fetch <me>
> to re-populate the remotes/ structures.
> 
> Also, for any branch you already have that tracks a personal or vendor 
> branch upstream, you might need to run
>    git config branch.<name>.remote <new-remote>
> 
> so that merges and pushes go to the right place (I haven't attempted to 
> automate this last part).
> 
> For vendors, the new structure means that
> 
>    git checkout -b <vendor>/<branch> remotes/vendors/<vendor>/<branch>
> 
> will correctly set up a remote tracking branch.
> 
> Please be aware that if you have multiple personal branches set up, then
> 
>    git push <me>
> 
> will still consider all of them for pushing.  If you only want to push 
> one branch, then either write
>    git push <me> HEAD
> or
>    git push <me> <me>/branch
> as appropriate.
> 
> And don't forget '-n' (--dry-run) to see what would be done if this were 
> not a dry run.
> 
> Finally, now that the vendors spaces are isolated from each other and 
> from the other spaces, I've added an option "--enable-push" to 
> git-fetch-vendor.sh.  If passed, then a "push" spec will be added for 
> that vendor to enable pushing to the upstream.  If you re-run the script 
> for the same vendor without the option, the push spec will be removed.
> 
>      * gcc-git-customization.sh: New personal and vendor layout.
>      Convert any existing fetch/push rules to new layout.
>      * git-fetch-vendor.sh: New vendor layout.  Add --enable-push
>      option.

Now pushed.

R.

Re: [v2] contrib: New remotes structure for vendor and personal refs

[Hans-Peter Nilsson]

> From: "Richard Earnshaw (lists)" <Richard.Earnshaw@arm.com>
> Date: Fri, 17 Jan 2020 12:21:07 +0100

> As far as possible, I've made the script automatically restructure any 
> existing fetch or push lines that earlier versions of the scripts may 
> have created - the gcc-git-customization.sh script will convert all 
> vendor refs that it can find, so it is not necessary to re-add any 
> vendors you've already added.

I fail, using these instructions, trying to create a
vendor-branch named axis/cris-decc0, using git-2.11.0 from
Debian 9.

> You might, however, want to run
>    git remote prune <origin>
> after running to clean up any stale upstream-refs that might still be in 
> your local repo, and then
>    git fetch vendors/<vendor>
> or
>    git fetch <me>
> to re-populate the remotes/ structures.

(I did not use gcc-git-customization.sh or git-fetch-vendor.sh before
XXXXXX, so there's presumably nothing to clean up.)

I've done
$ ./contrib/gcc-git-customization.sh
and
$ ./contrib/git-fetch-vendor.sh --enable-push axis

> Also, for any branch you already have that tracks a personal or vendor 
> branch upstream, you might need to run
>    git config branch.<name>.remote <new-remote>
> 
> so that merges and pushes go to the right place (I haven't attempted to 
> automate this last part).
> 
> For vendors, the new structure means that
> 
>    git checkout -b <vendor>/<branch> remotes/vendors/<vendor>/<branch>
> 
> will correctly set up a remote tracking branch.

On master, doing

$ git checkout -b axis/cris-decc0 remotes/vendors/axis/cris-decc0
fatal: Cannot update paths and switch to branch 'axis/cris-decc0' at the same time.
Did you intend to checkout 'remotes/vendors/axis/cris-decc0' which can not be resolved as commit?

My .git/config looks like this after the gcc-descr and
gcc-undescr lines:

[diff "md"]
	xfuncname = ^\\(define.*$
[gcc-config]
	upstream = origin
	user = hp
	userpfx = me
[remote "me"]
	url = git+ssh://gcc.gnu.org/git/gcc.git
	fetch = +refs/users/hp/heads/*:refs/remotes/me/*
	fetch = +refs/users/hp/tags/*:refs/tags/me/*
	push = refs/heads/me/*:refs/users/hp/heads/*
[remote "vendors/axis"]
	url = git+ssh://gcc.gnu.org/git/gcc.git
	fetch = +refs/vendors/axis/heads/*:refs/remotes/vendors/axis/*
	fetch = +refs/vendors/axis/tags/*:refs/tags/vendors/axis/*
	push = refs/heads/axis/*:refs/vendors/axis/heads/*

Bug in script (undiscovered because e.g. everybody else uses an
existing vendor or branch) or PEBKAC?

I'm past git 101, maybe even intermediate, for some definition
thereof, but this refs-configury is way beyond my
error-correction capabilities; I can't tell typos.

I'm about to create a devel/ branch instead, as that seems way
simpler than playing hide-and-seek like this, but that will make
everyone else fetch an additional blob that may be several
kilobytes (compressed).  Probably much larger than this email. :)

brgds, H-P

Re: [v2] contrib: New remotes structure for vendor and personal refs

[Hans-Peter Nilsson]

> From: Hans-Peter Nilsson <hp@axis.com>
> Date: Tue, 21 Jan 2020 02:47:57 +0100

> (I did not use gcc-git-customization.sh or git-fetch-vendor.sh before
> XXXXXX, so there's presumably nothing to clean up.)

Bah; "before 24b178184f260a6ec1516cfb8bb8876874a078a7".

brgds, H-P

Re: [v2] contrib: New remotes structure for vendor and personal refs

[Richard Earnshaw (lists)]

On 21/01/2020 01:47, Hans-Peter Nilsson wrote:
>> From: "Richard Earnshaw (lists)" <Richard.Earnshaw@arm.com>
>> Date: Fri, 17 Jan 2020 12:21:07 +0100
> 
>> As far as possible, I've made the script automatically restructure any
>> existing fetch or push lines that earlier versions of the scripts may
>> have created - the gcc-git-customization.sh script will convert all
>> vendor refs that it can find, so it is not necessary to re-add any
>> vendors you've already added.
> 
> I fail, using these instructions, trying to create a
> vendor-branch named axis/cris-decc0, using git-2.11.0 from
> Debian 9.
> 
>> You might, however, want to run
>>     git remote prune <origin>
>> after running to clean up any stale upstream-refs that might still be in
>> your local repo, and then
>>     git fetch vendors/<vendor>
>> or
>>     git fetch <me>
>> to re-populate the remotes/ structures.
> 
> (I did not use gcc-git-customization.sh or git-fetch-vendor.sh before
> XXXXXX, so there's presumably nothing to clean up.)
> 
> I've done
> $ ./contrib/gcc-git-customization.sh
> and
> $ ./contrib/git-fetch-vendor.sh --enable-push axis
> 
>> Also, for any branch you already have that tracks a personal or vendor
>> branch upstream, you might need to run
>>     git config branch.<name>.remote <new-remote>
>>
>> so that merges and pushes go to the right place (I haven't attempted to
>> automate this last part).
>>
>> For vendors, the new structure means that
>>
>>     git checkout -b <vendor>/<branch> remotes/vendors/<vendor>/<branch>
>>
>> will correctly set up a remote tracking branch.
> 
> On master, doing
> 
> $ git checkout -b axis/cris-decc0 remotes/vendors/axis/cris-decc0
> fatal: Cannot update paths and switch to branch 'axis/cris-decc0' at the same time.
> Did you intend to checkout 'remotes/vendors/axis/cris-decc0' which can not be resolved as commit?
> 

It's nothing to do with setting up your vendors space, but a consequence 
that you can't track a remote branch that hasn't been created yet; 
you'll see the same thing even for personal branches or for other new 
vendor branches.  I'll need to document that.

Initially, you'll need to create the upstream branch, something like 
(but untested).

# Set up axis vendor area
contrib/git-fetch-vendor.sh --enable-push axis
# Using master, or some other branch you want to base this on.
git checkout -b axis/cris-decc0 master
# create upstream
git push vendors/axis axis/cris-decc0:refs/vendors/axis/heads/cris-decc0

(In theory, you should be able to add '-u' to the above, but 
unfortunately, that doesn't work.  So, after the above)

# re-fetch the new branch
git fetch vendors/asis
git branch --set-upstream remotes/vendors/axis/cris-decc0

Now things should push and pull properly.

> My .git/config looks like this after the gcc-descr and
> gcc-undescr lines:
> 
> [diff "md"]
> 	xfuncname = ^\\(define.*$
> [gcc-config]
> 	upstream = origin
> 	user = hp
> 	userpfx = me
> [remote "me"]
> 	url = git+ssh://gcc.gnu.org/git/gcc.git
> 	fetch = +refs/users/hp/heads/*:refs/remotes/me/*
> 	fetch = +refs/users/hp/tags/*:refs/tags/me/*
> 	push = refs/heads/me/*:refs/users/hp/heads/*
> [remote "vendors/axis"]
> 	url = git+ssh://gcc.gnu.org/git/gcc.git
> 	fetch = +refs/vendors/axis/heads/*:refs/remotes/vendors/axis/*
> 	fetch = +refs/vendors/axis/tags/*:refs/tags/vendors/axis/*
> 	push = refs/heads/axis/*:refs/vendors/axis/heads/*
> 
> Bug in script (undiscovered because e.g. everybody else uses an
> existing vendor or branch) or PEBKAC?
> 
> I'm past git 101, maybe even intermediate, for some definition
> thereof, but this refs-configury is way beyond my
> error-correction capabilities; I can't tell typos.
> 
> I'm about to create a devel/ branch instead, as that seems way
> simpler than playing hide-and-seek like this, but that will make
> everyone else fetch an additional blob that may be several
> kilobytes (compressed).  Probably much larger than this email. :)
> 
> brgds, H-P
> 

Hope that helps,

R.

Re: [v2] contrib: New remotes structure for vendor and personal refs

[Richard Earnshaw (lists)]

On 21/01/2020 10:26, Richard Earnshaw (lists) wrote:
> On 21/01/2020 01:47, Hans-Peter Nilsson wrote:
>>> From: "Richard Earnshaw (lists)" <Richard.Earnshaw@arm.com>
>>> Date: Fri, 17 Jan 2020 12:21:07 +0100
>>
>>> As far as possible, I've made the script automatically restructure any
>>> existing fetch or push lines that earlier versions of the scripts may
>>> have created - the gcc-git-customization.sh script will convert all
>>> vendor refs that it can find, so it is not necessary to re-add any
>>> vendors you've already added.
>>
>> I fail, using these instructions, trying to create a
>> vendor-branch named axis/cris-decc0, using git-2.11.0 from
>> Debian 9.
>>
>>> You might, however, want to run
>>>     git remote prune <origin>
>>> after running to clean up any stale upstream-refs that might still be in
>>> your local repo, and then
>>>     git fetch vendors/<vendor>
>>> or
>>>     git fetch <me>
>>> to re-populate the remotes/ structures.
>>
>> (I did not use gcc-git-customization.sh or git-fetch-vendor.sh before
>> XXXXXX, so there's presumably nothing to clean up.)
>>
>> I've done
>> $ ./contrib/gcc-git-customization.sh
>> and
>> $ ./contrib/git-fetch-vendor.sh --enable-push axis
>>
>>> Also, for any branch you already have that tracks a personal or vendor
>>> branch upstream, you might need to run
>>>     git config branch.<name>.remote <new-remote>
>>>
>>> so that merges and pushes go to the right place (I haven't attempted to
>>> automate this last part).
>>>
>>> For vendors, the new structure means that
>>>
>>>     git checkout -b <vendor>/<branch> remotes/vendors/<vendor>/<branch>
>>>
>>> will correctly set up a remote tracking branch.
>>
>> On master, doing
>>
>> $ git checkout -b axis/cris-decc0 remotes/vendors/axis/cris-decc0
>> fatal: Cannot update paths and switch to branch 'axis/cris-decc0' at 
>> the same time.
>> Did you intend to checkout 'remotes/vendors/axis/cris-decc0' which can 
>> not be resolved as commit?
>>
> 
> It's nothing to do with setting up your vendors space, but a consequence 
> that you can't track a remote branch that hasn't been created yet; 
> you'll see the same thing even for personal branches or for other new 
> vendor branches.  I'll need to document that.
> 
> Initially, you'll need to create the upstream branch, something like 
> (but untested).
> 
> # Set up axis vendor area
> contrib/git-fetch-vendor.sh --enable-push axis
> # Using master, or some other branch you want to base this on.
> git checkout -b axis/cris-decc0 master
> # create upstream
> git push vendors/axis axis/cris-decc0:refs/vendors/axis/heads/cris-decc0
> 
> (In theory, you should be able to add '-u' to the above, but 
> unfortunately, that doesn't work.  So, after the above)
> 
> # re-fetch the new branch
> git fetch vendors/asis
> git branch --set-upstream remotes/vendors/axis/cris-decc0

--set-upstream-to of course (I said it was untested:)

R.

> 
> Now things should push and pull properly.
> 
>> My .git/config looks like this after the gcc-descr and
>> gcc-undescr lines:
>>
>> [diff "md"]
>>     xfuncname = ^\\(define.*$
>> [gcc-config]
>>     upstream = origin
>>     user = hp
>>     userpfx = me
>> [remote "me"]
>>     url = git+ssh://gcc.gnu.org/git/gcc.git
>>     fetch = +refs/users/hp/heads/*:refs/remotes/me/*
>>     fetch = +refs/users/hp/tags/*:refs/tags/me/*
>>     push = refs/heads/me/*:refs/users/hp/heads/*
>> [remote "vendors/axis"]
>>     url = git+ssh://gcc.gnu.org/git/gcc.git
>>     fetch = +refs/vendors/axis/heads/*:refs/remotes/vendors/axis/*
>>     fetch = +refs/vendors/axis/tags/*:refs/tags/vendors/axis/*
>>     push = refs/heads/axis/*:refs/vendors/axis/heads/*
>>
>> Bug in script (undiscovered because e.g. everybody else uses an
>> existing vendor or branch) or PEBKAC?
>>
>> I'm past git 101, maybe even intermediate, for some definition
>> thereof, but this refs-configury is way beyond my
>> error-correction capabilities; I can't tell typos.
>>
>> I'm about to create a devel/ branch instead, as that seems way
>> simpler than playing hide-and-seek like this, but that will make
>> everyone else fetch an additional blob that may be several
>> kilobytes (compressed).  Probably much larger than this email. :)
>>
>> brgds, H-P
>>
> 
> Hope that helps,
> 
> R.

[patch] contrib: script to create a new vendor branch

[Richard Earnshaw (lists)]

[-- Attachment #1: Type: text/plain, Size: 613 bytes --]

This script is intended to create a new vendor branch.  Doing so is not 
completely obvious if
you are not familiar with the upstream structure, so this takes the pain 
out of getting it right.

It doesn't check out the branch locally, but does set everything up so 
that, if you have push enabled for your vendor branches, then

   git push vendors/<vendor> <branch>

will work as expected.

Run the script as

contrib/git-add-vendor-branch.sh <vendor>/<branch> <start-point>

the <vendor> space must have previously been set up in the way 
git-fetch-vendor.sh expects.

	* git-add-vendor-bransh.sh: New file.



[-- Attachment #2: vend-br.patch --]
[-- Type: text/x-patch, Size: 1282 bytes --]

diff --git a/contrib/git-add-vendor-branch.sh b/contrib/git-add-vendor-branch.sh
new file mode 100755
index 00000000000..04d39ed6b63
--- /dev/null
+++ b/contrib/git-add-vendor-branch.sh
@@ -0,0 +1,43 @@
+#! /bin/sh -xve
+
+# Create a new upstream vendor branch.
+
+# Usage:
+#  contrib/git-add-vendor-branch.sh <vendor>/<branch-name> <base>
+
+usage ()
+{
+    echo "Usage:"
+    echo "  $0 <vendor>/<branch-name> <start-point>"
+    echo
+    echo "<vendor> must have already been set up using contrib/git-fetch-vendor.sh"
+    exit 1
+}
+
+if [ $# != 2 ]
+then
+    usage
+fi
+
+vendor=$(echo "$1" | sed -r "s:([^/]*)/.*$:\1:")
+branch=$(echo "$1" | sed -r "s:[^/]*/(.*)$:\1:")
+start=$2
+
+if [ -z "$vendor" -o -z "$branch" ]
+then
+    usage
+fi
+
+# Check that we know about the vendor
+url=$(git config --get "remote.vendors/${vendor}.url"||true)
+if [ -z "$url" ]
+then
+    echo "Cannot locate remote data for vendor ${vendor}.  Have you set it up?"
+    exit 1
+fi
+
+git branch --no-track ${vendor}/${branch} ${start}
+git push vendors/${vendor} ${vendor}/${branch}:refs/vendors/${vendor}/heads/${branch}
+git fetch -q vendors/${vendor}
+git branch --set-upstream-to=remotes/vendors/${vendor}/${branch} ${vendor}/$branch
+echo "Now ready to check out ${vendor}/${branch}"

Re: [patch] contrib: script to create a new vendor branch

[Richard Earnshaw (lists)]

On 21/01/2020 13:29, Richard Earnshaw (lists) wrote:
> This script is intended to create a new vendor branch.  Doing so is not 
> completely obvious if
> you are not familiar with the upstream structure, so this takes the pain 
> out of getting it right.
> 
> It doesn't check out the branch locally, but does set everything up so 
> that, if you have push enabled for your vendor branches, then
> 
>    git push vendors/<vendor> <branch>
> 

Correction, the branch should be named <vendor>/<branch>, so the push 
should be

git push vendors/<vendor> <vendor>/<branch>

For example, for the ARM vendor, the push would be

git push vendors/ARM ARM/<branch>

R.

> will work as expected.
> 
> Run the script as
> 
> contrib/git-add-vendor-branch.sh <vendor>/<branch> <start-point>
> 
> the <vendor> space must have previously been set up in the way 
> git-fetch-vendor.sh expects.
> 
>      * git-add-vendor-bransh.sh: New file.
> 
> 

Re: [patch] contrib: script to create a new vendor branch

[Hans-Peter Nilsson]

> From: "Richard Earnshaw (lists)" <Richard.Earnshaw@arm.com>
> Date: Tue, 21 Jan 2020 14:36:32 +0100

> Correction, the branch should be named <vendor>/<branch>, so the push 
> should be
> 
> git push vendors/<vendor> <vendor>/<branch>
> 
> For example, for the ARM vendor, the push would be
> 
> git push vendors/ARM ARM/<branch>
> 
> R.
> 
> > will work as expected.
> > 
> > Run the script as
> > 
> > contrib/git-add-vendor-branch.sh <vendor>/<branch> <start-point>
> > 
> > the <vendor> space must have previously been set up in the way 
> > git-fetch-vendor.sh expects.
> > 
> >      * git-add-vendor-bransh.sh: New file.

(typo "bransh")

Thanks, this and your previous reply certainly helped!

Any chance of a git-add-user-branch.sh too, while you're at it?
Or maybe it's just the corresponding push command in there,
that's needed.

brgds, H-P

Re: [patch] contrib: script to create a new vendor branch

[Richard Earnshaw (lists)]

On 21/01/2020 23:23, Hans-Peter Nilsson wrote:
>> From: "Richard Earnshaw (lists)" <Richard.Earnshaw@arm.com>
>> Date: Tue, 21 Jan 2020 14:36:32 +0100
> 
>> Correction, the branch should be named <vendor>/<branch>, so the push
>> should be
>>
>> git push vendors/<vendor> <vendor>/<branch>
>>
>> For example, for the ARM vendor, the push would be
>>
>> git push vendors/ARM ARM/<branch>
>>
>> R.
>>
>>> will work as expected.
>>>
>>> Run the script as
>>>
>>> contrib/git-add-vendor-branch.sh <vendor>/<branch> <start-point>
>>>
>>> the <vendor> space must have previously been set up in the way
>>> git-fetch-vendor.sh expects.
>>>
>>>       * git-add-vendor-bransh.sh: New file.
> 
> (typo "bransh")
> 

Duh! will fix before push.

> Thanks, this and your previous reply certainly helped!
> 

Yes, I saw from the gcc-cvs list that you were merrily pushing now.

> Any chance of a git-add-user-branch.sh too, while you're at it?
> Or maybe it's just the corresponding push command in there,
> that's needed.
> 

On the todo list... unless someone beats me to it.  Mean-time my 
gitwrite.html patch describes how to set up a user branch.

R.

> brgds, H-P
> 

Re: [patch] contrib: script to create a new vendor branch

[Richard Earnshaw (lists)]

On 22/01/2020 10:04, Richard Earnshaw (lists) wrote:
> On 21/01/2020 23:23, Hans-Peter Nilsson wrote:
>>> From: "Richard Earnshaw (lists)" <Richard.Earnshaw@arm.com>
>>> Date: Tue, 21 Jan 2020 14:36:32 +0100
>>
>>> Correction, the branch should be named <vendor>/<branch>, so the push
>>> should be
>>>
>>> git push vendors/<vendor> <vendor>/<branch>
>>>
>>> For example, for the ARM vendor, the push would be
>>>
>>> git push vendors/ARM ARM/<branch>
>>>
>>> R.
>>>
>>>> will work as expected.
>>>>
>>>> Run the script as
>>>>
>>>> contrib/git-add-vendor-branch.sh <vendor>/<branch> <start-point>
>>>>
>>>> the <vendor> space must have previously been set up in the way
>>>> git-fetch-vendor.sh expects.
>>>>
>>>>       * git-add-vendor-bransh.sh: New file.
>>
>> (typo "bransh")
>>
> 
> Duh! will fix before push.
> 
>> Thanks, this and your previous reply certainly helped!
>>
> 
> Yes, I saw from the gcc-cvs list that you were merrily pushing now.
> 
>> Any chance of a git-add-user-branch.sh too, while you're at it?
>> Or maybe it's just the corresponding push command in there,
>> that's needed.
>>
> 
> On the todo list... unless someone beats me to it.  Mean-time my 
> gitwrite.html patch describes how to set up a user branch.
> 
> R.
> 
>> brgds, H-P
>>
> 


Now pushed.

R.