git conversion in progress

git conversion in progress

[Joseph Myers]

The GCC SVN repository is now read-only for the move to git, as is the old 
git-svn mirror; the cron job updating that mirror has been disabled, as 
have gccadmin's cron jobs updating DATESTAMP, generating snapshots and 
updating online documentation, until those are updated to work from git.

Assuming validation passes, I expect to make the converted repository 
available in place of the git-svn mirror (git://gcc.gnu.org/git/gcc.git 
and git+ssh://gcc.gnu.org/git/gcc.git) some time on Saturday.  That will 
be initially read-only to allow for people to review it in case any 
significant conversion issues are noticed late that would justify 
rerunning the conversion.  I propose to make it writable on Monday, at 
which point we'll be in development stage 4 for GCC 10 (and to leave 
gcc-reposurgeon-8.git around for a while in case people wish to use that, 
rather than the main repository, for any further testing, but to delete 
the older test conversions 1 through 7).  Some cron jobs may be re-enabled 
before then, subject to testing (I have git changes to gcc_release ready, 
for example, for testing snapshot generation), but the DATESTAMP updates 
won't be enabled until the repository is writable.

I encourage people to continue to work on improving the documentation for 
using git with GCC 
(<https://gcc.gnu.org/ml/gcc-patches/2020-01/msg00623.html> and 
<https://gcc.gnu.org/ml/gcc-patches/2020-01/msg00625.html> list some of 
the things that it seems it might be useful to document).  Once the 
conversion is done we'll need to update various places referring to SVN on 
the website and in the manuals.

The old git-svn mirror will remain available read-only as 
git://gcc.gnu.org/git/gcc-old.git (which is already available), in 
addition to the refs from that mirror being in the new repository as 
refs/git-old/heads/*, refs/git-old/tags/* and refs/git-svn-old/heads/* 
(thus, gitweb links to commits in that repository will continue to work, 
as long as those commits were reachable as of when git mirror went 
read-only rather than having been on an old version of a branch that was 
since deleted or rebased).

Where people have active branches in that mirror, I hope they will rebase 
those branches onto the new version of the history, putting the results in 
refs/heads/devel/* or refs/users/*/heads/* according to whether they are 
considered generally shared development branches (can't have 
non-fast-forward pushes, should be documented on the GCC website) or user 
branches (can have non-fast-forward pushes, need not be documented).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: git conversion in progress

[Richard Earnshaw]

On 11/01/2020 01:18, Joseph Myers wrote:
> The GCC SVN repository is now read-only for the move to git, as is the old 
> git-svn mirror; the cron job updating that mirror has been disabled, as 
> have gccadmin's cron jobs updating DATESTAMP, generating snapshots and 
> updating online documentation, until those are updated to work from git.
> 
> Assuming validation passes, I expect to make the converted repository 
> available in place of the git-svn mirror (git://gcc.gnu.org/git/gcc.git 
> and git+ssh://gcc.gnu.org/git/gcc.git) some time on Saturday.  That will 
> be initially read-only to allow for people to review it in case any 
> significant conversion issues are noticed late that would justify 
> rerunning the conversion.  I propose to make it writable on Monday, at 
> which point we'll be in development stage 4 for GCC 10 (and to leave 
> gcc-reposurgeon-8.git around for a while in case people wish to use that, 
> rather than the main repository, for any further testing, but to delete 
> the older test conversions 1 through 7).  Some cron jobs may be re-enabled 
> before then, subject to testing (I have git changes to gcc_release ready, 
> for example, for testing snapshot generation), but the DATESTAMP updates 
> won't be enabled until the repository is writable.
> 
> I encourage people to continue to work on improving the documentation for 
> using git with GCC 
> (<https://gcc.gnu.org/ml/gcc-patches/2020-01/msg00623.html> and 
> <https://gcc.gnu.org/ml/gcc-patches/2020-01/msg00625.html> list some of 
> the things that it seems it might be useful to document).  Once the 
> conversion is done we'll need to update various places referring to SVN on 
> the website and in the manuals.
> 
> The old git-svn mirror will remain available read-only as 
> git://gcc.gnu.org/git/gcc-old.git (which is already available), in 
> addition to the refs from that mirror being in the new repository as 
> refs/git-old/heads/*, refs/git-old/tags/* and refs/git-svn-old/heads/* 
> (thus, gitweb links to commits in that repository will continue to work, 
> as long as those commits were reachable as of when git mirror went 
> read-only rather than having been on an old version of a branch that was 
> since deleted or rebased).
> 
> Where people have active branches in that mirror, I hope they will rebase 
> those branches onto the new version of the history, putting the results in 
> refs/heads/devel/* or refs/users/*/heads/* according to whether they are 
> considered generally shared development branches (can't have 
> non-fast-forward pushes, should be documented on the GCC website) or user 
> branches (can have non-fast-forward pushes, need not be documented).
> 

Joseph is unlikely to upload his conversion to gcc.gnu.org until all the
tests have been completed.  However, I've done a parallel conversion
using the same scripts and the same version of reposurgeon.

For those that are desperate for an early look, I've uploaded a copy of
this to gitlab: https://gitlab.com/rearnsha/gcc.

Beware that testing is *still* in progress, but hopefully this will be
identical to the version that Joseph finally uploads.

Please let us know, ASAP, if you see any *critical* issues.

R.

Re: git conversion in progress

[Joseph Myers]

On Sat, 11 Jan 2020, Richard Earnshaw wrote:

> On 11/01/2020 01:18, Joseph Myers wrote:
> > The GCC SVN repository is now read-only for the move to git, as is the old 
> > git-svn mirror; the cron job updating that mirror has been disabled, as 
> > have gccadmin's cron jobs updating DATESTAMP, generating snapshots and 
> > updating online documentation, until those are updated to work from git.
> > 
> > Assuming validation passes, I expect to make the converted repository 
> > available in place of the git-svn mirror (git://gcc.gnu.org/git/gcc.git 
> > and git+ssh://gcc.gnu.org/git/gcc.git) some time on Saturday.  That will 
> > be initially read-only to allow for people to review it in case any 
> > significant conversion issues are noticed late that would justify 
> > rerunning the conversion.  I propose to make it writable on Monday, at 
> > which point we'll be in development stage 4 for GCC 10 (and to leave 

This conversion is now in place, read-only for checking purposes.  I've 
done all the usual validation, including in particular checking branch 
tips and tags against SVN.

> For those that are desperate for an early look, I've uploaded a copy of
> this to gitlab: https://gitlab.com/rearnsha/gcc.
> 
> Beware that testing is *still* in progress, but hopefully this will be
> identical to the version that Joseph finally uploads.

Richard's conversion was using older Bugzilla data (I updated the data 
from Bugzilla right before running my final conversion, Richard didn't), 
so they don't have identical commit summaries everywhere.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: git conversion in progress

[Jakub Jelinek]

On Sat, Jan 11, 2020 at 02:39:43PM +0000, Joseph Myers wrote:
> > On 11/01/2020 01:18, Joseph Myers wrote:
> > > The GCC SVN repository is now read-only for the move to git, as is the old 
> > > git-svn mirror; the cron job updating that mirror has been disabled, as 
> > > have gccadmin's cron jobs updating DATESTAMP, generating snapshots and 
> > > updating online documentation, until those are updated to work from git.
> > > 
> > > Assuming validation passes, I expect to make the converted repository 
> > > available in place of the git-svn mirror (git://gcc.gnu.org/git/gcc.git 
> > > and git+ssh://gcc.gnu.org/git/gcc.git) some time on Saturday.  That will 
> > > be initially read-only to allow for people to review it in case any 
> > > significant conversion issues are noticed late that would justify 
> > > rerunning the conversion.  I propose to make it writable on Monday, at 
> > > which point we'll be in development stage 4 for GCC 10 (and to leave 
> 
> This conversion is now in place, read-only for checking purposes.  I've 
> done all the usual validation, including in particular checking branch 
> tips and tags against SVN.

Thanks.
Is there or will there be any way using gitweb to browse the vendor and user
branches (and also even query which vendors/users and their branches there
are)?
I've tried
https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/vendors/
https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/vendors/redhat/
https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/vendors/redhat/gcc-9
https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/vendors/redhat/gcc-9-branch/
and none of that worked.  They don't show up in the heads list
https://gcc.gnu.org/git/?p=gcc.git;a=heads

	Jakub

Re: git conversion in progress

[Richard Earnshaw]

On 11/01/2020 14:48, Jakub Jelinek wrote:
> On Sat, Jan 11, 2020 at 02:39:43PM +0000, Joseph Myers wrote:
>>> On 11/01/2020 01:18, Joseph Myers wrote:
>>>> The GCC SVN repository is now read-only for the move to git, as is the old 
>>>> git-svn mirror; the cron job updating that mirror has been disabled, as 
>>>> have gccadmin's cron jobs updating DATESTAMP, generating snapshots and 
>>>> updating online documentation, until those are updated to work from git.
>>>>
>>>> Assuming validation passes, I expect to make the converted repository 
>>>> available in place of the git-svn mirror (git://gcc.gnu.org/git/gcc.git 
>>>> and git+ssh://gcc.gnu.org/git/gcc.git) some time on Saturday.  That will 
>>>> be initially read-only to allow for people to review it in case any 
>>>> significant conversion issues are noticed late that would justify 
>>>> rerunning the conversion.  I propose to make it writable on Monday, at 
>>>> which point we'll be in development stage 4 for GCC 10 (and to leave 
>>
>> This conversion is now in place, read-only for checking purposes.  I've 
>> done all the usual validation, including in particular checking branch 
>> tips and tags against SVN.
> 
> Thanks.
> Is there or will there be any way using gitweb to browse the vendor and user
> branches (and also even query which vendors/users and their branches there
> are)?
> I've tried
> https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/vendors/
> https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/vendors/redhat/
> https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/vendors/redhat/gcc-9
> https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/vendors/redhat/gcc-9-branch/
> and none of that worked.  They don't show up in the heads list
> https://gcc.gnu.org/git/?p=gcc.git;a=heads
> 
> 	Jakub
> 

Try refs/vendors/redhat/heads

R.

Re: git conversion in progress

[Richard Earnshaw]

On 11/01/2020 14:49, Richard Earnshaw wrote:
> On 11/01/2020 14:48, Jakub Jelinek wrote:
>> On Sat, Jan 11, 2020 at 02:39:43PM +0000, Joseph Myers wrote:
>>>> On 11/01/2020 01:18, Joseph Myers wrote:
>>>>> The GCC SVN repository is now read-only for the move to git, as is the old 
>>>>> git-svn mirror; the cron job updating that mirror has been disabled, as 
>>>>> have gccadmin's cron jobs updating DATESTAMP, generating snapshots and 
>>>>> updating online documentation, until those are updated to work from git.
>>>>>
>>>>> Assuming validation passes, I expect to make the converted repository 
>>>>> available in place of the git-svn mirror (git://gcc.gnu.org/git/gcc.git 
>>>>> and git+ssh://gcc.gnu.org/git/gcc.git) some time on Saturday.  That will 
>>>>> be initially read-only to allow for people to review it in case any 
>>>>> significant conversion issues are noticed late that would justify 
>>>>> rerunning the conversion.  I propose to make it writable on Monday, at 
>>>>> which point we'll be in development stage 4 for GCC 10 (and to leave 
>>>
>>> This conversion is now in place, read-only for checking purposes.  I've 
>>> done all the usual validation, including in particular checking branch 
>>> tips and tags against SVN.
>>
>> Thanks.
>> Is there or will there be any way using gitweb to browse the vendor and user
>> branches (and also even query which vendors/users and their branches there
>> are)?
>> I've tried
>> https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/vendors/
>> https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/vendors/redhat/
>> https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/vendors/redhat/gcc-9
>> https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/vendors/redhat/gcc-9-branch/
>> and none of that worked.  They don't show up in the heads list
>> https://gcc.gnu.org/git/?p=gcc.git;a=heads
>>
>> 	Jakub
>>
> 
> Try refs/vendors/redhat/heads
> 
> R.
> 

https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/vendors/redhat/heads/gcc-9-branch

Works

Or for tags s/heads/tags/

R.

Re: git conversion in progress

[Richard Earnshaw]

On 11/01/2020 14:51, Richard Earnshaw wrote:
> On 11/01/2020 14:49, Richard Earnshaw wrote:
>> On 11/01/2020 14:48, Jakub Jelinek wrote:
>>> On Sat, Jan 11, 2020 at 02:39:43PM +0000, Joseph Myers wrote:
>>>>> On 11/01/2020 01:18, Joseph Myers wrote:
>>>>>> The GCC SVN repository is now read-only for the move to git, as is the old 
>>>>>> git-svn mirror; the cron job updating that mirror has been disabled, as 
>>>>>> have gccadmin's cron jobs updating DATESTAMP, generating snapshots and 
>>>>>> updating online documentation, until those are updated to work from git.
>>>>>>
>>>>>> Assuming validation passes, I expect to make the converted repository 
>>>>>> available in place of the git-svn mirror (git://gcc.gnu.org/git/gcc.git 
>>>>>> and git+ssh://gcc.gnu.org/git/gcc.git) some time on Saturday.  That will 
>>>>>> be initially read-only to allow for people to review it in case any 
>>>>>> significant conversion issues are noticed late that would justify 
>>>>>> rerunning the conversion.  I propose to make it writable on Monday, at 
>>>>>> which point we'll be in development stage 4 for GCC 10 (and to leave 
>>>>
>>>> This conversion is now in place, read-only for checking purposes.  I've 
>>>> done all the usual validation, including in particular checking branch 
>>>> tips and tags against SVN.
>>>
>>> Thanks.
>>> Is there or will there be any way using gitweb to browse the vendor and user
>>> branches (and also even query which vendors/users and their branches there
>>> are)?

Once you have a checkout, "git ls-remote" will show all the refs on the
server.  I don't know if you can extract the list via the web UI, though.

>>> I've tried
>>> https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/vendors/
>>> https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/vendors/redhat/
>>> https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/vendors/redhat/gcc-9
>>> https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/vendors/redhat/gcc-9-branch/
>>> and none of that worked.  They don't show up in the heads list
>>> https://gcc.gnu.org/git/?p=gcc.git;a=heads
>>>
>>> 	Jakub
>>>
>>
>> Try refs/vendors/redhat/heads
>>
>> R.
>>
> 
> https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/vendors/redhat/heads/gcc-9-branch
> 
> Works
> 
> Or for tags s/heads/tags/
> 
> R.
> 

Re: git conversion in progress

[Jakub Jelinek]

On Sat, Jan 11, 2020 at 02:51:21PM +0000, Richard Earnshaw wrote:
> https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/vendors/redhat/heads/gcc-9-branch
> 
> Works
> 
> Or for tags s/heads/tags/

Indeed, this works, but if one doesn't know what branches there are for
particular vendor or what vendors there are (or what users), I can't find
how to query that.
Can one query it even in git, or just has to know the right spelling of a
particular vendor and then do git-fetch-vendor.sh that_vendor to see it?

	Jakub

Re: git conversion in progress

[Richard Earnshaw]

On 11/01/2020 14:58, Jakub Jelinek wrote:
> On Sat, Jan 11, 2020 at 02:51:21PM +0000, Richard Earnshaw wrote:
>> https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/vendors/redhat/heads/gcc-9-branch
>>
>> Works
>>
>> Or for tags s/heads/tags/
> 
> Indeed, this works, but if one doesn't know what branches there are for
> particular vendor or what vendors there are (or what users), I can't find
> how to query that.
> Can one query it even in git, or just has to know the right spelling of a
> particular vendor and then do git-fetch-vendor.sh that_vendor to see it?
> 

git ls-remote | grep vendors

will list all the vendor branches and tags

Filter that as appropriate.

R.

> 	Jakub
> 

Re: git conversion in progress

[Richard Earnshaw]

On 11/01/2020 15:01, Richard Earnshaw wrote:
> On 11/01/2020 14:58, Jakub Jelinek wrote:
>> On Sat, Jan 11, 2020 at 02:51:21PM +0000, Richard Earnshaw wrote:
>>> https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/vendors/redhat/heads/gcc-9-branch
>>>
>>> Works
>>>
>>> Or for tags s/heads/tags/
>>
>> Indeed, this works, but if one doesn't know what branches there are for
>> particular vendor or what vendors there are (or what users), I can't find
>> how to query that.
>> Can one query it even in git, or just has to know the right spelling of a
>> particular vendor and then do git-fetch-vendor.sh that_vendor to see it?
>>
> 
> git ls-remote | grep vendors
> 
> will list all the vendor branches and tags
> 
> Filter that as appropriate.
> 
> R.
> 
>> 	Jakub
>>
> 


$ git ls-remote|grep vendors|sed -r "s|.*vendors/([^/]+).*|\1|"|sort|uniq
apple
ARM
csl
google
ibm
ix86
linaro
microblaze
redhat
st
suse
ubuntu

We could add this to git-fetch-vendor if invoked without an option.

R.

Re: git conversion in progress

[Joseph Myers]

On Sat, 11 Jan 2020, Richard Earnshaw wrote:

> On 11/01/2020 14:58, Jakub Jelinek wrote:
> > On Sat, Jan 11, 2020 at 02:51:21PM +0000, Richard Earnshaw wrote:
> >> https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/vendors/redhat/heads/gcc-9-branch
> >>
> >> Works
> >>
> >> Or for tags s/heads/tags/
> > 
> > Indeed, this works, but if one doesn't know what branches there are for
> > particular vendor or what vendors there are (or what users), I can't find
> > how to query that.
> > Can one query it even in git, or just has to know the right spelling of a
> > particular vendor and then do git-fetch-vendor.sh that_vendor to see it?
> > 
> 
> git ls-remote | grep vendors

We should probably put something about git ls-remote in git.html.  (I put 
"git branch -a" and "git tag -l" in there as the usual ways to list 
branches / tags once you have a checkout, but ls-remote seems relevant 
both for listing branches and tags without a checkout, and for listing 
those that aren't fetched by default.)

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: git conversion in progress

[Thomas Koenig]

Am 11.01.20 um 15:39 schrieb Joseph Myers:
> This conversion is now in place, read-only for checking purposes.  I've
> done all the usual validation, including in particular checking branch
> tips and tags against SVN.

Is checkout via git+ssh supposed to work for this?  It does not for me
(something about public key rejected), but then I am a complete novice
at using git, so I am more or less doing vodoo git here.

Regards

	Thomas

Re: git conversion in progress

[Richard Earnshaw]

On 11/01/2020 16:33, Thomas Koenig wrote:
> Am 11.01.20 um 15:39 schrieb Joseph Myers:
>> This conversion is now in place, read-only for checking purposes.  I've
>> done all the usual validation, including in particular checking branch
>> tips and tags against SVN.
> 
> Is checkout via git+ssh supposed to work for this?  It does not for me
> (something about public key rejected), but then I am a complete novice
> at using git, so I am more or less doing vodoo git here.
> 
> Regards
> 
>     Thomas

The same key as you use for checking out the svn should work.

It seems that the gcc server is heavily loaded at present - many folk
checking out the new repo.  So it might just be a load issue

Have patience...

R.

Re: git conversion in progress

[Andreas Schwab]

On Jan 11 2020, Richard Earnshaw wrote:

> $ git ls-remote|grep vendors|sed -r "s|.*vendors/([^/]+).*|\1|"|sort|uniq

git ls-remote URL "*/vendors/*" | ...

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: git conversion in progress

[Andreas Schwab]

On Jan 11 2020, Richard Earnshaw wrote:

> Once you have a checkout, "git ls-remote" will show all the refs on the
> server.

You don't need a checkout, git ls-remote works on a remote URL.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: git conversion in progress

[Martin Liška]

On 1/11/20 5:33 PM, Thomas Koenig wrote:
> Am 11.01.20 um 15:39 schrieb Joseph Myers:
>> This conversion is now in place, read-only for checking purposes.  I've
>> done all the usual validation, including in particular checking branch
>> tips and tags against SVN.
> 
> Is checkout via git+ssh supposed to work for this?

It works for me:

$ git remote --verbose
origin	git+ssh://gcc.gnu.org/git/gcc.git (fetch)
origin	git+ssh://gcc.gnu.org/git/gcc.git (push)

>  It does not for me
> (something about public key rejected), but then I am a complete novice
> at using git, so I am more or less doing vodoo git here.

Please paste the error message you face?
Martin

> 
> Regards
> 
>      Thomas

Re: git conversion in progress

[Joseph Myers]

On Sat, 11 Jan 2020, Thomas Koenig wrote:

> Am 11.01.20 um 15:39 schrieb Joseph Myers:
> > This conversion is now in place, read-only for checking purposes.  I've
> > done all the usual validation, including in particular checking branch
> > tips and tags against SVN.
> 
> Is checkout via git+ssh supposed to work for this?  It does not for me
> (something about public key rejected), but then I am a complete novice
> at using git, so I am more or less doing vodoo git here.

If your sourceware username is different from your local username, you 
either need to put username@ in the URL (as described in gitwrite.html), 
or put something like

Host gcc.gnu.org
User <your-sourceware-username>

in your .ssh/config (I prefer the .ssh/config approach, then you never 
need to put the username in the URL).

One thing to note that we need to add to gitwrite.html is to set up git's 
notion of your name and email address,

git config --global user.name "Your Name"
git config --global user.email "Your Email Address"

if you don't already have that in ~/.gitconfig.  If the name or email 
address you want to use for contributing to GCC are different from those 
in ~/.gitconfig - if you want to use different details in different git 
repositories - then do such configuration without --global in your GCC 
checkout to set the name/email address to use when working in that clone.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: git conversion in progress

[Thomas Koenig]

Hi Martin,
>>   It does not for me
>> (something about public key rejected), but then I am a complete novice
>> at using git, so I am more or less doing vodoo git here.
> 
> Please paste the error message you face?

Currently, gcc.gnu.org is totally under water, even accessing the
wiki leads to a timeout, so I can't read the wiki on what to do.

I will try again in a few hours.

Hm... I just hope this is a one-time effect, and isn't an indication
that git uses much more resources, server-side, so the current
infrastructure is not up to the task.  Is git that much more
resource hungry than svn? Or is this unrelated?

Regards

	Thomas

Re: git conversion in progress

[Joseph Myers]

On Sat, 11 Jan 2020, Thomas Koenig wrote:

> Am 11.01.20 um 15:39 schrieb Joseph Myers:
> > This conversion is now in place, read-only for checking purposes.  I've
> > done all the usual validation, including in particular checking branch
> > tips and tags against SVN.
> 
> Is checkout via git+ssh supposed to work for this?  It does not for me
> (something about public key rejected), but then I am a complete novice
> at using git, so I am more or less doing vodoo git here.

If your sourceware username is different from your local username, you 
either need to put username@ in the URL (as described in gitwrite.html), 
or put something like

Host gcc.gnu.org
User <your-sourceware-username>

in your .ssh/config (I prefer the .ssh/config approach, then you never 
need to put the username in the URL).

One thing to note that we need to add to gitwrite.html is to set up git's 
notion of your name and email address,

git config --global user.name "Your Name"
git config --global user.email "Your Email Address"

if you don't already have that in ~/.gitconfig.  If the name or email 
address you want to use for contributing to GCC are different from those 
in ~/.gitconfig - if you want to use different details in different git 
repositories - then do such configuration without --global in your GCC 
checkout to set the name/email address to use when working in that clone.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: git conversion in progress

[Joseph Myers]

On Sat, 11 Jan 2020, Thomas Koenig wrote:

> Am 11.01.20 um 15:39 schrieb Joseph Myers:
> > This conversion is now in place, read-only for checking purposes.  I've
> > done all the usual validation, including in particular checking branch
> > tips and tags against SVN.
> 
> Is checkout via git+ssh supposed to work for this?  It does not for me
> (something about public key rejected), but then I am a complete novice
> at using git, so I am more or less doing vodoo git here.

If your sourceware username is different from your local username, you 
either need to put username@ in the URL (as described in gitwrite.html), 
or put something like

Host gcc.gnu.org
User <your-sourceware-username>

in your .ssh/config (I prefer the .ssh/config approach, then you never 
need to put the username in the URL).

One thing to note that we need to add to gitwrite.html is to set up git's 
notion of your name and email address,

git config --global user.name "Your Name"
git config --global user.email "Your Email Address"

if you don't already have that in ~/.gitconfig.  If the name or email 
address you want to use for contributing to GCC are different from those 
in ~/.gitconfig - if you want to use different details in different git 
repositories - then do such configuration without --global in your GCC 
checkout to set the name/email address to use when working in that clone.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: git conversion in progress

[Thomas Koenig]

Hi Martin,
>>   It does not for me
>> (something about public key rejected), but then I am a complete novice
>> at using git, so I am more or less doing vodoo git here.
> 
> Please paste the error message you face?

Currently, gcc.gnu.org is totally under water, even accessing the
wiki leads to a timeout, so I can't read the wiki on what to do.

I will try again in a few hours.

Hm... I just hope this is a one-time effect, and isn't an indication
that git uses much more resources, server-side, so the current
infrastructure is not up to the task.  Is git that much more
resource hungry than svn? Or is this unrelated?

Regards

	Thomas

Re: git conversion in progress

[Eric S. Raymond]

Thomas Koenig <tkoenig@netcologne.de>:
> Hm... I just hope this is a one-time effect, and isn't an indication
> that git uses much more resources, server-side, so the current
> infrastructure is not up to the task.  Is git that much more
> resource hungry than svn? Or is this unrelated?

Almost certanly unrelated. In normal use git is *spectacularly* faster than
SVN on equivalent operations. 
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: git conversion in progress

[Joseph Myers]

On Sat, 11 Jan 2020, Thomas Koenig wrote:

> Hm... I just hope this is a one-time effect, and isn't an indication
> that git uses much more resources, server-side, so the current
> infrastructure is not up to the task.  Is git that much more
> resource hungry than svn? Or is this unrelated?

I think fetching everything from scratch (which everyone is now doing) is 
much more resource hungry than updating an existing git clone.

The GitHub mirror has successfully updated: 
https://github.com/gcc-mirror/gcc (that has everything, including the 
not-fetched by default refs, plus GitHub-specific refs/pull/* refs for 
pull requests people created on GitHub in the past).

I've also pushed the converted repository to GitLab: 
https://gitlab.com/jsm28/gcc (that has everything as it came out of the 
conversion process, but not the hook configuration in refs/meta/config 
that I created after the conversion was done).

So you can clone either of those and then update the settings in 
.git/config to point to the repository on gcc.gnu.org instead for future 
fetches and commits.

The GitHub mirror should keep automatically updating in future, but I 
won't be updating the GitLab repository.

It's possible that doing a first clone with --mirror and then other clones 
using --reference to use the mirror clone for the objects will also reduce 
resource usage (it will certainly save on disk space locally if you have 
separate checkouts of different branches), but note the caveats about 
--reference in the git-clone manpage.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: git conversion in progress

[Joseph Myers]

On Sat, 11 Jan 2020, Joseph Myers wrote:

> the older test conversions 1 through 7).  Some cron jobs may be re-enabled 
> before then, subject to testing (I have git changes to gcc_release ready, 
> for example, for testing snapshot generation), but the DATESTAMP updates 
> won't be enabled until the repository is writable.

Snapshot generation is working (not yet enabled from cron).  I also have 
DATESTAMP updates working (tested with gcc-reposurgeon-8.git).  However, 
they seem to be much slower than expected.  The slow step is pushing a 
commit made locally to the gcc-reposurgeon-8.git repository; I see 
processes

gccadmin 121505  0.0  0.0 106080  1420 pts/2    S+   22:35   0:00      \_ /bin/sh ./update_version_git
gccadmin 121834  0.0  0.0 214308 52808 pts/2    Sl+  22:35   0:00          \_ git push origin master
gccadmin 121835  0.0  0.0 215396 54300 pts/2    Sl+  22:35   0:00              \_ git-receive-pack /home/gccadmin/gcc-reposurgeon-8.git
gccadmin 121839 99.3  0.8 1597152 613180 pts/2  R+   22:35   1:58              \_ git pack-objects --all-progress-implied --revs --stdout --thin --delta-base-offset --progress

and there should be no need for a time-consuming pack-objects step, the 
DATESTAMP update commit should never add more than four small new objects 
(one commit, two trees, one blob) and such a push when done from a git+ssh 
checkout runs very quickly.  Any advice on why pushing a trival commit 
locally (but not remotely) should be so slow?

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: git conversion in progress

[Joseph Myers]

The repository is now open for commits.  Snapshot generation and DATESTAMP 
updates from cron are now enabled.  I intend to work on the scripts to 
update online documentation, but they aren't done yet.

*Please help with updating documentation for using git with GCC*.  
<https://gcc.gnu.org/ml/gcc-patches/2020-01/msg00623.html> has a list of 
things that ought to be covered in git.html and gitwrite.html (in the 
gcc-wwwdocs.git repository), most of which aren't covered yet, and in 
general any hints and tips about git usage that have come up in recent 
discussions should be added to the documentation if not already there.  In 
addition, once git.html is more complete (has the list of branches added, 
at least) we need to update the GCC home page to link to the new pages in 
place of those for SVN, redirect the old pages to the new ones, and 
generally update references to SVN in wwwdocs and the GCC manuals.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: git conversion in progress

[Georg-Johann Lay]

Am 11.01.20 um 02:18 schrieb Joseph Myers:
> I encourage people to continue to work on improving the documentation for
> using git with GCC
> (<https://gcc.gnu.org/ml/gcc-patches/2020-01/msg00623.html> and
> <https://gcc.gnu.org/ml/gcc-patches/2020-01/msg00625.html> list some of
> the things that it seems it might be useful to document).  Once the
> conversion is done we'll need to update various places referring to SVN on
> the website and in the manuals.

Hi, some random notes:

In order to set user.name and user.email, the doc is using --global.  At 
least for me, the latter is not what I want because I am using git in 
other contexts than contributing to GCC (and am using different e-mail 
then).

In my git setup there is the following which I found quite useful when 
changing .md, or more specifically diff'ing md's and having them reviewed.

diff files allow to indicate what has been changed, for example when you 
changed some function, in the respective hunk the function name is 
displayed.  We can achieve the same for define_insn and similar:

In .gitconfig:

[diff "md"]
       # .git/info/attributes maps *.md to diff=md
       # You can also use .gitattributes for this
       xfuncname = "^[ \t]*\\([ \t]*define_.*$"

This defines the hunk to match define_insn, define_expand etc.  The 
mapping can be performed in the files mentioned above and will read:

# Allow for custom diff hunks in your (global) .gitconfig
# e.g. for machine descriptions the respective entry could read
#
# [diff "md"]
#      # GCC/.gitattributes maps *.md to diff=md
#      xfuncname = "^[ \t]*\\([ \t]*define_.*$"

# GCC machine description
*.md  diff=md

I am not planning to propose respective changes, so if you find it 
helpful, maybe some maintainer can add it.

To see the difference, try

$ git show ec9b602c167fb7dfba717

for example.  In the case it's regarded as helpful, maybe it can added 
where it applies to all branches, not only to one specific branch. 
(Dunno what changing .git/info, .git/config actually does, presumably it 
only applies if the repo is cloned).


Sometimes, one wants a different editor for commit and merge messages 
than the default one (dunno which one this is, presumably determined by 
some env variable).

The respective entry in .gitconfig is:
[core]
	editor = my-editor

or can be achieved by

git config [--global] core.editor="...".  In my case, it's a script that 
gets diffs from ChangeLog deltas and composes them to a commit message 
and then calls my preferred editor on that.

And a question wrt e-mail addresses: I have 2 addresses associated with 
my GCC contributions: one @gcc.gnu.org from my GCC account and one 
non-gnu as mentioned in MAINTAINERS.  Which one should I use for commits?
As far as I remember, the gnu address can cause some problems, but I 
don't remember where...


Johann

Re: git conversion in progress

[Georg-Johann Lay]

Am 11.01.20 um 02:18 schrieb Joseph Myers:
> I encourage people to continue to work on improving the documentation for
> using git with GCC
> (<https://gcc.gnu.org/ml/gcc-patches/2020-01/msg00623.html> and
> <https://gcc.gnu.org/ml/gcc-patches/2020-01/msg00625.html> list some of
> the things that it seems it might be useful to document).

Nitpick:

https://gcc.gnu.org/git.html#tags "Branches and Tags"

writes:

A branch called branchname can be checked out with the following command:

     git clone -b branchname ...

Referring to this as "checking out" is confusing IMO, because it may be 
confused with
     git checkout -b branchname

Whereas the latter is a checkout of some branch (or changing the branch) 
within the same local copy, the former is *cloning* into an individual, 
fresh local copy which has only the mentioned branch.

Dito for tags.

Cloning with only 1 branch is usually done because it faster downloads 
and consumes less local memory.  When one works with more than one 
branch (usually when backporting stuff etc.), the mentioned

git clone --reference original-gcc ...

might be preferable.

Johann

Re: git conversion in progress

[Richard Earnshaw]

On 14/01/2020 11:01, Georg-Johann Lay wrote:
> Am 11.01.20 um 02:18 schrieb Joseph Myers:
>> I encourage people to continue to work on improving the documentation for
>> using git with GCC
>> (<https://gcc.gnu.org/ml/gcc-patches/2020-01/msg00623.html> and
>> <https://gcc.gnu.org/ml/gcc-patches/2020-01/msg00625.html> list some of
>> the things that it seems it might be useful to document).
> 
> Nitpick:
> 
> https://gcc.gnu.org/git.html#tags "Branches and Tags"
> 
> writes:
> 
> A branch called branchname can be checked out with the following command:
> 
>     git clone -b branchname ...
> 
> Referring to this as "checking out" is confusing IMO, because it may be
> confused with
>     git checkout -b branchname
> 
> Whereas the latter is a checkout of some branch (or changing the branch)
> within the same local copy, the former is *cloning* into an individual,
> fresh local copy which has only the mentioned branch.
> 
> Dito for tags.
> 
> Cloning with only 1 branch is usually done because it faster downloads
> and consumes less local memory.  When one works with more than one
> branch (usually when backporting stuff etc.), the mentioned

Well it's likely that the server would have to repack the objects on the
fly to supply just one branch; and it does that less well than the base
pack).  So you'd probably end up with a much larger initial download
than just fetching the entire history (currently ~750M).  It would
likely take longer to fetch as well since the server has to do much more
work to generate that custom pack.

Yes, if you want to work with that single branch long-term you can
repack it again locally (if you have enough ram to produce a really
dense pack).

So frankly, I wouldn't recommend downloading just a single branch.

> 
> git clone --reference original-gcc ...
> 
> might be preferable.
> 
> Johann

Re: git conversion in progress

[Andreas Schwab]

On Jan 14 2020, Georg-Johann Lay wrote:

> or can be achieved by
>
> git config [--global] core.editor="...".

Space instead of = between key and value.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: git conversion in progress

[Andreas Schwab]

On Jan 14 2020, Georg-Johann Lay wrote:

> git clone --reference original-gcc ...

Don't use --reference.  It is too easy to lose work if you don't know
what you are doing.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: git conversion in progress

[Georg-Johann Lay]

Am 14.01.20 um 12:34 schrieb Andreas Schwab:
> On Jan 14 2020, Georg-Johann Lay wrote:
> 
>> git clone --reference original-gcc ...
> 
> Don't use --reference.  It is too easy to lose work if you don't know
> what you are doing.
> 
> Andreas.

Well, then it should not be proposed in git.html then?

There may be git noobs like me that follow the recommendations from there.

Johann

Re: git conversion in progress

[Jakub Jelinek]

On Tue, Jan 14, 2020 at 12:34:13PM +0100, Andreas Schwab wrote:
> On Jan 14 2020, Georg-Johann Lay wrote:
> 
> > git clone --reference original-gcc ...
> 
> Don't use --reference.  It is too easy to lose work if you don't know
> what you are doing.

Wouldn't git clone --reference original-gcc --dissociate ...
be ok?  I.e. just save bandwidth when needing second repo if for whatever
reason git worktree isn't possible?

	Jakub

Re: git conversion in progress

[Andreas Schwab]

On Jan 14 2020, Jakub Jelinek wrote:

> On Tue, Jan 14, 2020 at 12:34:13PM +0100, Andreas Schwab wrote:
>> On Jan 14 2020, Georg-Johann Lay wrote:
>> 
>> > git clone --reference original-gcc ...
>> 
>> Don't use --reference.  It is too easy to lose work if you don't know
>> what you are doing.
>
> Wouldn't git clone --reference original-gcc --dissociate ...
> be ok?  I.e. just save bandwidth when needing second repo if for whatever
> reason git worktree isn't possible?

Yes, this is safe, since the new clone will be independent and
standalone.  It will contain a duplicate of the object database, though.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: git conversion in progress

[Jonathan Wakely]

On Tue, 14 Jan 2020 at 11:37, Georg-Johann Lay <avr@gjlay.de> wrote:
>
> Am 14.01.20 um 12:34 schrieb Andreas Schwab:
> > On Jan 14 2020, Georg-Johann Lay wrote:
> >
> >> git clone --reference original-gcc ...
> >
> > Don't use --reference.  It is too easy to lose work if you don't know
> > what you are doing.
> >
> > Andreas.
>
> Well, then it should not be proposed in git.html then?

It's a work in progress. I've already suggested that worktrees are a
better solution for people who want to work on multiple branches but
save disk space.

Re: git conversion in progress

[Andrew Stubbs]

On 14/01/2020 13:00, Jonathan Wakely wrote:
> On Tue, 14 Jan 2020 at 11:37, Georg-Johann Lay <avr@gjlay.de> wrote:
>>
>> Am 14.01.20 um 12:34 schrieb Andreas Schwab:
>>> On Jan 14 2020, Georg-Johann Lay wrote:
>>>
>>>> git clone --reference original-gcc ...
>>>
>>> Don't use --reference.  It is too easy to lose work if you don't know
>>> what you are doing.
>>>
>>> Andreas.
>>
>> Well, then it should not be proposed in git.html then?
> 
> It's a work in progress. I've already suggested that worktrees are a
> better solution for people who want to work on multiple branches but
> save disk space.

Worktrees are better, for most purposes, but they still have the same 
risks as reference repos.... "I don't need that old tree any more....."

Both are best left for the advanced tricks section.

Andrew

Re: git conversion in progress

[Martin Jambor]

Hi,

On Tue, Jan 14 2020, Andreas Schwab wrote:
> On Jan 14 2020, Georg-Johann Lay wrote:
>
>> git clone --reference original-gcc ...
>
> Don't use --reference.  It is too easy to lose work if you don't know
> what you are doing.

What are the risks, assuming I won't delete the referenced repo which
sits on the same partition of the same local disk as the new one?

I prefer it to worktree because I often just do have master checked out
in multiple directories and worktree would be unhappy about that... and
the risk of accidentally deleting the base worktree are IMHO the same as
the risk of deleting the referenced repo.

Anything that I am missing?  (In other words, do I know what I am doing? :-)

Thanks,

Martin

Re: git conversion in progress

[Andreas Schwab]

On Jan 14 2020, Martin Jambor wrote:

> Hi,
>
> On Tue, Jan 14 2020, Andreas Schwab wrote:
>> On Jan 14 2020, Georg-Johann Lay wrote:
>>
>>> git clone --reference original-gcc ...
>>
>> Don't use --reference.  It is too easy to lose work if you don't know
>> what you are doing.
>
> What are the risks, assuming I won't delete the referenced repo which
> sits on the same partition of the same local disk as the new one?

The risk is if the original repository is gc'd (and nowadays git
automatically runs git gc --auto from time to time) it may lose objects
that are still needed by the referencing repository.  That doesn't
happen with git worktree as the main repository knows about all
references, including worktree local reflogs.

> I prefer it to worktree because I often just do have master checked out
> in multiple directories and worktree would be unhappy about that... and
> the risk of accidentally deleting the base worktree are IMHO the same as
> the risk of deleting the referenced repo.

I'd suggest making more use of local branches.  They are cheep and easy
to use with git.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: git conversion in progress

[Jonathan Wakely]

On Tue, 14 Jan 2020 at 14:28, Martin Jambor <mjambor@suse.cz> wrote:
>
> Hi,
>
> On Tue, Jan 14 2020, Andreas Schwab wrote:
> > On Jan 14 2020, Georg-Johann Lay wrote:
> >
> >> git clone --reference original-gcc ...
> >
> > Don't use --reference.  It is too easy to lose work if you don't know
> > what you are doing.
>
> What are the risks, assuming I won't delete the referenced repo which
> sits on the same partition of the same local disk as the new one?

If I understand correctly (and I don't use --reference so I might be
wrong), with --reference you need to do a git fetch in each separate
repo when you want to upate, because although they share objects they
each have their own "view" of the remotes.

If you do a git pull in a clone created with --reference it will fetch
new objects and store them locally, it won't update the referenced
repo. This means they can diverge over time and aren't sharing as much
as possible (I assume that can be resolved by repack or gc? I don't
know).

Also, anything committed to any worktree is instantly visible to the
others. I have worktrees for master, gcc-9 and gcc-8. After I backport
something to gcc-9 (but before I push it upstream) I can go to the
gcc-8 worktree and do 'git cherry-pick gcc-9' to apply the newest
commit from gcc-9 branch to the gcc-8 branch. There's no need to push
or pull anything between the worktrees, because they are all using the
same clone with the same local branches.

> I prefer it to worktree because I often just do have master checked out
> in multiple directories and worktree would be unhappy about that...

Sounds like a good use for branches :-)

> and
> the risk of accidentally deleting the base worktree are IMHO the same as
> the risk of deleting the referenced repo.

For me, there is zero chance that I'll ever think ~/src/gcc/gcc is an
old tree I don't use. It's also the one where I do work on 'master'
(or branches created from 'master') so I'm unlikely to ever think
that's an unwanted throwaway tree that I don't need. With worktrees
you'll only have 'master' checked out in one of your trees, so it
doesn't take a lot of discipline to know which one is the one
containing all the .git/objects and other repository metadata.

> Anything that I am missing?  (In other words, do I know what I am doing? :-)

Re: git conversion in progress

[Joseph Myers]

On Tue, 14 Jan 2020, Georg-Johann Lay wrote:

> In order to set user.name and user.email, the doc is using --global.  At least
> for me, the latter is not what I want because I am using git in other contexts
> than contributing to GCC (and am using different e-mail then).

Using --global is the simple case for someone who hasn't used git before 
on the system where they are committing changes to GCC (the page says "If 
you have not already configured those in $HOME/.gitconfig).  It gives 
instructions for the case of using a different email address with GCC - 
after giving instructions for the simplest case of a new git user.

> In .gitconfig:
> 
> [diff "md"]
>       # .git/info/attributes maps *.md to diff=md
>       # You can also use .gitattributes for this
>       xfuncname = "^[ \t]*\\([ \t]*define_.*$"

That's already in contrib/gcc-git-customization.sh - but we need to 
document that script in git.html, and also need to add the relevant line 
to the checked-in .gitattributes file.

> And a question wrt e-mail addresses: I have 2 addresses associated with my GCC
> contributions: one @gcc.gnu.org from my GCC account and one non-gnu as
> mentioned in MAINTAINERS.  Which one should I use for commits?
> As far as I remember, the gnu address can cause some problems, but I don't
> remember where...

You can use whatever address you prefer for committing to GCC.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: git conversion in progress

[Joseph Myers]

On Tue, 14 Jan 2020, Georg-Johann Lay wrote:

> A branch called branchname can be checked out with the following command:
> 
>     git clone -b branchname ...
> 
> Referring to this as "checking out" is confusing IMO, because it may be
> confused with
>     git checkout -b branchname
> 
> Whereas the latter is a checkout of some branch (or changing the branch)
> within the same local copy, the former is *cloning* into an individual, fresh
> local copy which has only the mentioned branch.

I think the simple case for git beginners is having separate, independent 
directories for each branch being used, but I see it's been changed to 
describe checking out a branch in an existing clone instead.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: git conversion in progress

[Joseph Myers]

On Tue, 14 Jan 2020, Richard Earnshaw wrote:

> Well it's likely that the server would have to repack the objects on the
> fly to supply just one branch; and it does that less well than the base
> pack).  So you'd probably end up with a much larger initial download
> than just fetching the entire history (currently ~750M).  It would
> likely take longer to fetch as well since the server has to do much more
> work to generate that custom pack.

If you use clone with -b, it still fetches all the branches and tags that 
are fetched by default and puts the branches in remotes/origin/, it just 
changes which branch gets a corresponding branch set up in refs/heads/ and 
checked out initially.  So there is no extra repacking involved.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: git conversion in progress

[Jason Merrill]

On Tue, Jan 14, 2020 at 9:56 AM Andreas Schwab <schwab@linux-m68k.org>
wrote:

> On Jan 14 2020, Martin Jambor wrote:
>


> > On Tue, Jan 14 2020, Andreas Schwab wrote:
> >> On Jan 14 2020, Georg-Johann Lay wrote:
> >>
> >>> git clone --reference original-gcc ...
> >>
> >> Don't use --reference.  It is too easy to lose work if you don't know
> >> what you are doing.
> >
> > What are the risks, assuming I won't delete the referenced repo which
> > sits on the same partition of the same local disk as the new one?
>
> The risk is if the original repository is gc'd (and nowadays git
> automatically runs git gc --auto from time to time) it may lose objects
> that are still needed by the referencing repository.  That doesn't
> happen with git worktree as the main repository knows about all
> references, including worktree local reflogs.
>

Exactly.  I used --reference with my local copy of the old git mirror, and
it's hopelessly corrupt now due to needed objects getting gc'd from the
reference repository.  Very much not for use by novices without
--dissociate.  As the clone man page says,

           NOTE: this is a possibly dangerous operation; do not use it
unless
           you understand what it does. If you clone your repository using
           this option and then delete branches (or use any other Git
command
           that makes any existing commit unreferenced) in the source
           repository, some objects may become unreferenced (or dangling).
           These objects may be removed by normal Git operations (such as
git
           commit) which automatically call git gc --auto. (See git-gc(1).)
If
           these objects are removed and were referenced by the cloned
           repository, then the cloned repository will become corrupt.

I notice that git.html on the website doesn't match what's currently in
wwwdocs git, is automatic updating broken?

Jason

Re: git conversion in progress

[Joseph Myers]

On Tue, 14 Jan 2020, Jason Merrill wrote:

> I notice that git.html on the website doesn't match what's currently in
> wwwdocs git, is automatic updating broken?

/www/gcc/wwwdocs-checkout/cgi-bin/gcc-gitref.cgi had local changes 
(committed, but not reverted in that checkout before they were committed), 
thereby breaking automatic updates:

error: Your local changes to the following files would be overwritten by merge:
        cgi-bin/gcc-gitref.cgi
Please commit your changes or stash them before you merge.
Aborting

I've now reverted that and updated that checkout and am running 
update_web_docs_git which preprocesses the whole website among other 
things.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: git conversion in progress

[Jakub Jelinek]

On Tue, Jan 14, 2020 at 06:06:36PM +0000, Joseph Myers wrote:
> On Tue, 14 Jan 2020, Jason Merrill wrote:
> 
> > I notice that git.html on the website doesn't match what's currently in
> > wwwdocs git, is automatic updating broken?
> 
> /www/gcc/wwwdocs-checkout/cgi-bin/gcc-gitref.cgi had local changes 
> (committed, but not reverted in that checkout before they were committed), 
> thereby breaking automatic updates:
> 
> error: Your local changes to the following files would be overwritten by merge:
>         cgi-bin/gcc-gitref.cgi
> Please commit your changes or stash them before you merge.
> Aborting
> 
> I've now reverted that and updated that checkout and am running 
> update_web_docs_git which preprocesses the whole website among other 
> things.

Oops, sorry for that, forgot to remove those after I was done with testing
it.

	Jakub

Re: git conversion in progress

[Martin Jambor]

Hi,

On Tue, Jan 14 2020, Andreas Schwab wrote:
> On Jan 14 2020, Martin Jambor wrote:
>
>> Hi,
>>
>> On Tue, Jan 14 2020, Andreas Schwab wrote:
>>> On Jan 14 2020, Georg-Johann Lay wrote:
>>>
>>>> git clone --reference original-gcc ...
>>>
>>> Don't use --reference.  It is too easy to lose work if you don't know
>>> what you are doing.
>>
>> What are the risks, assuming I won't delete the referenced repo which
>> sits on the same partition of the same local disk as the new one?
>
> The risk is if the original repository is gc'd (and nowadays git
> automatically runs git gc --auto from time to time) it may lose objects
> that are still needed by the referencing repository.  That doesn't
> happen with git worktree as the main repository knows about all
> references, including worktree local reflogs.

Aha, that makes sense.  In my intended setup, that *probably* would not
happen because the referenced repo would only hold stuff from the
upstream one but the danger sounds real enough for me to reconsider and
use worktrees and a number of branches pointing to the same commit as
master.

Thanks,

Martin

Re: git conversion in progress

[Georg-Johann Lay]

Am 11.01.20 um 02:18 schrieb Joseph Myers:
> I encourage people to continue to work on improving the documentation for
> using git with GCC

Hi, the front page reads "Our sources are readily and freely available 
via SVN...", similar recommendation for SVN in

https://gcc.gnu.org/snapshots.html

Johann

Re: git conversion in progress

[Jonathan Wakely]

On Thu, 16 Jan 2020 at 09:55, Georg-Johann Lay <avr@gjlay.de> wrote:
>
> Am 11.01.20 um 02:18 schrieb Joseph Myers:
> > I encourage people to continue to work on improving the documentation for
> > using git with GCC
>
> Hi, the front page reads "Our sources are readily and freely available
> via SVN...", similar recommendation for SVN in
>
> https://gcc.gnu.org/snapshots.html

Yes, it's been said more than once that the web pages will be updated
once the docs for using git are ready.

Re: git conversion in progress

[Gerald Pfeifer]

On Thu, 16 Jan 2020, Jonathan Wakely wrote:
> On Thu, 16 Jan 2020 at 09:55, Georg-Johann Lay <avr@gjlay.de> wrote:
>> Hi, the front page reads "Our sources are readily and freely available
>> via SVN...", similar recommendation for SVN in
> Yes, it's been said more than once that the web pages will be updated
> once the docs for using git are ready.

I've been replacing references to SVN by ones to Git as time permits,
and hit that instance this evening:

  https://gcc.gnu.org/ml/gcc-patches/2020-01/msg00990.html

Given that SVN does not work any more (in the sense that it's not
updated any longer, it still appears accessible if frozen), I don't
think that's harmful, and I believe what we have in place now isn't
actually too bad?

Gerald

Re: git conversion in progress

[Andreas Schwab]

All (annotated) release tags are misnamed.  For example,
refs/tags/releases/gcc-9.2.0 is really named gcc_9_2_0_release:

    $ git cat-file -p releases/gcc-9.2.0 
    object a0c06cc27d2146b7d86758ffa236516c6143d62c
    type commit
    tag gcc_9_2_0_release
    tagger Jakub Jelinek <jakub@gcc.gnu.org> 1565595539 +0200

    Tagging source as tags/gcc_9_2_0_release

That breaks git describe:

    $ git describe releases/gcc-9.2.0 
    warning: tag 'gcc_9_2_0_release' is really 'releases/gcc-9.2.0' here
    gcc_9_2_0_release

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: git conversion in progress

[Gerald Pfeifer]

On Mon, 13 Jan 2020, Joseph Myers wrote:
> In addition, once git.html is more complete (has the list of branches 
> added, at least) we need to update the GCC home page to link to the new 
> pages in place of those for SVN, redirect the old pages to the new ones, 
> and generally update references to SVN in wwwdocs and the GCC manuals.

I have removed all references to svnwrite.html and svn.html from our
own pages, added redirects to gitwrite.html and git.html, respectively,
and after svnwrite.html a few days ago now also removed svn.html.

There's some further changes I have in the queue, but the web site side
of things should be mostly converted now, modulo

 - more Git-specifics as you had pointed out,
 - bugs/reghunt.html which Jonathan offered to look into, and 
 - simtest-howto.html which looks a little dead to me.

Gerald

Re: git conversion in progress

[Peter Bergner]

On 1/22/20 3:26 AM, Gerald Pfeifer wrote:
> On Mon, 13 Jan 2020, Joseph Myers wrote:
>> In addition, once git.html is more complete (has the list of branches 
>> added, at least) we need to update the GCC home page to link to the new 
>> pages in place of those for SVN, redirect the old pages to the new ones, 
>> and generally update references to SVN in wwwdocs and the GCC manuals.
> 
> I have removed all references to svnwrite.html and svn.html from our
> own pages, added redirects to gitwrite.html and git.html, respectively,
> and after svnwrite.html a few days ago now also removed svn.html.

The rsync.html page can be removed too, since that was a way to download
the entire svn repo.  With git clone, you get the entire repo, so rsync
isn't needed anymore.

Peter

Re: git conversion in progress

[Jakub Jelinek]

On Wed, Jan 22, 2020 at 11:41:22AM -0600, Peter Bergner wrote:
> On 1/22/20 3:26 AM, Gerald Pfeifer wrote:
> > On Mon, 13 Jan 2020, Joseph Myers wrote:
> >> In addition, once git.html is more complete (has the list of branches 
> >> added, at least) we need to update the GCC home page to link to the new 
> >> pages in place of those for SVN, redirect the old pages to the new ones, 
> >> and generally update references to SVN in wwwdocs and the GCC manuals.
> > 
> > I have removed all references to svnwrite.html and svn.html from our
> > own pages, added redirects to gitwrite.html and git.html, respectively,
> > and after svnwrite.html a few days ago now also removed svn.html.
> 
> The rsync.html page can be removed too, since that was a way to download
> the entire svn repo.  With git clone, you get the entire repo, so rsync
> isn't needed anymore.

I disagree, it isn't just about downloading a svn repo, but mailing list
archives too:
rsync rsync://gcc.gnu.org/ | grep gcc
crossgcc-ftp   	crossgcc ftp area ()
gcc-ftp        	gcc ftp area ()
gcc-svn        	gcc svn repository ()
gcc-cvs        	gcc cvs repository ()
gcc-gnats      	gcc gnats database ()
gcc-htdocs     	gcc htdocs ()
crossgcc-ml-archive	crossgcc mailing list archive ()
gcc-ml-archive 	gcc mailing list archive ()
gccadmin-ml-archive	gccadmin mailing list archive ()
gcc-announce-ml-archive	gcc-announce mailing list archive ()
gcc-bugs-ml-archive	gcc-bugs mailing list archive ()
gcc-cvs-ml-archive	gcc-cvs mailing list archive ()
gcc-cvs-wwwdocs-ml-archive	gcc-cvs-wwwdocs mailing list archive ()
gcc-help-ml-archive	gcc-help mailing list archive ()
gcc-patches-ml-archive	gcc-patches mailing list archive ()
gcc-prs-ml-archive	gcc-prs mailing list archive ()
gcc-regression-ml-archive	gcc-regression mailing list archive ()
gcc-testresults-ml-archive	gcc-testresults mailing list archive ()

	Jakub

Re: git conversion in progress

[Gerald Pfeifer]

On Wed, 22 Jan 2020, Jakub Jelinek wrote:
>> The rsync.html page can be removed too, since that was a way to download
>> the entire svn repo.  With git clone, you get the entire repo, so rsync
>> isn't needed anymore.
> I disagree, it isn't just about downloading a svn repo, but mailing list
> archives too:

And notably our download site (for use by mirror sites):

> gcc-ftp        	gcc ftp area ()

I'll take the AI to document this explicitly after my current set
of outstanding changes.

Gerald