Re: Trouble with rsync-access

Re: Trouble with rsync-access

[Volker Reichelt]

On  1 Jun, Ian Lance Taylor wrote:
> Volker Reichelt <reichelt@igpm.rwth-aachen.de> writes:
> 
>> since a couple of days rsync-access to the gcc sources doesn't
>> work as it used to. Rsyncing my local archive and then checking
>> out from there results in an error message:
>> 
>>   cvs checkout: Updating gcc
>>   cvs [checkout aborted]: cannot stat /sourceware/cvs-lockdir/gcc:
>>     No such file or directory
>> 
>> The culprit seems to be the line
>> 
>>   LockDir=/sourceware/cvs-lockdir/gcc
>> 
>> which got added to CVSROOT/config recently:
>> 
>>   date	2004.05.30.01.29.28;	author root;	state Exp;
>> 
>> After deleting this line manually, the check-out works fine.
>> But that's only a work-around since the next rsync will break things again.
>> 
>> Can the change be reverted or be worked around in some other way?
> 
> We made that change intentionally.  I don't know that we have ever
> promised that you can rsync our CVS repository and then use it
> locally.  It's great if it works, but I don't think we're going to
> accept responsibility for having it always work.

Well, but it is desirable to make it work as rsync.html states:

  In an ongoing effort to accelerate development of GCC and provide an open development
  environment, we are offering our CVS repository and various other data through anonymous
  rsync access.

  That way you can make local copies of the GCC CVS repository to ease the burden on the
  GCC main site, and browse the source locally using cvs.

So, making rsync difficult to use seems to be counter-productive, since
it puts more load on GCC's CVS.

In addition bugs/reghunt.html states:

  Using rsync to get a local copy of the GCC CVS repository is highly recommended for
  regression hunts. You'll be checking out the tree used for the regression search over
  and over again and won't want to affect access times for other GCC developers
  who are using the real repository, and it will also be faster for you.

That's why I would appreciate rsync-access to be as easy as possible.
However, if there's no chance of going back to the old version we should
at least document the new behavior in rsync.html.

> That said, you can make it continue to work in a straightforward
> manner by creating the directory /sourceware/cvs-lockdir/gcc on your
> system, possibly using symlinks.

Not everybody has write permissions outside his/her home directory :-(
(I'll have to use a shell script that does the patching after the
rsync.)

> Ian

Regards,
Volker

Re: Trouble with rsync-access

[Christopher Faylor]

On Tue, Jun 01, 2004 at 05:52:58PM +0200, Volker Reichelt wrote:
>On  1 Jun, Ian Lance Taylor wrote:
>> Volker Reichelt <reichelt@igpm.rwth-aachen.de> writes:
>> 
>>> since a couple of days rsync-access to the gcc sources doesn't
>>> work as it used to. Rsyncing my local archive and then checking
>>> out from there results in an error message:
>>> 
>>>   cvs checkout: Updating gcc
>>>   cvs [checkout aborted]: cannot stat /sourceware/cvs-lockdir/gcc:
>>>     No such file or directory
>>> 
>>> The culprit seems to be the line
>>> 
>>>   LockDir=/sourceware/cvs-lockdir/gcc
>>> 
>>> which got added to CVSROOT/config recently:
>>> 
>>>   date	2004.05.30.01.29.28;	author root;	state Exp;
>>> 
>>> After deleting this line manually, the check-out works fine.
>>> But that's only a work-around since the next rsync will break things again.
>>> 
>>> Can the change be reverted or be worked around in some other way?
>> 
>> We made that change intentionally.  I don't know that we have ever
>> promised that you can rsync our CVS repository and then use it
>> locally.  It's great if it works, but I don't think we're going to
>> accept responsibility for having it always work.
>
>Well, but it is desirable to make it work as rsync.html states:
>
>  In an ongoing effort to accelerate development of GCC and provide an open development
>  environment, we are offering our CVS repository and various other data through anonymous
>  rsync access.
>
>  That way you can make local copies of the GCC CVS repository to ease the burden on the
>  GCC main site, and browse the source locally using cvs.
>
>So, making rsync difficult to use seems to be counter-productive, since
>it puts more load on GCC's CVS.

Hmm.  That wording sort of presupposes that rsync uses less resources
than cvs.  I suspect that isn't true, i.e., it is not a one-to-one
rsync==good, cvs==bad situation.  For instance, if you are just using
rsync to download the repository for a couple of people to use, I
suspect that you are putting more load on the machine than you would be
if you were just using cvs.

One of the reasons for this CVSROOT/config change was to reduce the load
on cvs so, as Frank says, we are not going to revert the change unless
someone reports a serious problem.  Having to edit CVSROOT/config after
an rsync does not quality as a serious problem.

Note that savannah.gnu.org uses rsync to grab the gcc repositories and
cvs checkouts from savannah are apparently unaffected.

>Not everybody has write permissions outside his/her home directory :-(
>(I'll have to use a shell script that does the patching after the
>rsync.)

...or just exclude that one file when you are doing rsyncs.  That's
apparently what savannah does.

cgf

Re: Trouble with rsync-access

[Alexandre Oliva]

On Jun  1, 2004, Christopher Faylor <cgf@alum.bu.edu> wrote:

> One of the reasons for this CVSROOT/config change was to reduce the load
> on cvs so, as Frank says, we are not going to revert the change unless
> someone reports a serious problem.  Having to edit CVSROOT/config after
> an rsync does not quality as a serious problem.

I actually find the need to edit the file a feature: this ensures I
don't accidentally check something only in my local copy of the CVS
tree, instead of in the upstream repository.

That said, it would be quite nice if the pathname to the lock
directory was something beginning with /tmp/, such that someone could
more easily duplicate the lock tree without root access or polluting
the / directory.

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

Re: Trouble with rsync-access

[Marc Espie]

In article <20040601171711.GA11911@coe.casa.cgf.cx> you write:
>Hmm.  That wording sort of presupposes that rsync uses less resources
>than cvs.  I suspect that isn't true, i.e., it is not a one-to-one
>rsync==good, cvs==bad situation.  For instance, if you are just using
>rsync to download the repository for a couple of people to use, I
>suspect that you are putting more load on the machine than you would be
>if you were just using cvs.

It's very hard to use more resources than cvs.  cvs has never been
designed to handle client/server gracefully. It's not in their design
spec.

Rsync is a bit disk-intensive, since it needs to access all files. Not
more disk-intensive than cvs, mind you, unless you use cvs to acces a
small part of the repository.

I have lots of experience in using rsync and cvs, and I can guarantee
that having an rsync'ed repository is very, very useful, and much
faster than cvs as soon as you start to use it. Think of all the times
you're going to read through the Changelog, look at files history,
perform diffs and annotate, especially looking for regressions. In all
those case, rsync is a clear win.

If you're still concerned about the load, all the cvs-savvy projects
(that is, all the BSD) are now using cvsync to mirror repositories to
facilitate anonymous access (and speed up read queries). It's very easy
to set up, especially compared to the old cvsup alternative (no need for
a functioning modula3 installation), real quickly, and designed like an
rsync optimized for cvs purposes...

Re: Trouble with rsync-access

[Ian Lance Taylor]

espie@quatramaran.ens.fr (Marc Espie) writes:

> It's very hard to use more resources than cvs.  cvs has never been
> designed to handle client/server gracefully. It's not in their design
> spec.
> 
> Rsync is a bit disk-intensive, since it needs to access all files. Not
> more disk-intensive than cvs, mind you, unless you use cvs to acces a
> small part of the repository.

I want to note here that we are running with some special patches to
CVS, which I wrote, which make the common case of 'cvs update' to the
trunk quite a bit faster.  In that case, our CVS, on the server, does
not need to open a file which is up to date on the client.  This leads
to visible speed-ups for the client, and less load on the server.

But of course rsync is fine too.

Ian

Re: Trouble with rsync-access

[Volker Reichelt]

On  1 Jun, Andreas Schwab wrote:
> Volker Reichelt <reichelt@igpm.rwth-aachen.de> writes:
> 
>> The culprit seems to be the line
>>
>>   LockDir=/sourceware/cvs-lockdir/gcc
>>
>> which got added to CVSROOT/config recently:
>>
>>   date	2004.05.30.01.29.28;	author root;	state Exp;
>>
>> After deleting this line manually, the check-out works fine.
>> But that's only a work-around since the next rsync will break things again.
>>
>> Can the change be reverted or be worked around in some other way?
> 
> Add "--exclude=CVSROOT/config" to the rsync command line.

That seems to be the easiest thing to do. And it works, too! :-)
I'll cough up a patch to document this in rsync.html later.

Btw, I'd suggest to exclude CVSROOT/history and CVS/updatelog, too,
since these are big files of little use in a local repository, right?

> Andreas.

Thanks,
Volker

Re: Trouble with rsync-access

[Andreas Schwab]

Volker Reichelt <reichelt@igpm.rwth-aachen.de> writes:

> The culprit seems to be the line
>
>   LockDir=/sourceware/cvs-lockdir/gcc
>
> which got added to CVSROOT/config recently:
>
>   date	2004.05.30.01.29.28;	author root;	state Exp;
>
> After deleting this line manually, the check-out works fine.
> But that's only a work-around since the next rsync will break things again.
>
> Can the change be reverted or be worked around in some other way?

Add "--exclude=CVSROOT/config" to the rsync command line.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Trouble with rsync-access

[Frank Ch. Eigler]

[-- Attachment #1: Type: text/plain, Size: 812 bytes --]

Hi -


Volker Reichelt <reichelt@igpm.rwth-aachen.de> wrote:

> [...]
>   LockDir=/sourceware/cvs-lockdir/gcc
> After deleting this line manually, the check-out works fine.
> But that's only a work-around since the next rsync will break things again.

Please consider creating such a directory on your mirror machine,
or perhaps using a script to always remove that directive.

> Can the change be reverted or be worked around in some other way?
> [...]

Perhaps.  Directives like this were placed in most or all
sourceware-hosted cvs repositories, so as to secure anoncvs, 
concentrate locking to a single directory, and to still allow
uberbaum to work.  We are interested in any other problems,
though ones at most as serious than this one likely won't
justify a switch back.


- FChE

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Trouble with rsync-access

[Ian Lance Taylor]

Volker Reichelt <reichelt@igpm.rwth-aachen.de> writes:

> since a couple of days rsync-access to the gcc sources doesn't
> work as it used to. Rsyncing my local archive and then checking
> out from there results in an error message:
> 
>   cvs checkout: Updating gcc
>   cvs [checkout aborted]: cannot stat /sourceware/cvs-lockdir/gcc:
>     No such file or directory
> 
> The culprit seems to be the line
> 
>   LockDir=/sourceware/cvs-lockdir/gcc
> 
> which got added to CVSROOT/config recently:
> 
>   date	2004.05.30.01.29.28;	author root;	state Exp;
> 
> After deleting this line manually, the check-out works fine.
> But that's only a work-around since the next rsync will break things again.
> 
> Can the change be reverted or be worked around in some other way?

We made that change intentionally.  I don't know that we have ever
promised that you can rsync our CVS repository and then use it
locally.  It's great if it works, but I don't think we're going to
accept responsibility for having it always work.

That said, you can make it continue to work in a straightforward
manner by creating the directory /sourceware/cvs-lockdir/gcc on your
system, possibly using symlinks.

Ian

Trouble with rsync-access

[Volker Reichelt]

Hi,

since a couple of days rsync-access to the gcc sources doesn't
work as it used to. Rsyncing my local archive and then checking
out from there results in an error message:

  cvs checkout: Updating gcc
  cvs [checkout aborted]: cannot stat /sourceware/cvs-lockdir/gcc:
    No such file or directory

The culprit seems to be the line

  LockDir=/sourceware/cvs-lockdir/gcc

which got added to CVSROOT/config recently:

  date	2004.05.30.01.29.28;	author root;	state Exp;

After deleting this line manually, the check-out works fine.
But that's only a work-around since the next rsync will break things again.

Can the change be reverted or be worked around in some other way?

Btw, the CVSROOT directory contains a lot of old temporary ".#*" files
that can probably be deleted.

Regards,
Volker