The GNU CVS Server: subversions

The GNU CVS Server: subversions

[Jeff Bailey]

Friends, Redhatters, Country(wo)men...

Many of you have heard of `subversions' the new GNU CVS server.  If not, 
basically, it's intended to be a fast, secure, and incredably well-named 
server dedicated to hosting CVS repositories for GNU projects.  The 
general idea is that hopefully all GNU projects will find there way over 
there.  There are 2 problems with this:

1) GNU doesn't have the greatest reputation for secure and fast servers.
2) Y'all already have a perfectly functioning `sourceware' server.

The solution: With your permission, I'd like to rsync over the 
repositories so that they are available on subversions.  I have already 
tested this (with tromey's permission) for automake and it appears to 
work.  I haven't figured out a reasonable update schedule yet that 
balances accuracy and bandwidth, but that doesn't have to be done yet.

The subversions server features anoncvs access, cvs-web, and anonymous 
rsync access.  For projects that are hosted there, write access is 
controlled per module.  We are trying to position it so that anyone who 
needs a snapshot of GNU code can go to a single place (or one of it's 
mirrors) and find it.  I'd appreciate your help in acheiving this.

As we used to say in debating: "I now stand ready for cross-examination" =)

-- 
Do not meddle in the affairs of cyborgs for you are conductive and
can support 110 volts.

Re: The GNU CVS Server: subversions

[Chris Faylor]

On Sat, Mar 11, 2000 at 06:23:44AM -0800, Jeff Bailey wrote:
>Friends, Redhatters, Country(wo)men...
>
>Many of you have heard of `subversions' the new GNU CVS server.  If not, 
>basically, it's intended to be a fast, secure, and incredably well-named 
>server dedicated to hosting CVS repositories for GNU projects.  The 
>general idea is that hopefully all GNU projects will find there way over 
>there.  There are 2 problems with this:
>
>1) GNU doesn't have the greatest reputation for secure and fast servers.
>2) Y'all already have a perfectly functioning `sourceware' server.
>
>The solution: With your permission, I'd like to rsync over the 
>repositories so that they are available on subversions.  I have already 
>tested this (with tromey's permission) for automake and it appears to 
>work.  I haven't figured out a reasonable update schedule yet that 
>balances accuracy and bandwidth, but that doesn't have to be done yet.
>
>The subversions server features anoncvs access, cvs-web, and anonymous 
>rsync access.  For projects that are hosted there, write access is 
>controlled per module.  We are trying to position it so that anyone who 
>needs a snapshot of GNU code can go to a single place (or one of it's 
>mirrors) and find it.  I'd appreciate your help in acheiving this.
>
>As we used to say in debating: "I now stand ready for cross-examination" =)

I don't understand the goal of periodically transferring a CVS
repository to another server.  If I was interested in contributing to a
project I would use the main repository.  This also applies to reading
from a repository.  One of the reasons (the only reason?) people use CVS
repositories is to ensure that they are up to date with the main
development source.

A repository that has been synced with the master is going to be out of
date most of the time if there is active development going on.  I am
not sure that I want to handle the inevitable user questions and
confusion that would result from this.

cgf

Re: The GNU CVS Server: subversions

[Jeff Bailey]

On Sat, Mar 11, 2000 at 10:34:46AM -0500, Chris Faylor wrote:

> I don't understand the goal of periodically transferring a CVS
> repository to another server.  If I was interested in contributing to a
> project I would use the main repository.  This also applies to reading
> from a repository.  One of the reasons (the only reason?) people use CVS
> repositories is to ensure that they are up to date with the main
> development source.

The other reason that people checkout sources is to get access to code 
that has a longer release cycle.  For example, I frequently take 
snapshots of glibc, because the release cycles are too infrequent to meet 
my needs for the Hurd.  I have also done the same with cygwin, not 
because I wanted bleeding edge sources (I'd download winsup for that), 
but because I wanted to use `cvs diff'.

This is intended to be a read-only mirror, and people contributing to the
source should use the master repository.  Having everything on the
subversions repository is intended to serve casual developers (and
observers) to whom it's not critical that the source be up to the minute. 

> A repository that has been synced with the master is going to be out of
> date most of the time if there is active development going on.  I am
> not sure that I want to handle the inevitable user questions and
> confusion that would result from this.

This is why I wanted to discuss update frequencies.  With some effort, I 
might be able to add monitor the commit lists and have it rsync after 
each one - I don't know if that's good or not.  Otherwise, I was thinking 
of every 6 hours.

-- 
Do not meddle in the affairs of cyborgs for you are conductive and
can support 110 volts.

Re: The GNU CVS Server: subversions

[Jason Molenda]

On Sat, Mar 11, 2000 at 10:34:46AM -0500, Chris Faylor wrote:

> I don't understand the goal of periodically transferring a CVS
> repository to another server.  

I like for a few reasons:

  (a) it distributes anoncvs load
  (b) it gives better network fault tolerance for when the Internet wobbles
  (c) it means more copies of the repo are in existance, which means
      I sleep better if the Cygnus machine room goes up in flames

Of course, I set up the rsync server on sourceware so that anyone,
anywhere, at any time can rsync the CVS repositories on sourceware.
Even if the overseers list reached some kind of consensus that we
didn't like this, they'll be free to do it on their own.  I suppose
we (at overseers) could not-advertise the FSF mirrors if we really
didn't want them doing this.

The FreeBSD project distributes their CVS load widely via cvsup;
I think it is a good idea.

People with cvs write access would still need to come to sourceware,
of course.

> A repository that has been synced with the master is going to be out of
> date most of the time if there is active development going on.  

I assume there would be a sync'ing schedule of at least once a day.
That's not unreasonable -- it is rare that things _really_ change
significantly in any one twenty-four hour period.  If it were to
be done more frequently (say two or maybe even three times a day),
it'd make sense to measure load on the sourceware system over a 24
hour period for a week (maybe counting the number of concurrent
anoncvs users via ps aux greppery, which is a good measure of how
busy the cvs disk is) and do the mirroring at a time when the system
is less busy.

> I am
> not sure that I want to handle the inevitable user questions and
> confusion that would result from this.

It's up to us how to document and advertise the existance of a cvs
mirror.  When there is only one, I admit it would complicate the
instructions.  If we ever pick up more cvs mirrors, then we can
have nicely generalized instructions with a pointer to a list of
mirrors for folks who want something closer to home.

On the FSF web pages, they'll undoubtedly want to provide instructions
for pserver access to (say) automake that refer to the repo on
subversions.  I think that's just fine.

Incidentally, I assume that the subversions folks are only interested
in mirroring GNU stuff like automake, binutils, or GDB.

Well, anyway, MHO

Jason

Re: The GNU CVS Server: subversions

[Jeff Bailey]

On Sat, Mar 11, 2000 at 11:55:21AM -0800, Jason Molenda wrote:

> Of course, I set up the rsync server on sourceware so that anyone,
> anywhere, at any time can rsync the CVS repositories on sourceware.
> Even if the overseers list reached some kind of consensus that we
> didn't like this, they'll be free to do it on their own.  I suppose
> we (at overseers) could not-advertise the FSF mirrors if we really
> didn't want them doing this.

While it's true that it's setup that I could do this without asking 
first, I figured that going out of my way to piss you guys off would be 
SOP^H^H^Hexcessively annoying behaviour.  =)

> > A repository that has been synced with the master is going to be out of
> > date most of the time if there is active development going on.  
> 
> I assume there would be a sync'ing schedule of at least once a day.
> That's not unreasonable -- it is rare that things _really_ change
> significantly in any one twenty-four hour period.  If it were to
> be done more frequently (say two or maybe even three times a day),
> it'd make sense to measure load on the sourceware system over a 24
> hour period for a week (maybe counting the number of concurrent
> anoncvs users via ps aux greppery, which is a good measure of how
> busy the cvs disk is) and do the mirroring at a time when the system
> is less busy.

Absolutely: we could arrange to sync just before the busy period to give 
the best snapshots, or whatever is appropriate.

> It's up to us how to document and advertise the existance of a cvs
> mirror.  When there is only one, I admit it would complicate the
> instructions.  If we ever pick up more cvs mirrors, then we can
> have nicely generalized instructions with a pointer to a list of
> mirrors for folks who want something closer to home.

I don't know if it's reasonable to assume that someone somewhere will 
choose to mirror from subversions.  Those instructions would be useful 
for that as well.

> On the FSF web pages, they'll undoubtedly want to provide instructions
> for pserver access to (say) automake that refer to the repo on
> subversions.  I think that's just fine.

The maintainers of the packages are welcome to refer to whichever 
repository they feel happiest.  The higher-volume packages may want to 
explicetly refer people to or away from the master to suit their needs 
(bandwidth vs. convergence times)

> Incidentally, I assume that the subversions folks are only interested
> in mirroring GNU stuff like automake, binutils, or GDB.

Correct.  My example using cygwin was probably misleading.  Sorry.

-- 
Do not meddle in the affairs of cyborgs for you are conductive and
can support 110 volts.

Re: The GNU CVS Server: subversions

[Jeffrey A Law]

  In message < 20000311062344.H24678@sparky.nisa.net >you write:
  > The solution: With your permission, I'd like to rsync over the 
  > repositories so that they are available on subversions.  I have already 
  > tested this (with tromey's permission) for automake and it appears to 
  > work.  I haven't figured out a reasonable update schedule yet that 
  > balances accuracy and bandwidth, but that doesn't have to be done yet.
  > 
  > The subversions server features anoncvs access, cvs-web, and anonymous 
  > rsync access.  For projects that are hosted there, write access is 
  > controlled per module.  We are trying to position it so that anyone who 
  > needs a snapshot of GNU code can go to a single place (or one of it's 
  > mirrors) and find it.  I'd appreciate your help in acheiving this.
Please do -- we've been meaning to start providing anoncvs mirrors for some
time, but just haven't gotten around to it.

Let us know when it's up and we'll note the existence of the CVS mirrors on
our web pages.

jeff

Re: The GNU CVS Server: subversions

[Jeffrey A Law]

  In message < 20000311115236.A15400@sparky.nisa.net >you write:
  > > A repository that has been synced with the master is going to be out of
  > > date most of the time if there is active development going on.  I am
  > > not sure that I want to handle the inevitable user questions and
  > > confusion that would result from this.
  > 
  > This is why I wanted to discuss update frequencies.  With some effort, I 
  > might be able to add monitor the commit lists and have it rsync after 
  > each one - I don't know if that's good or not.  Otherwise, I was thinking 
  > of every 6 hours.
The thing to remember is we can (and likely will) adjust it over time as
necessary.  Pick whatever you think is reasonable and we'll fine tune it
as we get some experience with this stuff.

Thanks,
jeff

Re: The GNU CVS Server: subversions

[Chris Faylor]

On Mon, Mar 13, 2000 at 10:10:01AM -0700, Jeffrey A Law wrote:
>
>  In message < 20000311062344.H24678@sparky.nisa.net >you write:
>  > The solution: With your permission, I'd like to rsync over the 
>  > repositories so that they are available on subversions.  I have already 
>  > tested this (with tromey's permission) for automake and it appears to 
>  > work.  I haven't figured out a reasonable update schedule yet that 
>  > balances accuracy and bandwidth, but that doesn't have to be done yet.
>  > 
>  > The subversions server features anoncvs access, cvs-web, and anonymous 
>  > rsync access.  For projects that are hosted there, write access is 
>  > controlled per module.  We are trying to position it so that anyone who 
>  > needs a snapshot of GNU code can go to a single place (or one of it's 
>  > mirrors) and find it.  I'd appreciate your help in acheiving this.
>Please do -- we've been meaning to start providing anoncvs mirrors for some
>time, but just haven't gotten around to it.
>
>Let us know when it's up and we'll note the existence of the CVS mirrors on
>our web pages.

Has cvs access been a problem on sourceware, bandwidth-wise?

Just curious.

cgf

bandwidth usage

[Jim Kingdon]

> Has cvs access been a problem on sourceware, bandwidth-wise?

The conventional wisdom is that FTP downloads dominate other users of
bandwidth.  And a quick check reveals this seem to be right:

    [kingdon@sourceware kingdon]$ ps auxww | grep ftp | wc -l
	 29
    [kingdon@sourceware kingdon]$ ps auxww | grep cvs | wc -l
	  2
    [kingdon@sourceware kingdon]$ 

However, we don't have any particularly good way of measuring this
(CVS doesn't keep any useful logs, and if the OS has any way of doing
so we aren't using it).

Re: bandwidth usage

[Chris Faylor]

On Mon, Mar 13, 2000 at 12:56:08PM -0500, Jim Kingdon wrote:
>> Has cvs access been a problem on sourceware, bandwidth-wise?
>
>The conventional wisdom is that FTP downloads dominate other users of
>bandwidth.  And a quick check reveals this seem to be right:
>
>    [kingdon@sourceware kingdon]$ ps auxww | grep ftp | wc -l
>	 29
>    [kingdon@sourceware kingdon]$ ps auxww | grep cvs | wc -l
>	  2
>    [kingdon@sourceware kingdon]$ 
>
>However, we don't have any particularly good way of measuring this
>(CVS doesn't keep any useful logs, and if the OS has any way of doing
>so we aren't using it).

I would think that CVS would be a fraction of FTP, at least once the
repository is checked out.

On reflection, I guess it really doesn't matter if CVS is a bandwidth
hog.  As long as it is sharing a channel with a theorized resource eater
like ftp downloads, it makes sense to distribute.

cgf

Re: The GNU CVS Server: subversions

[Jeffrey A Law]

  In message < 20000313125125.A1933@cygnus.com >you write:
  > >Please do -- we've been meaning to start providing anoncvs mirrors for som
  > e
  > >time, but just haven't gotten around to it.
  > >
  > >Let us know when it's up and we'll note the existence of the CVS mirrors o
  > n
  > >our web pages.
  > 
  > Has cvs access been a problem on sourceware, bandwidth-wise?
anoncvs is the single biggest IO & cpu consumer we've got and was the main
reason the old server was so slow.  I haven't seen any kind of bandwidth
reports on it yet, but I suspect it's a significant component of the network
usage on the sourceware T1.

The long term plan has always been to set up some anoncvs mirrors, then make
the main box available only to folks with r/w access.  In fact, the old 
sourceware machine was supposed to be the first mirror, but neither Jason
nor I have had the chance to set it up yet.

Right now we've got cpu/io to burn, but that won't always be the case as 
projects continue to grow and new projects appear on sourceware.  Obviously, 
anoncvs mirrors only help network bandwidth when we have off-site anoncvs
servers :-)

jeff

Re: The GNU CVS Server: subversions

[Jeff Bailey]

On Mon, Mar 13, 2000 at 10:10:01AM -0700, Jeffrey A Law wrote:
>   > We are trying to position it so that anyone who 
>   > needs a snapshot of GNU code can go to a single place (or one of it's 
>   > mirrors) and find it.  I'd appreciate your help in acheiving this.
> Please do -- we've been meaning to start providing anoncvs mirrors for some
> time, but just haven't gotten around to it.
> 
> Let us know when it's up and we'll note the existence of the CVS mirrors on
> our web pages.

Great, I'm just getting ready for doing this - What's the best way to get 
a list of the modules that I should echo over?  I don't know if, for 
instance, libiberty is a separate module or even if it's GNU.

tks,
Jeff Bailey

-- 
There is no sin except stupidity.
 - Oscar Wilde

Re: The GNU CVS Server: subversions

[Jeffrey A Law]

  In message < 20000323010309.B29597@sparky.nisa.net >you write:
  > Great, I'm just getting ready for doing this - What's the best way to get 
  > a list of the modules that I should echo over?  I don't know if, for 
  > instance, libiberty is a separate module or even if it's GNU.
The two repositories you'll probably want to mirror are

sourceware.cygnus.com:/cvs/src  binutils & gdb modules
gcc.gnu.org:/cvs/gcc		everything


jeff

Re: The GNU CVS Server: subversions

[Jeff Bailey]

On Thu, Mar 23, 2000 at 10:04:04AM -0700, Jeffrey A Law wrote:

>   > Great, I'm just getting ready for doing this - What's the best way to get 
>   > a list of the modules that I should echo over?  I don't know if, for 
>   > instance, libiberty is a separate module or even if it's GNU.
> The two repositories you'll probably want to mirror are
> 
> sourceware.cygnus.com:/cvs/src  binutils & gdb modules
> gcc.gnu.org:/cvs/gcc		everything

I'm finally getting to do these (I've already got libc, and automake 
working), and I'm having some difficulties:

rsync@subversions:~> rsync --archive --delete --checksum --compress 
rsync://sourceware.cygnus.com/cvs/src /home/rsync/cygnus
@ERROR: Unknown module 'cvs'

rsync@subversions:~> rsync --archive --delete --checksum --compress 
rsync://gcc.gnu.org/cvs/gcc /home/rsync/gcc
@ERROR: Unknown module 'cvs'

Any suggestions?

Tks,
Jeff Bailey

-- 
"It is easy to be blinded to the essential uselessness of computers by
the sense of accomplishment you get from getting them to work at all."
 - Douglas Adams

Re: The GNU CVS Server: subversions

[Jason Molenda]

On Fri, Sep 01, 2000 at 03:42:02PM -0700, Jeff Bailey wrote:

> rsync@subversions:~> rsync --archive --delete --checksum --compress 
> rsync://sourceware.cygnus.com/cvs/src /home/rsync/cygnus

rsync@subversions:~> rsync --archive --delete --checksum --compress  \
                     rsync://sourceware.cygnus.com/src-cvs /home/rsync/cygnus


cf http://sources.redhat.com/sourceware/rsync.html

Re: The GNU CVS Server: subversions

[Jeffrey A Law]

  In message < 20000311062344.H24678@sparky.nisa.net >you write:
  > The solution: With your permission, I'd like to rsync over the 
  > repositories so that they are available on subversions.  I have already 
  > tested this (with tromey's permission) for automake and it appears to 
  > work.  I haven't figured out a reasonable update schedule yet that 
  > balances accuracy and bandwidth, but that doesn't have to be done yet.
  > 
  > The subversions server features anoncvs access, cvs-web, and anonymous 
  > rsync access.  For projects that are hosted there, write access is 
  > controlled per module.  We are trying to position it so that anyone who 
  > needs a snapshot of GNU code can go to a single place (or one of it's 
  > mirrors) and find it.  I'd appreciate your help in acheiving this.
Please do -- we've been meaning to start providing anoncvs mirrors for some
time, but just haven't gotten around to it.

Let us know when it's up and we'll note the existence of the CVS mirrors on
our web pages.

jeff

Re: The GNU CVS Server: subversions

[Jeffrey A Law]

  In message < 20000313125125.A1933@cygnus.com >you write:
  > >Please do -- we've been meaning to start providing anoncvs mirrors for som
  > e
  > >time, but just haven't gotten around to it.
  > >
  > >Let us know when it's up and we'll note the existence of the CVS mirrors o
  > n
  > >our web pages.
  > 
  > Has cvs access been a problem on sourceware, bandwidth-wise?
anoncvs is the single biggest IO & cpu consumer we've got and was the main
reason the old server was so slow.  I haven't seen any kind of bandwidth
reports on it yet, but I suspect it's a significant component of the network
usage on the sourceware T1.

The long term plan has always been to set up some anoncvs mirrors, then make
the main box available only to folks with r/w access.  In fact, the old 
sourceware machine was supposed to be the first mirror, but neither Jason
nor I have had the chance to set it up yet.

Right now we've got cpu/io to burn, but that won't always be the case as 
projects continue to grow and new projects appear on sourceware.  Obviously, 
anoncvs mirrors only help network bandwidth when we have off-site anoncvs
servers :-)

jeff

Re: The GNU CVS Server: subversions

[Jeff Bailey]

On Sat, Mar 11, 2000 at 11:55:21AM -0800, Jason Molenda wrote:

> Of course, I set up the rsync server on sourceware so that anyone,
> anywhere, at any time can rsync the CVS repositories on sourceware.
> Even if the overseers list reached some kind of consensus that we
> didn't like this, they'll be free to do it on their own.  I suppose
> we (at overseers) could not-advertise the FSF mirrors if we really
> didn't want them doing this.

While it's true that it's setup that I could do this without asking 
first, I figured that going out of my way to piss you guys off would be 
SOP^H^H^Hexcessively annoying behaviour.  =)

> > A repository that has been synced with the master is going to be out of
> > date most of the time if there is active development going on.  
> 
> I assume there would be a sync'ing schedule of at least once a day.
> That's not unreasonable -- it is rare that things _really_ change
> significantly in any one twenty-four hour period.  If it were to
> be done more frequently (say two or maybe even three times a day),
> it'd make sense to measure load on the sourceware system over a 24
> hour period for a week (maybe counting the number of concurrent
> anoncvs users via ps aux greppery, which is a good measure of how
> busy the cvs disk is) and do the mirroring at a time when the system
> is less busy.

Absolutely: we could arrange to sync just before the busy period to give 
the best snapshots, or whatever is appropriate.

> It's up to us how to document and advertise the existance of a cvs
> mirror.  When there is only one, I admit it would complicate the
> instructions.  If we ever pick up more cvs mirrors, then we can
> have nicely generalized instructions with a pointer to a list of
> mirrors for folks who want something closer to home.

I don't know if it's reasonable to assume that someone somewhere will 
choose to mirror from subversions.  Those instructions would be useful 
for that as well.

> On the FSF web pages, they'll undoubtedly want to provide instructions
> for pserver access to (say) automake that refer to the repo on
> subversions.  I think that's just fine.

The maintainers of the packages are welcome to refer to whichever 
repository they feel happiest.  The higher-volume packages may want to 
explicetly refer people to or away from the master to suit their needs 
(bandwidth vs. convergence times)

> Incidentally, I assume that the subversions folks are only interested
> in mirroring GNU stuff like automake, binutils, or GDB.

Correct.  My example using cygwin was probably misleading.  Sorry.

-- 
Do not meddle in the affairs of cyborgs for you are conductive and
can support 110 volts.

Re: The GNU CVS Server: subversions

[Jason Molenda]

On Sat, Mar 11, 2000 at 10:34:46AM -0500, Chris Faylor wrote:

> I don't understand the goal of periodically transferring a CVS
> repository to another server.  

I like for a few reasons:

  (a) it distributes anoncvs load
  (b) it gives better network fault tolerance for when the Internet wobbles
  (c) it means more copies of the repo are in existance, which means
      I sleep better if the Cygnus machine room goes up in flames

Of course, I set up the rsync server on sourceware so that anyone,
anywhere, at any time can rsync the CVS repositories on sourceware.
Even if the overseers list reached some kind of consensus that we
didn't like this, they'll be free to do it on their own.  I suppose
we (at overseers) could not-advertise the FSF mirrors if we really
didn't want them doing this.

The FreeBSD project distributes their CVS load widely via cvsup;
I think it is a good idea.

People with cvs write access would still need to come to sourceware,
of course.

> A repository that has been synced with the master is going to be out of
> date most of the time if there is active development going on.  

I assume there would be a sync'ing schedule of at least once a day.
That's not unreasonable -- it is rare that things _really_ change
significantly in any one twenty-four hour period.  If it were to
be done more frequently (say two or maybe even three times a day),
it'd make sense to measure load on the sourceware system over a 24
hour period for a week (maybe counting the number of concurrent
anoncvs users via ps aux greppery, which is a good measure of how
busy the cvs disk is) and do the mirroring at a time when the system
is less busy.

> I am
> not sure that I want to handle the inevitable user questions and
> confusion that would result from this.

It's up to us how to document and advertise the existance of a cvs
mirror.  When there is only one, I admit it would complicate the
instructions.  If we ever pick up more cvs mirrors, then we can
have nicely generalized instructions with a pointer to a list of
mirrors for folks who want something closer to home.

On the FSF web pages, they'll undoubtedly want to provide instructions
for pserver access to (say) automake that refer to the repo on
subversions.  I think that's just fine.

Incidentally, I assume that the subversions folks are only interested
in mirroring GNU stuff like automake, binutils, or GDB.

Well, anyway, MHO

Jason

Re: The GNU CVS Server: subversions

[Jason Molenda]

On Fri, Sep 01, 2000 at 03:42:02PM -0700, Jeff Bailey wrote:

> rsync@subversions:~> rsync --archive --delete --checksum --compress 
> rsync://sourceware.cygnus.com/cvs/src /home/rsync/cygnus

rsync@subversions:~> rsync --archive --delete --checksum --compress  \
                     rsync://sourceware.cygnus.com/src-cvs /home/rsync/cygnus


cf http://sources.redhat.com/sourceware/rsync.html

Re: The GNU CVS Server: subversions

[Jeff Bailey]

On Thu, Mar 23, 2000 at 10:04:04AM -0700, Jeffrey A Law wrote:

>   > Great, I'm just getting ready for doing this - What's the best way to get 
>   > a list of the modules that I should echo over?  I don't know if, for 
>   > instance, libiberty is a separate module or even if it's GNU.
> The two repositories you'll probably want to mirror are
> 
> sourceware.cygnus.com:/cvs/src  binutils & gdb modules
> gcc.gnu.org:/cvs/gcc		everything

I'm finally getting to do these (I've already got libc, and automake 
working), and I'm having some difficulties:

rsync@subversions:~> rsync --archive --delete --checksum --compress 
rsync://sourceware.cygnus.com/cvs/src /home/rsync/cygnus
@ERROR: Unknown module 'cvs'

rsync@subversions:~> rsync --archive --delete --checksum --compress 
rsync://gcc.gnu.org/cvs/gcc /home/rsync/gcc
@ERROR: Unknown module 'cvs'

Any suggestions?

Tks,
Jeff Bailey

-- 
"It is easy to be blinded to the essential uselessness of computers by
the sense of accomplishment you get from getting them to work at all."
 - Douglas Adams

Re: The GNU CVS Server: subversions

[Chris Faylor]

On Mon, Mar 13, 2000 at 10:10:01AM -0700, Jeffrey A Law wrote:
>
>  In message < 20000311062344.H24678@sparky.nisa.net >you write:
>  > The solution: With your permission, I'd like to rsync over the 
>  > repositories so that they are available on subversions.  I have already 
>  > tested this (with tromey's permission) for automake and it appears to 
>  > work.  I haven't figured out a reasonable update schedule yet that 
>  > balances accuracy and bandwidth, but that doesn't have to be done yet.
>  > 
>  > The subversions server features anoncvs access, cvs-web, and anonymous 
>  > rsync access.  For projects that are hosted there, write access is 
>  > controlled per module.  We are trying to position it so that anyone who 
>  > needs a snapshot of GNU code can go to a single place (or one of it's 
>  > mirrors) and find it.  I'd appreciate your help in acheiving this.
>Please do -- we've been meaning to start providing anoncvs mirrors for some
>time, but just haven't gotten around to it.
>
>Let us know when it's up and we'll note the existence of the CVS mirrors on
>our web pages.

Has cvs access been a problem on sourceware, bandwidth-wise?

Just curious.

cgf

Re: The GNU CVS Server: subversions

[Jeffrey A Law]

  In message < 20000311115236.A15400@sparky.nisa.net >you write:
  > > A repository that has been synced with the master is going to be out of
  > > date most of the time if there is active development going on.  I am
  > > not sure that I want to handle the inevitable user questions and
  > > confusion that would result from this.
  > 
  > This is why I wanted to discuss update frequencies.  With some effort, I 
  > might be able to add monitor the commit lists and have it rsync after 
  > each one - I don't know if that's good or not.  Otherwise, I was thinking 
  > of every 6 hours.
The thing to remember is we can (and likely will) adjust it over time as
necessary.  Pick whatever you think is reasonable and we'll fine tune it
as we get some experience with this stuff.

Thanks,
jeff

Re: The GNU CVS Server: subversions

[Jeff Bailey]

On Mon, Mar 13, 2000 at 10:10:01AM -0700, Jeffrey A Law wrote:
>   > We are trying to position it so that anyone who 
>   > needs a snapshot of GNU code can go to a single place (or one of it's 
>   > mirrors) and find it.  I'd appreciate your help in acheiving this.
> Please do -- we've been meaning to start providing anoncvs mirrors for some
> time, but just haven't gotten around to it.
> 
> Let us know when it's up and we'll note the existence of the CVS mirrors on
> our web pages.

Great, I'm just getting ready for doing this - What's the best way to get 
a list of the modules that I should echo over?  I don't know if, for 
instance, libiberty is a separate module or even if it's GNU.

tks,
Jeff Bailey

-- 
There is no sin except stupidity.
 - Oscar Wilde

Re: The GNU CVS Server: subversions

[Jeff Bailey]

On Sat, Mar 11, 2000 at 10:34:46AM -0500, Chris Faylor wrote:

> I don't understand the goal of periodically transferring a CVS
> repository to another server.  If I was interested in contributing to a
> project I would use the main repository.  This also applies to reading
> from a repository.  One of the reasons (the only reason?) people use CVS
> repositories is to ensure that they are up to date with the main
> development source.

The other reason that people checkout sources is to get access to code 
that has a longer release cycle.  For example, I frequently take 
snapshots of glibc, because the release cycles are too infrequent to meet 
my needs for the Hurd.  I have also done the same with cygwin, not 
because I wanted bleeding edge sources (I'd download winsup for that), 
but because I wanted to use `cvs diff'.

This is intended to be a read-only mirror, and people contributing to the
source should use the master repository.  Having everything on the
subversions repository is intended to serve casual developers (and
observers) to whom it's not critical that the source be up to the minute. 

> A repository that has been synced with the master is going to be out of
> date most of the time if there is active development going on.  I am
> not sure that I want to handle the inevitable user questions and
> confusion that would result from this.

This is why I wanted to discuss update frequencies.  With some effort, I 
might be able to add monitor the commit lists and have it rsync after 
each one - I don't know if that's good or not.  Otherwise, I was thinking 
of every 6 hours.

-- 
Do not meddle in the affairs of cyborgs for you are conductive and
can support 110 volts.

Re: bandwidth usage

[Chris Faylor]

On Mon, Mar 13, 2000 at 12:56:08PM -0500, Jim Kingdon wrote:
>> Has cvs access been a problem on sourceware, bandwidth-wise?
>
>The conventional wisdom is that FTP downloads dominate other users of
>bandwidth.  And a quick check reveals this seem to be right:
>
>    [kingdon@sourceware kingdon]$ ps auxww | grep ftp | wc -l
>	 29
>    [kingdon@sourceware kingdon]$ ps auxww | grep cvs | wc -l
>	  2
>    [kingdon@sourceware kingdon]$ 
>
>However, we don't have any particularly good way of measuring this
>(CVS doesn't keep any useful logs, and if the OS has any way of doing
>so we aren't using it).

I would think that CVS would be a fraction of FTP, at least once the
repository is checked out.

On reflection, I guess it really doesn't matter if CVS is a bandwidth
hog.  As long as it is sharing a channel with a theorized resource eater
like ftp downloads, it makes sense to distribute.

cgf

Re: The GNU CVS Server: subversions

[Chris Faylor]

On Sat, Mar 11, 2000 at 06:23:44AM -0800, Jeff Bailey wrote:
>Friends, Redhatters, Country(wo)men...
>
>Many of you have heard of `subversions' the new GNU CVS server.  If not, 
>basically, it's intended to be a fast, secure, and incredably well-named 
>server dedicated to hosting CVS repositories for GNU projects.  The 
>general idea is that hopefully all GNU projects will find there way over 
>there.  There are 2 problems with this:
>
>1) GNU doesn't have the greatest reputation for secure and fast servers.
>2) Y'all already have a perfectly functioning `sourceware' server.
>
>The solution: With your permission, I'd like to rsync over the 
>repositories so that they are available on subversions.  I have already 
>tested this (with tromey's permission) for automake and it appears to 
>work.  I haven't figured out a reasonable update schedule yet that 
>balances accuracy and bandwidth, but that doesn't have to be done yet.
>
>The subversions server features anoncvs access, cvs-web, and anonymous 
>rsync access.  For projects that are hosted there, write access is 
>controlled per module.  We are trying to position it so that anyone who 
>needs a snapshot of GNU code can go to a single place (or one of it's 
>mirrors) and find it.  I'd appreciate your help in acheiving this.
>
>As we used to say in debating: "I now stand ready for cross-examination" =)

I don't understand the goal of periodically transferring a CVS
repository to another server.  If I was interested in contributing to a
project I would use the main repository.  This also applies to reading
from a repository.  One of the reasons (the only reason?) people use CVS
repositories is to ensure that they are up to date with the main
development source.

A repository that has been synced with the master is going to be out of
date most of the time if there is active development going on.  I am
not sure that I want to handle the inevitable user questions and
confusion that would result from this.

cgf

bandwidth usage

[Jim Kingdon]

> Has cvs access been a problem on sourceware, bandwidth-wise?

The conventional wisdom is that FTP downloads dominate other users of
bandwidth.  And a quick check reveals this seem to be right:

    [kingdon@sourceware kingdon]$ ps auxww | grep ftp | wc -l
	 29
    [kingdon@sourceware kingdon]$ ps auxww | grep cvs | wc -l
	  2
    [kingdon@sourceware kingdon]$ 

However, we don't have any particularly good way of measuring this
(CVS doesn't keep any useful logs, and if the OS has any way of doing
so we aren't using it).

Re: The GNU CVS Server: subversions

[Jeffrey A Law]

  In message < 20000323010309.B29597@sparky.nisa.net >you write:
  > Great, I'm just getting ready for doing this - What's the best way to get 
  > a list of the modules that I should echo over?  I don't know if, for 
  > instance, libiberty is a separate module or even if it's GNU.
The two repositories you'll probably want to mirror are

sourceware.cygnus.com:/cvs/src  binutils & gdb modules
gcc.gnu.org:/cvs/gcc		everything


jeff

The GNU CVS Server: subversions

[Jeff Bailey]

Friends, Redhatters, Country(wo)men...

Many of you have heard of `subversions' the new GNU CVS server.  If not, 
basically, it's intended to be a fast, secure, and incredably well-named 
server dedicated to hosting CVS repositories for GNU projects.  The 
general idea is that hopefully all GNU projects will find there way over 
there.  There are 2 problems with this:

1) GNU doesn't have the greatest reputation for secure and fast servers.
2) Y'all already have a perfectly functioning `sourceware' server.

The solution: With your permission, I'd like to rsync over the 
repositories so that they are available on subversions.  I have already 
tested this (with tromey's permission) for automake and it appears to 
work.  I haven't figured out a reasonable update schedule yet that 
balances accuracy and bandwidth, but that doesn't have to be done yet.

The subversions server features anoncvs access, cvs-web, and anonymous 
rsync access.  For projects that are hosted there, write access is 
controlled per module.  We are trying to position it so that anyone who 
needs a snapshot of GNU code can go to a single place (or one of it's 
mirrors) and find it.  I'd appreciate your help in acheiving this.

As we used to say in debating: "I now stand ready for cross-examination" =)

-- 
Do not meddle in the affairs of cyborgs for you are conductive and
can support 110 volts.