From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jason Molenda To: Jim Kingdon Cc: overseers@sourceware.cygnus.com Subject: Re: ftp mirrors Date: Sat, 30 Dec 2000 06:08:00 -0000 Message-id: <20000611150823.A9895@shell17.ba.best.com> References: <200006091545.LAA00938.cygnus.project.sourcemaster@envy.delorie.com> <3941CC03.B74B3373@cygnus.com> <394223B7.A1A849E9@cygnus.com> <3942E532.5D17574@cygnus.com> <20000610235503.A8352@shell17.ba.best.com> <200006111524.LAA14449@panix6.panix.com> X-SW-Source: 2000/msg00638.html On Sun, Jun 11, 2000 at 11:24:09AM -0400, Jim Kingdon wrote: > > You're suggesting that rsync is clever enough to see two files on > > the server, foo-2000-06-09 and foo-2000-06-10, see that the client > > already has foo-2000-06-09, and make the leap that the -09 and -10 > > files are probably pretty close to each other, so do a diff between > > the server's -09 and -10 and send that diff? > > If rsync just concatenates the files or something equivalent it should > be able to get this result without a special case (at least some of > the time). See > > http://rsync.samba.org/rsync/tech_report/ > > especially the section on "checksum searching". Of course it would be > good if someone would actually test it to verify this happens in real > world cases like snapshots. rsync doesn't concatenate files like that AFAIK - the whole discussion about checksum search is only relevant to when a file that exists on both the server and the client has changed and rsync wants to find the block that change. Most importantly, all of this is irrelevant to Andrew's reported problem. Andrew says that a developer complained that they had to download full tarballs every day while following a release process. I am at a loss to guess why Andrew made a leap from that problem to the solution of having a copy of the ftp dir with uncompressed versions of the tarballs, and offering rsync access to that. I'm unable to understand what (a) this has to do with the problem Andrew described (was the developer actually rsync'ing the gdb ftp dir? I don't think I've ever seen anyone download snapshots via rsync) (b) why the developer shouldn't just download diffs, and finally, (c) what benefit, exactly, is gained by having uncompressed tarballs with rsync. Maybe Andrew is just throwing out non-sequitors to frustrate me or yank on my chain - I'm at a loss of what the point of this discussion is. God forbid an ftp mirror site mirrored the uncompressed versions of the ftp directory; the communication between the mirror site and sourceware would be compressed, but all of the files in the mirror site's ftp dir would be uncompressed, just like the uncompressed dir on sourceware. It'd be a huge disaster. Jason From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jason Molenda To: Jim Kingdon Cc: overseers@sourceware.cygnus.com Subject: Re: ftp mirrors Date: Sun, 11 Jun 2000 15:08:00 -0000 Message-ID: <20000611150823.A9895@shell17.ba.best.com> References: <200006091545.LAA00938.cygnus.project.sourcemaster@envy.delorie.com> <3941CC03.B74B3373@cygnus.com> <394223B7.A1A849E9@cygnus.com> <3942E532.5D17574@cygnus.com> <20000610235503.A8352@shell17.ba.best.com> <200006111524.LAA14449@panix6.panix.com> X-SW-Source: 2000-q2/msg00331.html Message-ID: <20000611150800._oUKjKV8yRKWYIn2VSRf-I6ZHtzd0QGvPBDiVS1PtvQ@z> On Sun, Jun 11, 2000 at 11:24:09AM -0400, Jim Kingdon wrote: > > You're suggesting that rsync is clever enough to see two files on > > the server, foo-2000-06-09 and foo-2000-06-10, see that the client > > already has foo-2000-06-09, and make the leap that the -09 and -10 > > files are probably pretty close to each other, so do a diff between > > the server's -09 and -10 and send that diff? > > If rsync just concatenates the files or something equivalent it should > be able to get this result without a special case (at least some of > the time). See > > http://rsync.samba.org/rsync/tech_report/ > > especially the section on "checksum searching". Of course it would be > good if someone would actually test it to verify this happens in real > world cases like snapshots. rsync doesn't concatenate files like that AFAIK - the whole discussion about checksum search is only relevant to when a file that exists on both the server and the client has changed and rsync wants to find the block that change. Most importantly, all of this is irrelevant to Andrew's reported problem. Andrew says that a developer complained that they had to download full tarballs every day while following a release process. I am at a loss to guess why Andrew made a leap from that problem to the solution of having a copy of the ftp dir with uncompressed versions of the tarballs, and offering rsync access to that. I'm unable to understand what (a) this has to do with the problem Andrew described (was the developer actually rsync'ing the gdb ftp dir? I don't think I've ever seen anyone download snapshots via rsync) (b) why the developer shouldn't just download diffs, and finally, (c) what benefit, exactly, is gained by having uncompressed tarballs with rsync. Maybe Andrew is just throwing out non-sequitors to frustrate me or yank on my chain - I'm at a loss of what the point of this discussion is. God forbid an ftp mirror site mirrored the uncompressed versions of the ftp directory; the communication between the mirror site and sourceware would be compressed, but all of the files in the mirror site's ftp dir would be uncompressed, just like the uncompressed dir on sourceware. It'd be a huge disaster. Jason