* splitting sourceware into pieces? @ 2001-02-21 11:10 Phil Edwards 2001-02-21 12:51 ` Andrew Cagney ` (2 more replies) 0 siblings, 3 replies; 10+ messages in thread From: Phil Edwards @ 2001-02-21 11:10 UTC (permalink / raw) To: overseers Not the physical machine, but rather its functionality. (Y'all can resume breathing now. :-) During the middle of the day, mail seems to be taking upwards of half an hour to get processed by the lists; CVS connections can be dog slow for anyone who has less than an OC-3, etc, etc, you all have heard this before. How about delgating email to one machine and everything else to another? Or maybe FTP and web traffic to one machine, and everything else to another. You get the picture; it doesn't need to all be on one system. Maybe this is a completely pointless idea considering the upcoming co-loc. Phil -- pedwards at disaster dot jaj dot com | pme at sources dot redhat dot com devphil at several other less interesting addresses in various dot domains The gods do not protect fools. Fools are protected by more capable fools. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: splitting sourceware into pieces? 2001-02-21 11:10 splitting sourceware into pieces? Phil Edwards @ 2001-02-21 12:51 ` Andrew Cagney 2001-02-21 14:28 ` Jason Molenda 2001-02-21 13:06 ` Jason Molenda 2001-02-21 14:31 ` Jeffrey A Law 2 siblings, 1 reply; 10+ messages in thread From: Andrew Cagney @ 2001-02-21 12:51 UTC (permalink / raw) To: Phil Edwards; +Cc: overseers Phil Edwards wrote: > > Not the physical machine, but rather its functionality. (Y'all can resume > breathing now. :-) Er, you mean functionality dedicated to specific physical machines. > During the middle of the day, mail seems to be taking upwards of half an > hour to get processed by the lists; CVS connections can be dog slow for > anyone who has less than an OC-3, etc, etc, you all have heard this before. > How about delgating email to one machine and everything else to another? > Or maybe FTP and web traffic to one machine, and everything else to another. > You get the picture; it doesn't need to all be on one system. While I might have a strong preference for one function, one physical machine (if someone hoses mail, they don't hose CVS, or my cron jobs don't hose web functionality ...) I do feel like I'm just doing a ``me too''. I don't see my self being able to contribute to this. I suspect Jason would have the most imput. Andrew ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: splitting sourceware into pieces? 2001-02-21 12:51 ` Andrew Cagney @ 2001-02-21 14:28 ` Jason Molenda 2001-02-21 14:41 ` Phil Edwards 0 siblings, 1 reply; 10+ messages in thread From: Jason Molenda @ 2001-02-21 14:28 UTC (permalink / raw) To: Andrew Cagney; +Cc: Phil Edwards, overseers On Wed, Feb 21, 2001 at 03:48:57PM -0500, Andrew Cagney wrote: > > While I might have a strong preference for one function, one physical > machine (if someone hoses mail, they don't hose CVS, or my cron jobs > don't hose web functionality ...) I do feel like I'm just doing a ``me > too''. I don't see my self being able to contribute to this. > > I suspect Jason would have the most input. Well, I have opinions anyway. :-) My opinions are just that - the current maintainers should (naturally) feel free to do as they see fit. I'm serious about the additional features we're able to do by having everything on one host - it creeps into lots of places. Tom's modifications log_accum which will notice a mention of a PR in a cvs commit msg (e.g. "Fixes gdb/30") and cause the commit notice to be appended to the PR if it exists. That'd be tough to do without the cvs server and the gnats database on the same host. I mentioned the mailing list mbox archives being put in the ftp server - a little bit harder to do if the mail server and the ftp server are separate. Or the link on the mailing list archive index pages which says like March 2000 [<a href="">click here for mbox bzip2'ed version (408k)</a>] Feb 2000 [<a href="">click here for mbox bzip2'ed version (350k)</a>] That'd be tougher to do if the web server and the ftp server weren't on the same host. Web archiving of the mail notes is a case where having the mail server and the web server on the same box simplifies things. The project web pages in cvs is easy because the web server is on the same box as the cvs server -- if the gdb home page was maintained on one system but served from another, you've got some additional stuff to work out to make it all happen. It probably wouldn't be as real-time-slick as the current setup. These are just off the top of my head. I love the compartmentalization style of security as much as the next person, but you gain a lot if you're willing to give that up. Jason PS- Another thing which could be considered -- ezmlm has facilities for multiple hosts distributing outgoing mail notes. Y'all could saturate the Cygnus T1 in addition to the Sourceware T1 is you put an ezmlm slave server on that side of the net and foisted some of the deliveries off to that. I've never looked in depth as this side of ezmlm's capabilities, but I know they exist. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: splitting sourceware into pieces? 2001-02-21 14:28 ` Jason Molenda @ 2001-02-21 14:41 ` Phil Edwards 2001-02-21 15:27 ` Jason Molenda 0 siblings, 1 reply; 10+ messages in thread From: Phil Edwards @ 2001-02-21 14:41 UTC (permalink / raw) To: Jason Molenda; +Cc: overseers On Wed, Feb 21, 2001 at 02:28:23PM -0800, Jason Molenda wrote: > Tom's > modifications log_accum which will notice a mention of a PR in a > cvs commit msg (e.g. "Fixes gdb/30") and cause the commit notice > to be appended to the PR if it exists. *boggle* I never knew that. Wow. Most cool. Thanks, Tom! I've seen people writing those notes but just thought it was "only" good style, not actually feeding into GNATS. Phil -- pedwards at disaster dot jaj dot com | pme at sources dot redhat dot com devphil at several other less interesting addresses in various dot domains The gods do not protect fools. Fools are protected by more capable fools. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: splitting sourceware into pieces? 2001-02-21 14:41 ` Phil Edwards @ 2001-02-21 15:27 ` Jason Molenda 0 siblings, 0 replies; 10+ messages in thread From: Jason Molenda @ 2001-02-21 15:27 UTC (permalink / raw) To: Phil Edwards; +Cc: overseers On Wed, Feb 21, 2001 at 05:53:23PM -0500, Phil Edwards wrote: > On Wed, Feb 21, 2001 at 02:28:23PM -0800, Jason Molenda wrote: > > Tom's > > modifications log_accum which will notice a mention of a PR in a > > cvs commit msg (e.g. "Fixes gdb/30") and cause the commit notice > > to be appended to the PR if it exists. > > *boggle* > > I never knew that. Wow. Most cool. Thanks, Tom! > > I've seen people writing those notes but just thought it was "only" good > style, not actually feeding into GNATS. I think you need to add the -G command line opt to your log_accum command line in CVSROOT/loginfo for it to be enabled. cf http://sourceware.cygnus.com/cgi-bin/cvsweb.cgi/CVSROOT/loginfo?rev=1.7&content-type=text/x-cvsweb-markup&cvsroot=java The -C cmd line option adds URLs to the bottom of cvs commit messages which will show you the changes for that commit (except for newly added files or just-deleted files). Tom added that one too. In fact, together these options make a nice cheap integration of the bug tracking and the revision control systems. You commit a change to foo.c which fixes gcc/35, the cvs commit message is appended to gcc/35 along with the URLs to get the patch via cvsweb. J ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: splitting sourceware into pieces? 2001-02-21 11:10 splitting sourceware into pieces? Phil Edwards 2001-02-21 12:51 ` Andrew Cagney @ 2001-02-21 13:06 ` Jason Molenda 2001-02-21 13:16 ` Jason Molenda ` (2 more replies) 2001-02-21 14:31 ` Jeffrey A Law 2 siblings, 3 replies; 10+ messages in thread From: Jason Molenda @ 2001-02-21 13:06 UTC (permalink / raw) To: Phil Edwards; +Cc: overseers On Wed, Feb 21, 2001 at 02:21:47PM -0500, Phil Edwards wrote: > During the middle of the day, mail seems to be taking upwards of half an > hour to get processed by the lists; CVS connections can be dog slow for > anyone who has less than an OC-3, etc, etc, you all have heard this before. I haven't monitored the system's performance in a long time, but I would suspect (read - I'd be surprised if I'm wrong) that the bottleneck is entirely in the T1 to the system. The oft-mentioned co location would address this problem. I gather a new system would be constructed at that time as well. The disks won't get much faster (drive technology hasn't advanced much in the past two years, and I'd put in the fastest ~$1k drives I could find back then), but the CPU will be much faster (it has a 450MHz P-III right now). I suppose memory might get a bit bigger, but it has 768MB right now. I kind of doubt we're CPU starved very often, though. If the network is the bottleneck, then separating the functionality across multiple hosts doesn't gain much. If there was a bonanza of extra CPU, a few more tricks could be used, like maybe using a loadable apache module which would gzip _all_ the content on the site on the fly. Right now there is a module which will send gzipped versions of content if they exist, but they have to be pre-generated, and it can't interoperate with server side includes. This change would decrease the response time for people browsing the web archives of mailing lists by quite a bit I think. > You get the picture; it doesn't need to all be on one system. It doesn't need to be, but it makes a lot of things a whole lot easier when it is. On the good side, all the services on sourceware are tightly integrated so we can have mailing list archives right in the ftp dir with no problems. On the bad side, it makes maintenance a lot harder because anyone wanting to change things has to figure out how all the hair sticks together. If it were necessary to stick to the T1 and people were unhappy with system performance, I'd attack the ftp traffic side first. It's something like half the traffic, and most of it is gzipped (not even bzip2'ed!!) gcc snapshots and the like. I'd at least get these compressed in .bz2 instead of .gz, and I'd probably make the snapshots etc available only to mirror sites and require users to get them from the mirrors. I haven't looked over the ftp logs (/www/logs/ftp*), but I'm sure some examination of what content is being downloaded the most would suggest some obvious alternatives for reducing the load on the T1. Jason ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: splitting sourceware into pieces? 2001-02-21 13:06 ` Jason Molenda @ 2001-02-21 13:16 ` Jason Molenda 2001-02-21 13:22 ` Phil Edwards 2001-02-26 8:43 ` Jeffrey A Law 2 siblings, 0 replies; 10+ messages in thread From: Jason Molenda @ 2001-02-21 13:16 UTC (permalink / raw) To: Phil Edwards; +Cc: overseers On Wed, Feb 21, 2001 at 01:06:10PM -0800, Jason Molenda wrote: > > I kind of doubt we're CPU starved very often, though. > I just logged into the system, and there actually is about 0% idle on the CPU with just a few processes executing concurrently. I still don't think this is where the real bottleneck is, but it does mean that getting a second CPU or upgrading to a faster CPU could help things a bit. Jason ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: splitting sourceware into pieces? 2001-02-21 13:06 ` Jason Molenda 2001-02-21 13:16 ` Jason Molenda @ 2001-02-21 13:22 ` Phil Edwards 2001-02-26 8:43 ` Jeffrey A Law 2 siblings, 0 replies; 10+ messages in thread From: Phil Edwards @ 2001-02-21 13:22 UTC (permalink / raw) To: Jason Molenda; +Cc: overseers On Wed, Feb 21, 2001 at 01:06:10PM -0800, Jason Molenda wrote: > If the network is the bottleneck, then separating the functionality > across multiple hosts doesn't gain much. True, of course. I had a list of "things to mention in my note to overseers," and the idea of splitting the tasks up amongst multiple boxes would only help if they aren't sharing the same piece of cable, the same hub, etc. Guess which of the items didn't get a check mark after I wrote the message. > If it were necessary to stick to the T1 and people were unhappy > with system performance, I'd attack the ftp traffic side first. [...] > and I'd probably make > the snapshots etc available only to mirror sites and require users > to get them from the mirrors. That sounds like a good idea even after the box is moved to the co-loc. Phil -- pedwards at disaster dot jaj dot com | pme at sources dot redhat dot com devphil at several other less interesting addresses in various dot domains The gods do not protect fools. Fools are protected by more capable fools. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: splitting sourceware into pieces? 2001-02-21 13:06 ` Jason Molenda 2001-02-21 13:16 ` Jason Molenda 2001-02-21 13:22 ` Phil Edwards @ 2001-02-26 8:43 ` Jeffrey A Law 2 siblings, 0 replies; 10+ messages in thread From: Jeffrey A Law @ 2001-02-26 8:43 UTC (permalink / raw) To: Jason Molenda; +Cc: Phil Edwards, overseers In message < 20010221130610.A11627@shell17.ba.best.com >you write: > I haven't monitored the system's performance in a long time, but > I would suspect (read - I'd be surprised if I'm wrong) that the > bottleneck is entirely in the T1 to the system. Most likely. Though I have seen CPU utilization increase a lot over the last year. More CPU power isn't going to hurt :-) > The oft-mentioned co location would address this problem. Yes. > I gather a new system would be constructed at that time as well. Yes. We need a rack-mountable box for the coloc. I've already got the new box roughly spec'd out. Figure it's 2X bigger in just about every category except physical size :-) jeff ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: splitting sourceware into pieces? 2001-02-21 11:10 splitting sourceware into pieces? Phil Edwards 2001-02-21 12:51 ` Andrew Cagney 2001-02-21 13:06 ` Jason Molenda @ 2001-02-21 14:31 ` Jeffrey A Law 2 siblings, 0 replies; 10+ messages in thread From: Jeffrey A Law @ 2001-02-21 14:31 UTC (permalink / raw) To: Phil Edwards; +Cc: overseers In message < 20010221142147.A20931@disaster.jaj.com >you write: > Not the physical machine, but rather its functionality. (Y'all can resume > breathing now. :-) > > During the middle of the day, mail seems to be taking upwards of half an > hour to get processed by the lists; CVS connections can be dog slow for > anyone who has less than an OC-3, etc, etc, you all have heard this before. > How about delgating email to one machine and everything else to another? > Or maybe FTP and web traffic to one machine, and everything else to another > . > You get the picture; it doesn't need to all be on one system. > > Maybe this is a completely pointless idea considering the upcoming co-loc. We're mostly concerned with getting ready for moving to the co-loc; as part of that move we're purchasing a new machine, roughly 2X the performance of the current box which should help a lot. jeff ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2001-02-26 8:43 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2001-02-21 11:10 splitting sourceware into pieces? Phil Edwards 2001-02-21 12:51 ` Andrew Cagney 2001-02-21 14:28 ` Jason Molenda 2001-02-21 14:41 ` Phil Edwards 2001-02-21 15:27 ` Jason Molenda 2001-02-21 13:06 ` Jason Molenda 2001-02-21 13:16 ` Jason Molenda 2001-02-21 13:22 ` Phil Edwards 2001-02-26 8:43 ` Jeffrey A Law 2001-02-21 14:31 ` Jeffrey A Law
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).