* Re: www out of space [not found] <5F1A0D7C-8A6A-11D8-A5BE-000A95DA505C@dberlin.org> @ 2004-04-09 21:24 ` Christopher Faylor 2004-04-09 21:35 ` Christopher Faylor 0 siblings, 1 reply; 11+ messages in thread From: Christopher Faylor @ 2004-04-09 21:24 UTC (permalink / raw) To: Daniel Berlin; +Cc: overseers On Fri, Apr 09, 2004 at 05:10:47PM -0400, Daniel Berlin wrote: >/dev/vg1/lvol6 23980436 22742628 0 100% /sourceware/www ARGH. It had a little less than a gigabyte yesterday. I wonder if the fsck reset the free space. cgf ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: www out of space 2004-04-09 21:24 ` www out of space Christopher Faylor @ 2004-04-09 21:35 ` Christopher Faylor 2004-04-09 22:22 ` Daniel Berlin ` (2 more replies) 0 siblings, 3 replies; 11+ messages in thread From: Christopher Faylor @ 2004-04-09 21:35 UTC (permalink / raw) To: Daniel Berlin, overseers On Fri, Apr 09, 2004 at 05:17:20PM -0400, Christopher Faylor wrote: >On Fri, Apr 09, 2004 at 05:10:47PM -0400, Daniel Berlin wrote: >>/dev/vg1/lvol6 23980436 22742628 0 100% /sourceware/www > >ARGH. It had a little less than a gigabyte yesterday. > >I wonder if the fsck reset the free space. Nope. It looks like something gobbled up 700MB in 21 hours. cgf ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: www out of space 2004-04-09 21:35 ` Christopher Faylor @ 2004-04-09 22:22 ` Daniel Berlin 2004-04-10 4:01 ` Christopher Faylor 2004-04-09 22:26 ` Christopher Faylor 2004-04-10 1:48 ` Angela Marie Thomas 2 siblings, 1 reply; 11+ messages in thread From: Daniel Berlin @ 2004-04-09 22:22 UTC (permalink / raw) To: Christopher Faylor; +Cc: overseers On Apr 9, 2004, at 5:24 PM, Christopher Faylor wrote: > On Fri, Apr 09, 2004 at 05:17:20PM -0400, Christopher Faylor wrote: >> On Fri, Apr 09, 2004 at 05:10:47PM -0400, Daniel Berlin wrote: >>> /dev/vg1/lvol6 23980436 22742628 0 100% >>> /sourceware/www >> >> ARGH. It had a little less than a gigabyte yesterday. >> >> I wonder if the fsck reset the free space. > > Nope. It looks like something gobbled up 700MB in 21 hours. > It wasn't bugzilla, i checked for that :) Logs is 1.1 gig. Is that normal? > cgf ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: www out of space 2004-04-09 22:22 ` Daniel Berlin @ 2004-04-10 4:01 ` Christopher Faylor 2004-04-09 22:22 ` Daniel Berlin 2004-04-10 1:48 ` Christopher Faylor 0 siblings, 2 replies; 11+ messages in thread From: Christopher Faylor @ 2004-04-10 4:01 UTC (permalink / raw) To: Daniel Berlin; +Cc: overseers On Fri, Apr 09, 2004 at 05:35:07PM -0400, Daniel Berlin wrote: > >On Apr 9, 2004, at 5:24 PM, Christopher Faylor wrote: > >>On Fri, Apr 09, 2004 at 05:17:20PM -0400, Christopher Faylor wrote: >>>On Fri, Apr 09, 2004 at 05:10:47PM -0400, Daniel Berlin wrote: >>>>/dev/vg1/lvol6 23980436 22742628 0 100% >>>>/sourceware/www >>> >>>ARGH. It had a little less than a gigabyte yesterday. >>> >>>I wonder if the fsck reset the free space. >> >>Nope. It looks like something gobbled up 700MB in 21 hours. > >It wasn't bugzilla, i checked for that :) >Logs is 1.1 gig. Is that normal? Last weeks logs were ~100MB compressed, so I'd say no, it isn't normal. cgf ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: www out of space 2004-04-10 4:01 ` Christopher Faylor @ 2004-04-09 22:22 ` Daniel Berlin 2004-04-10 1:48 ` Christopher Faylor 1 sibling, 0 replies; 11+ messages in thread From: Daniel Berlin @ 2004-04-09 22:22 UTC (permalink / raw) To: Christopher Faylor; +Cc: overseers On Apr 9, 2004, at 5:42 PM, Christopher Faylor wrote: > On Fri, Apr 09, 2004 at 05:35:07PM -0400, Daniel Berlin wrote: >> >> On Apr 9, 2004, at 5:24 PM, Christopher Faylor wrote: >> >>> On Fri, Apr 09, 2004 at 05:17:20PM -0400, Christopher Faylor wrote: >>>> On Fri, Apr 09, 2004 at 05:10:47PM -0400, Daniel Berlin wrote: >>>>> /dev/vg1/lvol6 23980436 22742628 0 100% >>>>> /sourceware/www >>>> >>>> ARGH. It had a little less than a gigabyte yesterday. >>>> >>>> I wonder if the fsck reset the free space. >>> >>> Nope. It looks like something gobbled up 700MB in 21 hours. >> >> It wasn't bugzilla, i checked for that :) >> Logs is 1.1 gig. Is that normal? > > Last weeks logs were ~100MB compressed, so I'd say no, it isn't normal. > it seemed a bit large. Are you rotating based on date or size? -rw-r--r-- 1 apache apache 352521027 Apr 9 21:47 gcc-combined_log -rw-r--r-- 1 apache apache 26661181 Apr 9 21:47 ecos-combined_log -rw-r--r-- 1 apache apache 456554238 Apr 9 21:47 cygwin-combined_log -rw-r--r-- 1 apache apache 324403715 Apr 9 21:47 sourceware-combined_log > cgf ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: www out of space 2004-04-10 4:01 ` Christopher Faylor 2004-04-09 22:22 ` Daniel Berlin @ 2004-04-10 1:48 ` Christopher Faylor 1 sibling, 0 replies; 11+ messages in thread From: Christopher Faylor @ 2004-04-10 1:48 UTC (permalink / raw) To: Daniel Berlin, overseers On Fri, Apr 09, 2004 at 05:42:52PM -0400, Christopher Faylor wrote: >On Fri, Apr 09, 2004 at 05:35:07PM -0400, Daniel Berlin wrote: >> >>On Apr 9, 2004, at 5:24 PM, Christopher Faylor wrote: >> >>>On Fri, Apr 09, 2004 at 05:17:20PM -0400, Christopher Faylor wrote: >>>>On Fri, Apr 09, 2004 at 05:10:47PM -0400, Daniel Berlin wrote: >>>>>/dev/vg1/lvol6 23980436 22742628 0 100% >>>>>/sourceware/www >>>> >>>>ARGH. It had a little less than a gigabyte yesterday. >>>> >>>>I wonder if the fsck reset the free space. >>> >>>Nope. It looks like something gobbled up 700MB in 21 hours. >> >>It wasn't bugzilla, i checked for that :) >>Logs is 1.1 gig. Is that normal? > >Last weeks logs were ~100MB compressed, so I'd say no, it isn't normal. We were hit hard by at least to spammers scanning for email. I'm blocking them now. cgf ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: www out of space 2004-04-09 21:35 ` Christopher Faylor 2004-04-09 22:22 ` Daniel Berlin @ 2004-04-09 22:26 ` Christopher Faylor 2004-04-09 22:32 ` Daniel Berlin 2004-04-10 11:44 ` Frank Ch. Eigler 2004-04-10 1:48 ` Angela Marie Thomas 2 siblings, 2 replies; 11+ messages in thread From: Christopher Faylor @ 2004-04-09 22:26 UTC (permalink / raw) To: Daniel Berlin, overseers On Fri, Apr 09, 2004 at 05:24:53PM -0400, Christopher Faylor wrote: >On Fri, Apr 09, 2004 at 05:17:20PM -0400, Christopher Faylor wrote: >>On Fri, Apr 09, 2004 at 05:10:47PM -0400, Daniel Berlin wrote: >>>/dev/vg1/lvol6 23980436 22742628 0 100% /sourceware/www >> >>ARGH. It had a little less than a gigabyte yesterday. >> >>I wonder if the fsck reset the free space. > >Nope. It looks like something gobbled up 700MB in 21 hours. I added another 1.7G to /sourceware/www. (Sorry, Daniel, I killed two of your bash processes accessing that area since you didn't seem to be doing anything there and I needed to umount and fsck it.) I don't know if fche is still planning on doing something with /dev/sdc2 but, if not, we should probably move at least the logs directory to that partition to reduce the likelihood of this happening again. I also blocked the !@(*&( idiots who caused this to happen. I thought I'd previously blocked the robots responsible for this kind of thing at the httpd level but obviously I hadn't succeeded. Jason, are you responsible for the HTTP_USER_AGENT RewriteEngine stuff in httpd.conf? Could you see what I'm doing wrong with the "MS Search 4.0 Robot" detection? cgf ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: www out of space 2004-04-09 22:26 ` Christopher Faylor @ 2004-04-09 22:32 ` Daniel Berlin 2004-04-10 11:44 ` Frank Ch. Eigler 1 sibling, 0 replies; 11+ messages in thread From: Daniel Berlin @ 2004-04-09 22:32 UTC (permalink / raw) To: Christopher Faylor; +Cc: overseers Christopher Faylor wrote: >On Fri, Apr 09, 2004 at 05:24:53PM -0400, Christopher Faylor wrote: > > >>On Fri, Apr 09, 2004 at 05:17:20PM -0400, Christopher Faylor wrote: >> >> >>>On Fri, Apr 09, 2004 at 05:10:47PM -0400, Daniel Berlin wrote: >>> >>> >>>>/dev/vg1/lvol6 23980436 22742628 0 100% /sourceware/www >>>> >>>> >>>ARGH. It had a little less than a gigabyte yesterday. >>> >>>I wonder if the fsck reset the free space. >>> >>> >>Nope. It looks like something gobbled up 700MB in 21 hours. >> >> > >I added another 1.7G to /sourceware/www. > >(Sorry, Daniel, I killed two of your bash processes accessing that area >since you didn't seem to be doing anything there and I needed to umount >and fsck it.) > > > Oh, sorry. I thought i had logged out before i walked away. I wasn't doing anything. Sorry about that. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: www out of space 2004-04-09 22:26 ` Christopher Faylor 2004-04-09 22:32 ` Daniel Berlin @ 2004-04-10 11:44 ` Frank Ch. Eigler 2004-04-10 21:51 ` Christopher Faylor 1 sibling, 1 reply; 11+ messages in thread From: Frank Ch. Eigler @ 2004-04-10 11:44 UTC (permalink / raw) To: overseers Hi - > [...] > I don't know if fche is still planning on doing something with /dev/sdc2 > but, if not, we should probably move at least the logs directory to that > partition to reduce the likelihood of this happening again. [...] I was not going to interfere with /dev/sdc2 aka /pool until its current users (especially mysql) clear it. However, at the LVM level, there is already about 18GB more unallocated room on vg1 which could be added to the www filesystem, which can apparently be resized online. - FChE ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: www out of space 2004-04-10 11:44 ` Frank Ch. Eigler @ 2004-04-10 21:51 ` Christopher Faylor 0 siblings, 0 replies; 11+ messages in thread From: Christopher Faylor @ 2004-04-10 21:51 UTC (permalink / raw) To: Frank Ch. Eigler; +Cc: overseers On Sat, Apr 10, 2004 at 07:15:36AM -0400, Frank Ch. Eigler wrote: >>I don't know if fche is still planning on doing something with >>/dev/sdc2 but, if not, we should probably move at least the logs >>directory to that partition to reduce the likelihood of this happening >>again. [...] > >I was not going to interfere with /dev/sdc2 aka /pool until its current >users (especially mysql) clear it. However, at the LVM level, there is >already about 18GB more unallocated room on vg1 which could be added to >the www filesystem, which can apparently be resized online. Yes, I noticed this yesterday after I finished resizing the www partition. I wish I had noticed it before the resize. For some reason, I thought there was a lot less extra space left for this type of thing. cgf ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: www out of space 2004-04-09 21:35 ` Christopher Faylor 2004-04-09 22:22 ` Daniel Berlin 2004-04-09 22:26 ` Christopher Faylor @ 2004-04-10 1:48 ` Angela Marie Thomas 2 siblings, 0 replies; 11+ messages in thread From: Angela Marie Thomas @ 2004-04-10 1:48 UTC (permalink / raw) To: Christopher Faylor; +Cc: Daniel Berlin, overseers > Nope. It looks like something gobbled up 700MB in 21 hours. I don't see anything in the backup logs to show an obvious culprit. I ran an rsync -n to see what changed between the backups and nowish and also don't see anything obvious. --Angela ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2004-04-10 15:52 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <5F1A0D7C-8A6A-11D8-A5BE-000A95DA505C@dberlin.org>
2004-04-09 21:24 ` www out of space Christopher Faylor
2004-04-09 21:35 ` Christopher Faylor
2004-04-09 22:22 ` Daniel Berlin
2004-04-10 4:01 ` Christopher Faylor
2004-04-09 22:22 ` Daniel Berlin
2004-04-10 1:48 ` Christopher Faylor
2004-04-09 22:26 ` Christopher Faylor
2004-04-09 22:32 ` Daniel Berlin
2004-04-10 11:44 ` Frank Ch. Eigler
2004-04-10 21:51 ` Christopher Faylor
2004-04-10 1:48 ` Angela Marie Thomas
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).