* htdig problems?
@ 2002-10-30 13:08 Christopher Faylor
2002-11-07 7:34 ` htdig problems? (PING) Christopher Faylor
0 siblings, 1 reply; 13+ messages in thread
From: Christopher Faylor @ 2002-10-30 13:08 UTC (permalink / raw)
To: overseers
People are reporting that searches are taking a very long time and sometimes
reporting internal server errors. Is there a problem with htdig?
cgf
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: htdig problems? (PING)
2002-10-30 13:08 htdig problems? Christopher Faylor
@ 2002-11-07 7:34 ` Christopher Faylor
2002-11-07 10:00 ` Jason Molenda
2002-11-07 11:06 ` Hans-Peter Nilsson
0 siblings, 2 replies; 13+ messages in thread
From: Christopher Faylor @ 2002-11-07 7:34 UTC (permalink / raw)
To: overseers
On Wed, Oct 30, 2002 at 04:10:00PM -0500, Christopher Faylor wrote:
>People are reporting that searches are taking a very long time and sometimes
>reporting internal server errors. Is there a problem with htdig?
H-P do you have any idea why people are reporting internal server errors
when attempting to do searches? I've seen the same myself.
Someone reported that doing a search for "setup.exe" in the cygwin archives
had this effect.
cgf
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: htdig problems? (PING)
2002-11-07 7:34 ` htdig problems? (PING) Christopher Faylor
@ 2002-11-07 10:00 ` Jason Molenda
2002-11-07 11:06 ` Hans-Peter Nilsson
1 sibling, 0 replies; 13+ messages in thread
From: Jason Molenda @ 2002-11-07 10:00 UTC (permalink / raw)
To: overseers
On Thu, Nov 07, 2002 at 10:36:24AM -0500, Christopher Faylor wrote:
> Someone reported that doing a search for "setup.exe" in the cygwin archives
> had this effect.
We could force a clean database rebuild this weekend. If the database has
been corrupted in some way, that should clear things up. It takes a _long_
time, though, so it's definitely something to put off until the weekend
when system load goes down..
J
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: htdig problems? (PING)
2002-11-07 7:34 ` htdig problems? (PING) Christopher Faylor
2002-11-07 10:00 ` Jason Molenda
@ 2002-11-07 11:06 ` Hans-Peter Nilsson
2002-11-07 11:08 ` Christopher Faylor
1 sibling, 1 reply; 13+ messages in thread
From: Hans-Peter Nilsson @ 2002-11-07 11:06 UTC (permalink / raw)
To: Christopher Faylor; +Cc: overseers
On Thu, 7 Nov 2002, Christopher Faylor wrote:
> On Wed, Oct 30, 2002 at 04:10:00PM -0500, Christopher Faylor wrote:
> >People are reporting that searches are taking a very long time and sometimes
> >reporting internal server errors. Is there a problem with htdig?
>
> H-P do you have any idea why people are reporting internal server errors
> when attempting to do searches? I've seen the same myself.
I've mentioned this before, FWIW. If a search takes longer than
5min, htdig (or really: htsearch) catches (or actually, does not
catch) SIGALARM from its alarm(2) call and apache sees the
termination as a 5xx, which is arguably a correct interpretation.
You get this for searches that would return many hits, in big
databases such as sourceware's and gcc's. A subsequent attempt
is often successful (finishes within 5min) because the disk
cache is then a bit warmer.
I'm unsure if changing that alarm call would be for the better.
A database search run-wild or a DOS attack would quickly make
the machine unresponsive.
brgds, H-P
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: htdig problems? (PING)
2002-11-07 11:06 ` Hans-Peter Nilsson
@ 2002-11-07 11:08 ` Christopher Faylor
2002-11-07 11:25 ` Hans-Peter Nilsson
` (2 more replies)
0 siblings, 3 replies; 13+ messages in thread
From: Christopher Faylor @ 2002-11-07 11:08 UTC (permalink / raw)
To: Hans-Peter Nilsson; +Cc: overseers
On Thu, Nov 07, 2002 at 02:05:57PM -0500, Hans-Peter Nilsson wrote:
>On Thu, 7 Nov 2002, Christopher Faylor wrote:
>> On Wed, Oct 30, 2002 at 04:10:00PM -0500, Christopher Faylor wrote:
>> >People are reporting that searches are taking a very long time and sometimes
>> >reporting internal server errors. Is there a problem with htdig?
>>
>> H-P do you have any idea why people are reporting internal server errors
>> when attempting to do searches? I've seen the same myself.
>
>I've mentioned this before, FWIW. If a search takes longer than
>5min, htdig (or really: htsearch) catches (or actually, does not
>catch) SIGALARM from its alarm(2) call and apache sees the
>termination as a 5xx, which is arguably a correct interpretation.
I've just been seeing this much more frequently recently so I'm wondering
if something is wrong somewhere. Maybe sourceware is having disk problems.
cgf
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: htdig problems? (PING)
2002-11-07 11:08 ` Christopher Faylor
@ 2002-11-07 11:25 ` Hans-Peter Nilsson
2002-11-07 12:27 ` Angela Marie Thomas
2002-11-07 12:30 ` Angela Marie Thomas
2 siblings, 0 replies; 13+ messages in thread
From: Hans-Peter Nilsson @ 2002-11-07 11:25 UTC (permalink / raw)
To: Christopher Faylor; +Cc: overseers
On Thu, 7 Nov 2002, Christopher Faylor wrote:
> On Thu, Nov 07, 2002 at 02:05:57PM -0500, Hans-Peter Nilsson wrote:
> >If a search takes longer than
> >5min, htdig (or really: htsearch) catches (or actually, does not
> >catch) SIGALARM from its alarm(2) call and apache sees the
> >termination as a 5xx, which is arguably a correct interpretation.
>
> I've just been seeing this much more frequently recently so I'm wondering
> if something is wrong somewhere. Maybe sourceware is having disk problems.
Could be. The logs from htdig db updates look fine, FWIW: no
spurius failures. Speaking of htdig, one temporary file from
the gcc update is a bit too close to (1 << 31) bytes:
1905972789. (Quite expected, though.)
brgds, H-P
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: htdig problems? (PING)
2002-11-07 11:08 ` Christopher Faylor
2002-11-07 11:25 ` Hans-Peter Nilsson
@ 2002-11-07 12:27 ` Angela Marie Thomas
2002-11-07 12:29 ` Christopher Faylor
2002-11-07 12:30 ` Angela Marie Thomas
2 siblings, 1 reply; 13+ messages in thread
From: Angela Marie Thomas @ 2002-11-07 12:27 UTC (permalink / raw)
To: Christopher Faylor; +Cc: Hans-Peter Nilsson, overseers
> I've just been seeing this much more frequently recently so I'm wondering
> if something is wrong somewhere. Maybe sourceware is having disk problems.
I've started seeing longer than normal times backing up /sourceware/www.
The amount of data changing is roughly the same, so perhaps there is a
disk problem. I took a quick peek at the logs, but only saw a few SCSI
error messages, not tons like I'd expect.
--Angela
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: htdig problems? (PING)
2002-11-07 12:27 ` Angela Marie Thomas
@ 2002-11-07 12:29 ` Christopher Faylor
2002-11-07 12:36 ` Christopher Faylor
0 siblings, 1 reply; 13+ messages in thread
From: Christopher Faylor @ 2002-11-07 12:29 UTC (permalink / raw)
To: angela; +Cc: Hans-Peter Nilsson, overseers
On Thu, Nov 07, 2002 at 12:27:22PM -0800, Angela Marie Thomas wrote:
>> I've just been seeing this much more frequently recently so I'm wondering
>> if something is wrong somewhere. Maybe sourceware is having disk problems.
>
>I've started seeing longer than normal times backing up /sourceware/www.
>The amount of data changing is roughly the same, so perhaps there is a
>disk problem. I took a quick peek at the logs, but only saw a few SCSI
>error messages, not tons like I'd expect.
It *seems* like load averages have been higher than usual lately, too.
I haven't been able to figure out what is causing it though.
cgf
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: htdig problems? (PING)
2002-11-07 11:08 ` Christopher Faylor
2002-11-07 11:25 ` Hans-Peter Nilsson
2002-11-07 12:27 ` Angela Marie Thomas
@ 2002-11-07 12:30 ` Angela Marie Thomas
2 siblings, 0 replies; 13+ messages in thread
From: Angela Marie Thomas @ 2002-11-07 12:30 UTC (permalink / raw)
To: Christopher Faylor; +Cc: Hans-Peter Nilsson, overseers
Forgot to mention... This seemed to start happening frequently around
10/22 but there are some normal time runs between then and now as well.
The times vary wildly. Backups start at 02:23 US/Pacific and generally
end around 05:00 US/Pacific. Lately they've been ending aroung 10:00
or even as late as 11:00.
--Angela
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: htdig problems? (PING)
2002-11-07 12:29 ` Christopher Faylor
@ 2002-11-07 12:36 ` Christopher Faylor
2002-11-07 13:05 ` Hans-Peter Nilsson
0 siblings, 1 reply; 13+ messages in thread
From: Christopher Faylor @ 2002-11-07 12:36 UTC (permalink / raw)
To: angela, Hans-Peter Nilsson, overseers
On Thu, Nov 07, 2002 at 03:31:16PM -0500, Christopher Faylor wrote:
>On Thu, Nov 07, 2002 at 12:27:22PM -0800, Angela Marie Thomas wrote:
>>> I've just been seeing this much more frequently recently so I'm wondering
>>> if something is wrong somewhere. Maybe sourceware is having disk problems.
>>
>>I've started seeing longer than normal times backing up /sourceware/www.
>>The amount of data changing is roughly the same, so perhaps there is a
>>disk problem. I took a quick peek at the logs, but only saw a few SCSI
>>error messages, not tons like I'd expect.
>
>It *seems* like load averages have been higher than usual lately, too.
>I haven't been able to figure out what is causing it though.
Now that I've said that, the load average is currently at 1.29 of
course.
cgf
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: htdig problems? (PING)
2002-11-07 12:36 ` Christopher Faylor
@ 2002-11-07 13:05 ` Hans-Peter Nilsson
2002-11-07 13:08 ` Christopher Faylor
0 siblings, 1 reply; 13+ messages in thread
From: Hans-Peter Nilsson @ 2002-11-07 13:05 UTC (permalink / raw)
To: Christopher Faylor; +Cc: angela, overseers
On Thu, 7 Nov 2002, Christopher Faylor wrote:
> On Thu, Nov 07, 2002 at 03:31:16PM -0500, Christopher Faylor wrote:
> >It *seems* like load averages have been higher than usual lately, too.
> >I haven't been able to figure out what is causing it though.
>
> Now that I've said that, the load average is currently at 1.29 of
> course.
Which seems to coincide with the gcc htdig update finishing
12:12:27 PST, having started 00:17. I don't know why it took
this long. It used to do that some months ago too, then dropped
back to a few hours.
brgds, H-P
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: htdig problems? (PING)
2002-11-07 13:05 ` Hans-Peter Nilsson
@ 2002-11-07 13:08 ` Christopher Faylor
2002-11-07 13:12 ` Hans-Peter Nilsson
0 siblings, 1 reply; 13+ messages in thread
From: Christopher Faylor @ 2002-11-07 13:08 UTC (permalink / raw)
To: Hans-Peter Nilsson; +Cc: angela, overseers
On Thu, Nov 07, 2002 at 04:05:46PM -0500, Hans-Peter Nilsson wrote:
>On Thu, 7 Nov 2002, Christopher Faylor wrote:
>> On Thu, Nov 07, 2002 at 03:31:16PM -0500, Christopher Faylor wrote:
>> >It *seems* like load averages have been higher than usual lately, too.
>> >I haven't been able to figure out what is causing it though.
>>
>> Now that I've said that, the load average is currently at 1.29 of
>> course.
>
>Which seems to coincide with the gcc htdig update finishing
>12:12:27 PST, having started 00:17. I don't know why it took
>this long. It used to do that some months ago too, then dropped
>back to a few hours.
Right. I remember when we added the CPU things got much faster.
cgf
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: htdig problems? (PING)
2002-11-07 13:08 ` Christopher Faylor
@ 2002-11-07 13:12 ` Hans-Peter Nilsson
0 siblings, 0 replies; 13+ messages in thread
From: Hans-Peter Nilsson @ 2002-11-07 13:12 UTC (permalink / raw)
To: Christopher Faylor; +Cc: angela, overseers
On Thu, 7 Nov 2002, Christopher Faylor wrote:
> On Thu, Nov 07, 2002 at 04:05:46PM -0500, Hans-Peter Nilsson wrote:
> >On Thu, 7 Nov 2002, Christopher Faylor wrote:
> >> Now that I've said that, the load average is currently at 1.29 of
> >> course.
> >
> >Which seems to coincide with the gcc htdig update finishing
> >12:12:27 PST, having started 00:17. I don't know why it took
> >this long. It used to do that some months ago too, then dropped
> >back to a few hours.
>
> Right. I remember when we added the CPU things got much faster.
Doh! Yeah, I chose to remember just the effect or something.
Or is there another CPU to add? Can it be done regularly? :-)
brgds, H-P
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2002-11-07 21:12 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2002-10-30 13:08 htdig problems? Christopher Faylor
2002-11-07 7:34 ` htdig problems? (PING) Christopher Faylor
2002-11-07 10:00 ` Jason Molenda
2002-11-07 11:06 ` Hans-Peter Nilsson
2002-11-07 11:08 ` Christopher Faylor
2002-11-07 11:25 ` Hans-Peter Nilsson
2002-11-07 12:27 ` Angela Marie Thomas
2002-11-07 12:29 ` Christopher Faylor
2002-11-07 12:36 ` Christopher Faylor
2002-11-07 13:05 ` Hans-Peter Nilsson
2002-11-07 13:08 ` Christopher Faylor
2002-11-07 13:12 ` Hans-Peter Nilsson
2002-11-07 12:30 ` Angela Marie Thomas
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).