htdig problems?

htdig problems?

[Christopher Faylor]

I just did a search on the gdb archives and found that htdig reported
no matches on the string 'alloca' despite the fact that there was a
recent thread which discussed it.

Is there a problem with the archives?

cgf

Re: htdig problems?

[Christopher Faylor]

On Fri, Nov 10, 2000 at 01:39:33AM -0500, Christopher Faylor wrote:
>I just did a search on the gdb archives and found that htdig reported
>no matches on the string 'alloca' despite the fact that there was a
>recent thread which discussed it.
>
>Is there a problem with the archives?

Nevermind.  I tried it again and it worked.  Who knows?  Maybe I misspelled
"alloca".

Sorry for the false alarm.

cgf

Re: htdig problems?

[Jason Molenda]

On Fri, Nov 10, 2000 at 01:43:54AM -0500, Christopher Faylor wrote:

> 
> Nevermind.  I tried it again and it worked.  Who knows?  Maybe I misspelled
> "alloca".
> 

Your first search was for the gdb list, restricted to the 2000-11 month.
Your second, which worked, was for the gdb-patches list.


(the all seeing, all knowing) Jason

Re: htdig problems?

[Christopher Faylor]

On Thu, Nov 09, 2000 at 11:46:45PM -0800, Jason Molenda wrote:
>On Fri, Nov 10, 2000 at 01:43:54AM -0500, Christopher Faylor wrote:
>
>> 
>> Nevermind.  I tried it again and it worked.  Who knows?  Maybe I misspelled
>> "alloca".
>> 
>
>Your first search was for the gdb list, restricted to the 2000-11 month.
>Your second, which worked, was for the gdb-patches list.
>
>
>(the all seeing, all knowing) Jason

I probably did restrict my first search to 2000-11 but there are a number of messages
related to alloca in that month (from me) so I should have found something.

So maybe something is broken after all.

cgf

Re: htdig problems?

[Hans-Peter Nilsson]

On Fri, 10 Nov 2000, Christopher Faylor wrote:
> I probably did restrict my first search to 2000-11 but there are a number of messages
> related to alloca in that month (from me) so I should have found something.

Do you mean earlier than yesterday?  The messages of yesterday are indexed
now, but were not when you wrote the above.

> So maybe something is broken after all.

If there is a message containing alloca this month that was sent earlier
than yesterday, I'll have a closer look.  But please do me the favor of
finding and quoting one.

brgds, H-P

Re: htdig problems?

[Chris Faylor]

On Fri, Nov 10, 2000 at 01:23:17PM -0500, Hans-Peter Nilsson wrote:
>On Fri, 10 Nov 2000, Christopher Faylor wrote:
>> I probably did restrict my first search to 2000-11 but there are a number of messages
>> related to alloca in that month (from me) so I should have found something.
>
>Do you mean earlier than yesterday?  The messages of yesterday are indexed
>now, but were not when you wrote the above.

Nope.  I was looking for something written on yesterday, yesterday.  So this
is not a problem, then.  Again, sorry for the false alarm.

cgf

>> So maybe something is broken after all.
>
>If there is a message containing alloca this month that was sent earlier
>than yesterday, I'll have a closer look.  But please do me the favor of
>finding and quoting one.

Re: htdig problems?

[Jason Molenda]

On Fri, Nov 10, 2000 at 11:23:34AM -0500, Christopher Faylor wrote:

> >Your first search was for the gdb list, restricted to the 2000-11 month.
                                 ^^^
> >Your second, which worked, was for the gdb-patches list.
                                          ^^^^^^^^^^^

> I probably did restrict my first search to 2000-11 but there are a number
> of messages related to alloca in that month (from me) so I should have 
> found something.
> 
> So maybe something is broken after all.

No, your two searches were to different lists, hence the different results.


(do not doubt the all seeing, all knowing) Jason

Re: htdig problems?

[Christopher Faylor]

On Fri, Nov 10, 2000 at 10:53:27AM -0800, Jason Molenda wrote:
>On Fri, Nov 10, 2000 at 11:23:34AM -0500, Christopher Faylor wrote:
>>>Your first search was for the gdb list, restricted to the 2000-11 month.
>                                 ^^^
>>>Your second, which worked, was for the gdb-patches list.
>
>> I probably did restrict my first search to 2000-11 but there are a number
>> of messages related to alloca in that month (from me) so I should have 
>> found something.
>> 
>> So maybe something is broken after all.
>
>No, your two searches were to different lists, hence the different results.

Are you seeing my attempts in a log file somewhere?  You are making
definitive statements based on data that I don't recall providing, so I
assume that you are either really confident or there is a log somewhere.

When I noticed that gdb-patches worked, I went back to gdb and got results.
At that point I probably wasn't looking in 2000-11 only.

We really can let this thread die now, can't we?

cgf

Re: htdig problems?

[Jason Molenda]

On Fri, Nov 10, 2000 at 01:43:54AM -0500, Christopher Faylor wrote:

> 
> Nevermind.  I tried it again and it worked.  Who knows?  Maybe I misspelled
> "alloca".
> 

Your first search was for the gdb list, restricted to the 2000-11 month.
Your second, which worked, was for the gdb-patches list.


(the all seeing, all knowing) Jason

Re: htdig problems?

[Jason Molenda]

On Fri, Nov 10, 2000 at 11:23:34AM -0500, Christopher Faylor wrote:

> >Your first search was for the gdb list, restricted to the 2000-11 month.
                                 ^^^
> >Your second, which worked, was for the gdb-patches list.
                                          ^^^^^^^^^^^

> I probably did restrict my first search to 2000-11 but there are a number
> of messages related to alloca in that month (from me) so I should have 
> found something.
> 
> So maybe something is broken after all.

No, your two searches were to different lists, hence the different results.


(do not doubt the all seeing, all knowing) Jason

Re: htdig problems?

[Chris Faylor]

On Fri, Nov 10, 2000 at 01:23:17PM -0500, Hans-Peter Nilsson wrote:
>On Fri, 10 Nov 2000, Christopher Faylor wrote:
>> I probably did restrict my first search to 2000-11 but there are a number of messages
>> related to alloca in that month (from me) so I should have found something.
>
>Do you mean earlier than yesterday?  The messages of yesterday are indexed
>now, but were not when you wrote the above.

Nope.  I was looking for something written on yesterday, yesterday.  So this
is not a problem, then.  Again, sorry for the false alarm.

cgf

>> So maybe something is broken after all.
>
>If there is a message containing alloca this month that was sent earlier
>than yesterday, I'll have a closer look.  But please do me the favor of
>finding and quoting one.

Re: htdig problems?

[Hans-Peter Nilsson]

On Fri, 10 Nov 2000, Christopher Faylor wrote:
> I probably did restrict my first search to 2000-11 but there are a number of messages
> related to alloca in that month (from me) so I should have found something.

Do you mean earlier than yesterday?  The messages of yesterday are indexed
now, but were not when you wrote the above.

> So maybe something is broken after all.

If there is a message containing alloca this month that was sent earlier
than yesterday, I'll have a closer look.  But please do me the favor of
finding and quoting one.

brgds, H-P

Re: htdig problems?

[Christopher Faylor]

On Thu, Nov 09, 2000 at 11:46:45PM -0800, Jason Molenda wrote:
>On Fri, Nov 10, 2000 at 01:43:54AM -0500, Christopher Faylor wrote:
>
>> 
>> Nevermind.  I tried it again and it worked.  Who knows?  Maybe I misspelled
>> "alloca".
>> 
>
>Your first search was for the gdb list, restricted to the 2000-11 month.
>Your second, which worked, was for the gdb-patches list.
>
>
>(the all seeing, all knowing) Jason

I probably did restrict my first search to 2000-11 but there are a number of messages
related to alloca in that month (from me) so I should have found something.

So maybe something is broken after all.

cgf

Re: htdig problems?

[Christopher Faylor]

On Fri, Nov 10, 2000 at 01:39:33AM -0500, Christopher Faylor wrote:
>I just did a search on the gdb archives and found that htdig reported
>no matches on the string 'alloca' despite the fact that there was a
>recent thread which discussed it.
>
>Is there a problem with the archives?

Nevermind.  I tried it again and it worked.  Who knows?  Maybe I misspelled
"alloca".

Sorry for the false alarm.

cgf

htdig problems?

[Christopher Faylor]

I just did a search on the gdb archives and found that htdig reported
no matches on the string 'alloca' despite the fact that there was a
recent thread which discussed it.

Is there a problem with the archives?

cgf

Re: htdig problems?

[Christopher Faylor]

On Fri, Nov 10, 2000 at 10:53:27AM -0800, Jason Molenda wrote:
>On Fri, Nov 10, 2000 at 11:23:34AM -0500, Christopher Faylor wrote:
>>>Your first search was for the gdb list, restricted to the 2000-11 month.
>                                 ^^^
>>>Your second, which worked, was for the gdb-patches list.
>
>> I probably did restrict my first search to 2000-11 but there are a number
>> of messages related to alloca in that month (from me) so I should have 
>> found something.
>> 
>> So maybe something is broken after all.
>
>No, your two searches were to different lists, hence the different results.

Are you seeing my attempts in a log file somewhere?  You are making
definitive statements based on data that I don't recall providing, so I
assume that you are either really confident or there is a log somewhere.

When I noticed that gdb-patches worked, I went back to gdb and got results.
At that point I probably wasn't looking in 2000-11 only.

We really can let this thread die now, can't we?

cgf

Re: htdig problems?

[Hans-Peter Nilsson]

On Tue, 1 Feb 2005, Christopher Faylor wrote:
> It seems like htdig is causing problems again:
>
> USER       PID %CPU %MEM   VSZ  RSS TTY      STAT START   TIME COMMAND
> htdigid  24995  0.0  0.0  2128  980 ?        S    Jan31   0:00 /bin/sh /sourceware/htdig/gcc/bin/htupdate.sh
> htdigid   6030 15.3  4.8 104448 100704 ?     R    07:49  64:49 /sourceware/htdig/gcc/bin/htdig -s -a
> htdigid   6109  0.0  0.0  2120  784 ?        S    07:49   0:00 sh -c bzip2 >> /sourceware/htdig/gcc/db/db.wordlist.work
> htdigid   6111  2.2  0.3  8916 6876 ?        S    07:49   9:38 bzip2
> htdigid  29573  0.0  0.0  2136  948 ?        S    08:17   0:00 /bin/sh /sourceware/htdig/gcc/bin/htupdate.sh
> htdigid  22429 15.3  4.0 91060 84048 ?       D    08:44  56:19 /sourceware/htdig/gcc/bin/htdig -s -a
> htdigid  22439  0.0  0.0  2124  884 ?        S    08:44   0:00 sh -c bzip2 >> /sourceware/htdig/gcc/db/db.wordlist.work
> htdigid  22440  2.0  0.3  8912 6876 ?        S    08:44   7:24 bzip2
>
> The load average on sourceware.org is 32 currently...

People please, in the future, say if you changed some state.
It took me a while to notice that one of the htdig updates was
"kill -STOP":ed...

Htdig updates take A LOT longer these days.  I guess it's due to
the wiki; indexing through a cgi keeps htdig, php and apache
busy.  I'll exclude indexing for the wiki (sure indexing that
would likely be helpful but Sorry, No Can Do).  Since it took
longer than 24h to index(!), and today and yesterday both being
odd dates, the first update ran into the second one and blammo.
Right, there should be some kind of stamp/semaphore file to stop
that from ever happening.

I killed both updates, since both were likely damaged and
useless.  (The *used* DB is still there.)

brgds, H-P

Re: htdig problems?

[Hans-Peter Nilsson]

On Tue, 1 Feb 2005, Christopher Faylor wrote:

> It seems like htdig is causing problems again:

I'm on it.

brgds, H-P

htdig problems?

[Christopher Faylor]

It seems like htdig is causing problems again:

USER       PID %CPU %MEM   VSZ  RSS TTY      STAT START   TIME COMMAND
htdigid  24995  0.0  0.0  2128  980 ?        S    Jan31   0:00 /bin/sh /sourceware/htdig/gcc/bin/htupdate.sh
htdigid   6030 15.3  4.8 104448 100704 ?     R    07:49  64:49 /sourceware/htdig/gcc/bin/htdig -s -a
htdigid   6109  0.0  0.0  2120  784 ?        S    07:49   0:00 sh -c bzip2 >> /sourceware/htdig/gcc/db/db.wordlist.work
htdigid   6111  2.2  0.3  8916 6876 ?        S    07:49   9:38 bzip2
htdigid  29573  0.0  0.0  2136  948 ?        S    08:17   0:00 /bin/sh /sourceware/htdig/gcc/bin/htupdate.sh
htdigid  22429 15.3  4.0 91060 84048 ?       D    08:44  56:19 /sourceware/htdig/gcc/bin/htdig -s -a
htdigid  22439  0.0  0.0  2124  884 ?        S    08:44   0:00 sh -c bzip2 >> /sourceware/htdig/gcc/db/db.wordlist.work
htdigid  22440  2.0  0.3  8912 6876 ?        S    08:44   7:24 bzip2

The load average on sourceware.org is 32 currently...

cgf

htdig problems?

[Christopher Faylor]

People are reporting that searches are taking a very long time and sometimes
reporting internal server errors.  Is there a problem with htdig?

cgf

Re: htdig problems?

[Christopher Faylor]

On Tue, Jul 02, 2002 at 07:53:33PM -0400, Hans-Peter Nilsson wrote:
>On Tue, 2 Jul 2002, Hans-Peter Nilsson wrote:
>> On Tue, 2 Jul 2002, Christopher Faylor wrote:
>>>I was wondering if the database was in a strange state or something.
>>
>>I don't know.  (Checking search on "hanging"...)
>
>... i.e. with "hanging" as an example word on gcc: worked.  It
>*does* take time.  There's a timeout after 5m, resulting in a
>500 "internal something error" which people will hit with common
>words and a cold disk-cache.  Please forward complaints to me if
>appropriate.

Ok, will do.  People are complaining in the cygwin mailing list (what a
novel experience that is) that they are getting timeouts but they don't
mention what the search terms might be.

cgf

Re: htdig problems?

[Hans-Peter Nilsson]

On Tue, 2 Jul 2002, Hans-Peter Nilsson wrote:

> On Tue, 2 Jul 2002, Christopher Faylor wrote:
>
> > I was wondering if the database was in a strange state or something.
>
> I don't know.  (Checking search on "hanging"...)

... i.e. with "hanging" as an example word on gcc: worked.  It
*does* take time.  There's a timeout after 5m, resulting in a
500 "internal something error" which people will hit with common
words and a cold disk-cache.  Please forward complaints to me if
appropriate.

brgds, H-P

Re: htdig problems?

[Hans-Peter Nilsson]

On Tue, 2 Jul 2002, Christopher Faylor wrote:

> I was wondering if the database was in a strange state or something.

I don't know.  (Checking search on "hanging"...)

> I meant to add that I saw that you were monitoring this process so
> I'm not worried.  If you think it is nearly complete there is no
> reason to deviate from your plan.

My plan was to kill it if I didn't see progress "anytime soon"
(loosely defined :-)

> On Tue, Jul 02, 2002 at 06:47:41PM -0400, Hans-Peter Nilsson wrote:

> >Bottom line:  htdig 3.2.0b4
> >isn't usable as-is for sourceware.
>
> Too bad.  So, what are our options?  Are there any other packages
> that are similar to htdig?  Would compiling htdig to C from perl
> help any?

It's in C++ so that's not an option. :-)  (I don't know of htdig
alternatives that *would* handle near-2G data, have not
investigated, feel free.)  I think I've mentioned the plan, but
since you ask:

For the future (months), the plan is to fix htdig, given that
it's fixable without too much trouble.  The indexing delay is an
extreme performance regression from htdig-3.1.5, the kind you
get when someone has added a few objects with nontrivial
ctor/dtors and keep looping over them...

For the immediate situation (i.e. within days), I assume the
database *isn't* really >2G; that it's just a temporary
condition exposed by indexing from scratch rather than by
pieces.  So I'll index piece by piece, and keep the final result
for future from-scratch indexing.

brgds, H-P

Re: htdig problems?

[Christopher Faylor]

On Tue, Jul 02, 2002 at 06:47:41PM -0400, Hans-Peter Nilsson wrote:
>On Tue, 2 Jul 2002, Christopher Faylor wrote:
>> A lot of people are complaining that mailing list searches are hanging.
>
>That shouldn't be related to a running htdig process (except for
>CPU and disk contention).

I was wondering if the database was in a strange state or something.

>> I see one htdig process which has been running since 2002-06-28.
>
>That's right.  (I've kept a watch loop, killing it if it grow
>out of proportions.)  I'm considering killing it.  Well... since
>people start complaining, I'll do it right away.  What it hasn't
>finished in 3.5 days isn't waiting for.  It's probably just busy
>running constructors and destructors...

I meant to add that I saw that you were monitoring this process so
I'm not worried.  If you think it is nearly complete there is no
reason to deviate from your plan.

>> Is there a problem, maybe, or is this just recovery from the disk space
>> problem of last week?
>
>I mentioned in another message that usual indexing from scratch
>didn't work anymore because two of the gcc db files had grown
>larger than 2G.  The htdig process is just me checking if the
>newer htdig non-release was usable.  Bottom line:  htdig 3.2.0b4
>isn't usable as-is for sourceware.

Too bad.  So, what are our options?  Are there any other packages
that are similar to htdig?  Would compiling htdig to C from perl
help any?

cgf

Re: htdig problems?

[Hans-Peter Nilsson]

On Tue, 2 Jul 2002, Christopher Faylor wrote:
> A lot of people are complaining that mailing list searches are hanging.

That shouldn't be related to a running htdig process (except for
CPU and disk contention).

> I see one htdig process which has been running since 2002-06-28.

That's right.  (I've kept a watch loop, killing it if it grow
out of proportions.)  I'm considering killing it.  Well... since
people start complaining, I'll do it right away.  What it hasn't
finished in 3.5 days isn't waiting for.  It's probably just busy
running constructors and destructors...

> Is there a problem, maybe, or is this just recovery from the disk space
> problem of last week?

I mentioned in another message that usual indexing from scratch
didn't work anymore because two of the gcc db files had grown
larger than 2G.  The htdig process is just me checking if the
newer htdig non-release was usable.  Bottom line:  htdig 3.2.0b4
isn't usable as-is for sourceware.

brgds, H-P

htdig problems?

[Christopher Faylor]

A lot of people are complaining that mailing list searches are hanging.
I see one htdig process which has been running since 2002-06-28.

Is there a problem, maybe, or is this just recovery from the disk space
problem of last week?

cgf