htdig yet again

htdig yet again

[Hans-Peter Nilsson]

I wish I could blame the new system :-) but the only fault was
that the old gcc htdig db files weren't carried over, which may
or may not have been on purpose.  That in turn uncovered bugs
(all mine) in the reindex-failure-recovery system (*):

1 - The DB integrity test used the *old* files (the "live" ones
    used when searching) not the newly updated/created ones, so
    that failed when there were no previous files...
    This was due to a functionality change after a security
    patch embarrasingly long ago.
2 - A bug in the integrity test error message obfuscated...
3 - A bug in integrity test usage; faulty parameters after
    a recovery-reindex caused failure for all automatic
    re-indexing.

(*) As some may have forgot, htdig is (suspected of) sometimes
messing up its database files after an update, so a re-index run
is started if some integrity tests fail, searches for words
known to have at least a certain number of matches.  IIRC, the
integrity tests are also useful to cope with index failures due
to simple (possibly driver-related) disk media errors too. :-}

A new gcc re-index is running, and in a few minutes the
sourceware update will run too.  I think this system *can* take
that, and there certainly is enough for that now.

I'll try out an idea how to shrink the (~2G) databases tomorrow.

brgds, H-P

Re: htdig yet again

[Jason Molenda]

On Sun, Jan 19, 2003 at 07:17:43PM -0500, Hans-Peter Nilsson wrote:

> A new gcc re-index is running, and in a few minutes the
> sourceware update will run too.  I think this system *can* take
> that, and there certainly is enough for that now.

Normally, system load is very light for another 13 hours, so I
doubt it will cause troubles.

J

Re: htdig yet again

[Hans-Peter Nilsson]

On Sun, 19 Jan 2003, Hans-Peter Nilsson wrote:
> I think this system *can* take
> that, and there certainly is enough for that now.
Fumble-fingers                       ^ disk

Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/vg1/lvol9        25370020   3793820  20287600  16% /sourceware/htdig

brgds, H-P