Re: htdig, was Re: cygwin-xfree

[Hans-Peter Nilsson <hp@bitrange.com>]

On Wed, 3 May 2000, Frank Ch. Eigler wrote:
> For the htdig databases, adding "compression_level: 9"
> to htdig.conf should reduce the disk space usage by about 20%.

Tests show that the db.docdb "before merging" went from 324231168 to
180889600 bytes, so I made that change for the gcc installation.  It
is running its normally scheduled update right now, but from scratch
today to save extra.  Doh, I just think I should have thought about that
long ago.

For the curious (but not curious enough to read up) this uses zlib to
individually compress each "document head"; the text that shows up for
each search hit.  FWIW, in 2.0 compression will be done at the DB level,
which would be the right thing.

If nothing broke (I'm cautious sometimes :-) we can move that conf item 
from gcc.conf to site.conf.

> Also, the rundig scripts use htdig's "-i" option (causing the
> databases to be regenerated from scratch),

No, since we don't use the basic rundig, as Jason told you.

> the "db.wordlist"
> files may be deleted after each search, saving another 30%.

No, it's needed when you do an "update" (not running from scratch).

Thanks!

brgds, H-P