Subject: Re: indexes

From: "Byron C. Darrah" <[email-masked]>
Date: Wed, 7 Oct 2000 09:52:41 -0700 (PDT)

> On Wed, 7 Oct 2000, Mark Clear wrote:
> 
> > Ohmygoodness, optional, please!  With over 5K messages in some of my
> > archives (the client is always right), it's already all my poor little
> > CPU can do to crunch the index before the next message comes flying in.

Humble suggestion: If you really have messages coming in that frequently,
and they get piped to hypermail so that there's a chance two hypermail
processes could be competing to write the index files then there's probably
a better way to do what you want, so that you can eliminate the risk of
corrupted indexes.

For example, what I do at my site (and I would not be surprised to learn
that a lot of other people do too) is direct incoming mail for an archive
to an mbox file instead of piping it to hypermail.  Then, at periodic
intervals a cron job runs a Makefile that runs hypermail if and only
if there are any new messages to be added to the archive.

Of course, you would set the cron job's period to something greater than
the amount of time it takes for your computer to update the hypermail
archive to gurantee that two hypermail processes wont compete.

Make can check the status of an archive extremely efficiently with a rule
something like this:

     index.html: mbox
          # Do stuff to add new messages to archive

So even if you have ten archives set up this way and let the cron jobs run
as often as once per minute, you probably won't even be able to notice any
load on your system except when there are actually new messages being
processed.

Cheers.