Monthly mailing list archives: monthly-ml-tasks

Monthly mailing list archives: monthly-ml-tasks

[Gerald Pfeifer]

Matthias had reopred that the indices for the mailing lists are always
generated four to six hours late at the beginning of the month, even
though the actual pages are already there and new mails go to these new
pages.

I believe I found why this is the case.  Consider the following entry
in the crontab of listarch user:

  5 6 1 * * cd /sourceware/infra/monthly-ml-tasks; exec ./DOIT.sh

Indeed, this job is run on the first day of the month, at 6:05.  Are
there any objections if I move this to 0:07?

Gerald
-- 
Gerald Pfeifer (Jerry)   gerald@pfeifer.com   http://www.pfeifer.com/gerald/

Re: Monthly mailing list archives: monthly-ml-tasks

[Jason Molenda]

On Wed, Jun 16, 2004 at 11:43:39PM +0200, Gerald Pfeifer wrote:

> I believe I found why this is the case.  Consider the following entry
> in the crontab of listarch user:
> 
>   5 6 1 * * cd /sourceware/infra/monthly-ml-tasks; exec ./DOIT.sh
> 
> Indeed, this job is run on the first day of the month, at 6:05.  Are
> there any objections if I move this to 0:07?


Unless I'm remembering incorrectly, there is a entry for root which
HUPs apache a little while after the redirects/index pages have been
updated.  My guess is that the time of this crontab is related to
that crontab.

I'd recommend leaving some time between the two processes -- an hour
say -- so that any mail archiving that is in the middle of happening
has a chance to clear out.  But that's just MHO.  I purposefully wrote
in the 6 hour delay originally out of laziness - I didn't want to think
through the consequences of timing it closely and didn't think it really
mattered if there was a gap of a few hours.  I don't mind someone trying
to fix it, but I wouldn't try to get down to minutes-after-midnight.
If there is a big backlog of e-mail traffic to a mailing list like
gcc@, a lot of processes could be waiting for a lock in www-archives.sh.


J

Re: Monthly mailing list archives: monthly-ml-tasks

[Gerald Pfeifer]

On Wed, 16 Jun 2004, Jason Molenda wrote:
>>   5 6 1 * * cd /sourceware/infra/monthly-ml-tasks; exec ./DOIT.sh
>>
>> Indeed, this job is run on the first day of the month, at 6:05.  Are
>> there any objections if I move this to 0:07?
> Unless I'm remembering incorrectly, there is a entry for root which
> HUPs apache a little while after the redirects/index pages have been
> updated.  My guess is that the time of this crontab is related to
> that crontab.

Interestingly, it isn't really; the first rehup takes place before the
monthly-ml-tasks script is run:

  #### Re-hup the daemon each month so that the new ml-redirects
  #### can be loaded (cf infra/monthly-ml-tasks/update-http-redirects.sh)
  ####
  #### This should run shortly after listarch's monthly list duty
  #### script runs.
  # Those scripts can take oddly long amounts of time to finish, so two
  # extra rehups are here to ensure it gets done.

  45 5,8,12 1 * * /sbin/service httpd reload>/dev/null

I have now moved the monthyl-ml-tasks script from 6:05 to 3:05, to reduce
the windows of "outdated-ness" that Matthias complained about...

> I'd recommend leaving some time between the two processes -- an hour
> say -- so that any mail archiving that is in the middle of happening
> has a chance to clear out.  But that's just MHO.  I purposefully wrote
> in the 6 hour delay originally out of laziness - I didn't want to think
> through the consequences of timing it closely and didn't think it really
> mattered if there was a gap of a few hours.

...so given what you wrote above, the root crontab now should be fine,
without changes?

Gerald

Re: Monthly mailing list archives: monthly-ml-tasks

[Jason Molenda]

Hi Gerald,

>   # Those scripts can take oddly long amounts of time to finish, so two
>   # extra rehups are here to ensure it gets done.
> 
>   45 5,8,12 1 * * /sbin/service httpd reload>/dev/null
> 
> I have now moved the monthyl-ml-tasks script from 6:05 to 3:05, to reduce
> the windows of "outdated-ness" that Matthias complained about...

Sounds good -- thanks for following up on this.  We can all try to
remember to keep an eye on it on July 31 to make sure it all works
out correctly.  It's not the end of the world if the timing isn't
right and we fix it by hand a couple hours later.

Jason