fetchmail/procmail lost messages

fetchmail/procmail lost messages

[Kelly Felkins]

How can I configure fetchmail/procmail to fail *safe*? So that, if there are any mail processing problems, messages are not lost?

I recently started using fetchmail, procmail, and mutt to handle email on my pc in cygwin. 

Apparently, if procmail has a problem, the particular message is lost. My poor little procmail has had a variety of problems, and has lost a variety of messages at this point. Each problem has been easy to fix, but the messages are long gone. 

For example, today the hard drive that I use for both the spool file and my mail boxes was off-line. Five messages to /dev/null.

I'm inclined to think this is a bad design. Or have I got it configured incorrectly?

.fetchmailrc
set logfile /var/log/fetchmail.log
set invisible
set no bouncemail

poll "pop.mail.yahoo.com"
protocol pop3
username "mumble"
password "mumble"
fetchall
nokeep
mda "/usr/bin/procmail -d %T"



.procmailrc
MAILDIR=/cygdrive/g/cygwin/home/kellyf/Mail
DEFAULT=$MAILDIR/inbox

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: fetchmail/procmail lost messages

[Jason Tishler]

Kelly,

On Thu, Apr 22, 2004 at 05:59:06PM -0700, Kelly Felkins wrote:
> For example, today the hard drive that I use for both the spool file
> and my mail boxes was off-line. Five messages to /dev/null.
> 
> I'm inclined to think this is a bad design. Or have I got it
> configured incorrectly?

Possibly.

I highly recommend using a spool file that is *always* on-line (e.g.,
/var/spool/mail/$LOGNAME).  Nevertheless, fetchmail should not have
flushed the message if procmail exited with a nonzero status.

BTW, I have used Cygwin fetchmail/procmail to process my mail for about
3 years.  I process more 10,000 emails per month.  So, I have processed
more than 300,000 messages during this time period.  AFAICT, I have not
lost a single message.

Jason

-- 
PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
Fingerprint: 7A73 1405 7F2B E669 C19D  8784 1AFD E4CC ECF4 8EF6

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: fetchmail/procmail lost messages

[Kelly Felkins]

Jason Tishler <jason <at> tishler.net> writes:

> 
> Kelly,
> 
> On Thu, Apr 22, 2004 at 05:59:06PM -0700, Kelly Felkins wrote:
> > For example, today the hard drive that I use for both the spool file
> > and my mail boxes was off-line. Five messages to /dev/null.
> > 
> > I'm inclined to think this is a bad design. Or have I got it
> > configured incorrectly?
> 
> Possibly.
> 
> I highly recommend using a spool file that is *always* on-line (e.g.,
> /var/spool/mail/$LOGNAME).  Nevertheless, fetchmail should not have
> flushed the message if procmail exited with a nonzero status.
> 
> BTW, I have used Cygwin fetchmail/procmail to process my mail for about
> 3 years.  I process more 10,000 emails per month.  So, I have processed
> more than 300,000 messages during this time period.  AFAICT, I have not
> lost a single message.
> 
> Jason
> 

Thanks for responding. Your suggestion for locating the spool file is wise. A
few questions:

- How do you know you haven't lost messages? It appears that the only notice I
get is a brief message on my console, just before the message about "flushing"
-- it's a bad feeling to see an error message, followed by "flushing".

- Any suggestions for diagnosing the problem? It seems like procmail should
return an error code, and fetchmail should respond to that error code - this is
apparently not happening. I don't know if procmail is not returning an error
status, or if fetchmail is ignoring it. How can I verify this?

When everything is configured correctly the system works nicely. But I have to
say that I am deeply troubled that when there is a problem it looses messages. 

Any other suggestions? Like, should I dump fetchmail or procmail or both? Any
recommended alternatives?

Thanks very much.

-Kelly


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: fetchmail/procmail lost messages

[Jason Tishler]

Kelly,

On Fri, Apr 23, 2004 at 04:44:19PM +0000, Kelly Felkins wrote:
> Jason Tishler <jason <at> tishler.net> writes:
> > BTW, I have used Cygwin fetchmail/procmail to process my mail for
> > about 3 years.  I process more 10,000 emails per month.  So, I have
> > processed more than 300,000 messages during this time period.
> > AFAICT, I have not lost a single message.
> 
> Thanks for responding. Your suggestion for locating the spool file is
> wise. A few questions:
> 
> - How do you know you haven't lost messages? It appears that the only
> notice I get is a brief message on my console, just before the message
> about "flushing" -- it's a bad feeling to see an error message,
> followed by "flushing".

I don't really know, hence the "AFAICT".  Nevertheless, my perception is
that message delivery seems very reliable.

> - Any suggestions for diagnosing the problem?

Use VERBOSE=yes, as in the following:

    $ procmail VERBOSE=yes .procmailrc-test </tmp/jtishler.mail 
    procmail: [4400] Fri Apr 23 16:42:06 2004
    procmail: Rcfile: "/home/jtishler/.procmailrc-test"
    procmail: Assigning "MAILDIR=/home/jtishler"
    procmail: Assigning "MAILDIR=/cygdrive/g/cygwin/home/kellyf/Mail"
    procmail: Couldn't chdir to "/cygdrive/g/cygwin/home/kellyf/Mail"
    procmail: Assigning "MAILDIR=."
    procmail: Assigning "DEFAULT=./inbox"
    procmail: Locking "./inbox.lock"
    procmail: Assigning "LASTFOLDER=./inbox"
    procmail: Opening "./inbox"
    procmail: Acquiring kernel-lock
    procmail: Unlocking "./inbox.lock"
    From jtishler  Tue Mar 30 13:30:54 2004
     Subject: XYZ: FW: UNIX ID for Jason Tishler
      Folder: ./inbox

The above indicates that your mail was not, lost but stored in ~/inbox.
This was confirmed by Robert.  Can you confirm this too?

> It seems like procmail should return an error code, and fetchmail
> should respond to that error code - this is apparently not happening.
> I don't know if procmail is not returning an error status, or if
> fetchmail is ignoring it. How can I verify this?

procmail is returning 0, as confirmed by Robert:

    $ echo $?
    0

Which explains why fetchmail flushes the message.

> Any other suggestions? Like, should I dump fetchmail or procmail or
> both? Any recommended alternatives?

Use reasonable locations (e.g., local drives) to store your mail
messages.

Jason

-- 
PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
Fingerprint: 7A73 1405 7F2B E669 C19D  8784 1AFD E4CC ECF4 8EF6

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

Re: fetchmail/procmail lost messages

[Robert R Schneck]

Kelly Felkins <kellyfelkins2003@yahoo.com> wrote:
> How can I configure fetchmail/procmail to fail *safe*? So that, if
> there are any mail processing problems, messages are not lost?

This should just work.  Read "RETRIEVAL FAILURE MODES" in the 
fetchmail man page.

When I try to reproduce the problem with $MAILDIR on a non-existent
drive, procmail indeed doesn't fail, but delivers the mail to my home
directory instead.  Perhaps it would be better if procmail just failed
(and maybe it can even be configured to do so), but at least the message
isn't lost.

Robert


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/