ezmlm blocking and mailing list idea

ezmlm blocking and mailing list idea

[Christopher Faylor]

I was doing the research today to figure out how to add "if you are
a subscriber it's ok to post" to the rbl-check.sh blocking.

It is pretty straightforward to do, but then I started thinking that
since we have to do this for every submission to the mailing lists
anyway it would be sort of interesting to add an additional field like:

List-Poster-Is-Subscribed: yes

to the header.

That would enable you to know when it was necessary to Cc someone when
you are responding to the list and when it was not necessary.

I know that I could probably write something in mutt to detect this and
do the right thing when I replied to messages.  Would this be useful to
anyone else?

I'm assuming that this should be relatively easy to do with ezmlm but I
haven't yet researched if this is the case.

cgf

Re: ezmlm blocking and mailing list idea

[Gerald Pfeifer]

On Sat, 25 Nov 2000, Christopher Faylor wrote:
> I was doing the research today to figure out how to add "if you are
> a subscriber it's ok to post" to the rbl-check.sh blocking.

Looks like a rather nice addition!

> List-Poster-Is-Subscribed: yes
>
> That would enable you to know when it was necessary to Cc someone when
> you are responding to the list and when it was not necessary.

I happen to like the current way, because that way replies to messages of
mine are delivered to my (main) inbox, while the lists themselves are
filtered to a folder of their own that I do not check that often (and do
not read as carefully as my inboxes).

But perhaps that's just me, and everyone else prefers not to receive such
copies!

Gerald
-- 
Gerald "Jerry" pfeifer@dbai.tuwien.ac.at http://www.dbai.tuwien.ac.at/~pfeifer/

Re: ezmlm blocking and mailing list idea

[Jason Molenda]

On Sat, Nov 25, 2000 at 08:30:26PM -0500, Christopher Faylor wrote:
> I was doing the research today to figure out how to add "if you are
> a subscriber it's ok to post" to the rbl-check.sh blocking.

Cool!

> It is pretty straightforward to do, but then I started thinking that
> since we have to do this for every submission to the mailing lists
> anyway it would be sort of interesting to add an additional field like:
> 
> List-Poster-Is-Subscribed: yes
> 
> to the header.

You'd probably want to prefix it with "X-", and the "List-" headers
are defined in an RFC, so I'd avoid any resemblance to that namespace.
"X-Poster-Is-Subscribed:"?

It won't work when the person is subscribed as cgf@cygnus.com but
sends mail from cgf@redhat.com or cgf@rtl.cygnus.com, but I bet
it'll correctly identify the majority of senders.

I don't know if any list users would ever make good use of this
header, but I've often added little features to sourceware where
I'm probably the only one using them.  :-)  I say go for it.


> That would enable you to know when it was necessary to Cc someone when
> you are responding to the list and when it was not necessary.

I've always had a beef with schemes that try to get a Group-Reply to work
correctly.  The Reply-To header is the most obvious and worst of the bunch.
The clearest problem of these (beyond the fact that the subscriber may not
be subscribed to the list) is when a note goes to _two_ mailing lists.
The Reply-To style scheme will only reply to one of those.  And if you're
subscribed to both and have a procmailrc that supresses duplicates (don't
we all), it'll be random which Reply-To you get.


Anyway, with the header you're proposing, the MUA could send the
reply to the To and Cc listings, and suppress the From header.
And that'd certainly be a good step in the right direction.


Jason

PS-  One downside to appending this header:  If you're only checking
the subscriber list for spam blocking, you only need to get the
full subscriber list if their MTA is RBL'ed.  If you want to add
the header, you're going to need to get the subscriber list for
every note.  I don't think this is such a big deal for our site --
even the busiest list is only ~100 mail notes a day, so a few
additional seconds to get the subscriber list isn't a big deal.
But I thought I'd point it out.

Re: ezmlm blocking and mailing list idea

[Christopher Faylor]

On Sat, Nov 25, 2000 at 07:26:24PM -0800, Jason Molenda wrote:
>> It is pretty straightforward to do, but then I started thinking that
>> since we have to do this for every submission to the mailing lists
>> anyway it would be sort of interesting to add an additional field like:
>> 
>> List-Poster-Is-Subscribed: yes
>> 
>> to the header.
>
>You'd probably want to prefix it with "X-", and the "List-" headers
>are defined in an RFC, so I'd avoid any resemblance to that namespace.
>"X-Poster-Is-Subscribed:"?

Thanks.  I forgot that List- was in the RFC.

>It won't work when the person is subscribed as cgf@cygnus.com but
>sends mail from cgf@redhat.com or cgf@rtl.cygnus.com, but I bet
>it'll correctly identify the majority of senders.

Yeah.  The "subscribers allowed" scheme will also cause problems for
people who use a From: address that is not the address used to subscribe
to the list, too.

I wonder if I'm opening myself up to a new postmaster@sources.redhat.com
request:

"I understand that you can add my alternate address so that I can send
email to your mailing list?  Would you mind doing that for me?  I can't
change my From address because I use Lotus Notes 1.47." (repeat)

>I don't know if any list users would ever make good use of this
>header, but I've often added little features to sourceware where
>I'm probably the only one using them.  :-)  I say go for it.

I've read the ezmlm documentation and I can't find any way to add a dynamic
header like this, so I'm not 100% certain that it can be done.

>> That would enable you to know when it was necessary to Cc someone when
>> you are responding to the list and when it was not necessary.
>
>I've always had a beef with schemes that try to get a Group-Reply to work
>correctly.  The Reply-To header is the most obvious and worst of the bunch.
>The clearest problem of these (beyond the fact that the subscriber may not
>be subscribed to the list) is when a note goes to _two_ mailing lists.
>The Reply-To style scheme will only reply to one of those.  And if you're
>subscribed to both and have a procmailrc that supresses duplicates (don't
>we all), it'll be random which Reply-To you get.

Yeah, I use the Reply-To header in the cygwin mailing list because otherwise
I'd have 57 messages in my inbox every day rather than in my cygwin folder.
I've been meaning to play with procmail to see if I can come up with a scheme
for forcing anything that goes to a mailing list into its proper folder if it
has been Cc'ed to me.  You're right that the most popular method for eliminating
duplicates doesn't allow you to prioritize things, so this is a problem.

>PS- One downside to appending this header: If you're only checking the
>subscriber list for spam blocking, you only need to get the full
>subscriber list if their MTA is RBL'ed.  If you want to add the header,
>you're going to need to get the subscriber list for every note.  I
>don't think this is such a big deal for our site -- even the busiest
>list is only ~100 mail notes a day, so a few additional seconds to get
>the subscriber list isn't a big deal.  But I thought I'd point it out.

Hmm.  The way I've written this so far is that the subscriber test happens
prior to the rbl check.  I figured that this would be much faster than
the reverse since the rbl check requires name lookups over the internet
via a few hosts.

This will cause more wear and tear on the hard disk but doesn't it make
sense to do things like this, in the interests of speed?

cgf

Re: ezmlm blocking and mailing list idea

[Jason Molenda]

On Sat, Nov 25, 2000 at 10:37:13PM -0500, Christopher Faylor wrote:

> >I don't know if any list users would ever make good use of this
> >header, but I've often added little features to sourceware where
> >I'm probably the only one using them.  :-)  I say go for it.
> 
> I've read the ezmlm documentation and I can't find any way to add a dynamic
> header like this, so I'm not 100% certain that it can be done.


One nice thing about ezmlm is that it's always easy to add on lots of
grody hacks. :-)  You could prefix the one or two critical lines in 
the DIR/editor file, so

|ezmlm-send ....

goes to

|/usr/sourceware/bin/add-user-in-list-header.sh | ezmlm-send ...

Or what have you.  Man that's nasty.  I don't think it'd be worth it, but
it is possible.  Another way would be to modify ezmlm itself to modify
the mail note.  

> Yeah, I use the Reply-To header in the cygwin mailing list because otherwise
> I'd have 57 messages in my inbox every day rather than in my cygwin folder.
> I've been meaning to play with procmail to see if I can come up with a scheme
> for forcing anything that goes to a mailing list into its proper folder if it
> has been Cc'ed to me.  


I probably don't understand what you want to do.  Wouldn't a simple

:0
* ^TOcygwin@sources.redhat.com
  cygwin-list-mail

do it?  This would cause all mail sent to cygwin@sources to go in
to the cygwin-list-mail folder in your mail directory (and not into
your incoming mail box).  You could add another entry to handle
the @sourceware addr, or you can probably combine both with a
logical OR expression of some kind.

I have a bunch of lists where I save copies of all e-mail to a
dropbox (use ":0 c" instead of ":0" to save a copy to the dropbox),
and most of them I also have put in my incoming mail folder by
default.  I play fast and loose with my incoming mail folder, and
If I ever want to refer to the pristine mailing list archive, I go
look that over.

> Hmm.  The way I've written this so far is that the subscriber test happens
> prior to the rbl check.  I figured that this would be much faster than
> the reverse since the rbl check requires name lookups over the internet
> via a few hosts.

I hadn't thought of it that way.  Your way makes just as much sense.

> This will cause more wear and tear on the hard disk but doesn't it make
> sense to do things like this, in the interests of speed?

Yeah, but putting the RBL's first puts the load on _other people_'s systems,
whereas putting the subscriber check first puts the load on sourceware. :-)

Being realistic, a half a dozen seconds longer delay for either
check is not going to be noticable for the list users.

Jason

Re: ezmlm blocking and mailing list idea

[Christopher Faylor]

On Sat, Nov 25, 2000 at 07:58:58PM -0800, Jason Molenda wrote:
>I probably don't understand what you want to do.  Wouldn't a simple
>
>:0
>* ^TOcygwin@sources.redhat.com
>  cygwin-list-mail
>
>do it?  This would cause all mail sent to cygwin@sources to go in
>to the cygwin-list-mail folder in your mail directory (and not into
>your incoming mail box).  You could add another entry to handle
>the @sourceware addr, or you can probably combine both with a
>logical OR expression of some kind.

Yes.  It would.  Right now I am also doing filtering on one of the
"List-" headers.  I guess I could just filter on the TO.  I have
problems with this method when people Bcc things to me.  That is not an
issue for sourceware lists but I was thinking I'd try to come up with a
generic method for dealing with all of the lists that I am subscribed
to.

>I have a bunch of lists where I save copies of all e-mail to a
>dropbox (use ":0 c" instead of ":0" to save a copy to the dropbox),
>and most of them I also have put in my incoming mail folder by
>default.  I play fast and loose with my incoming mail folder, and
>If I ever want to refer to the pristine mailing list archive, I go
>look that over.

I do the same thing.

>>Hmm.  The way I've written this so far is that the subscriber test
>>happens prior to the rbl check.  I figured that this would be much
>>faster than the reverse since the rbl check requires name lookups over
>>the internet via a few hosts.
>
>I hadn't thought of it that way.  Your way makes just as much sense.
>
>>This will cause more wear and tear on the hard disk but doesn't it make
>>sense to do things like this, in the interests of speed?
>
>Yeah, but putting the RBL's first puts the load on _other people_'s
>systems, whereas putting the subscriber check first puts the load on
>sourceware.  :-)
>
>Being realistic, a half a dozen seconds longer delay for either check
>is not going to be noticable for the list users.

Ok.  I think I've gotten everything working.  Now I just have to take a
deep breath and check it in.  Even if I have screwed something up, the
worse that should happen is an increase in spam.  It shouldn't cause
any erroneous bounces.

I think I will change the rblcheck program so that it again defaults to
using relays.orbs.org rather than outputs.orbs.org, too.  We just had
some spam in the cygwin mailing list which would have been caught by
relays but was passed by outputs.

cgf

Re: ezmlm blocking and mailing list idea

[Ian Lance Taylor]

   Date: Sat, 25 Nov 2000 19:26:24 -0800
   From: Jason Molenda <jason@molenda.com>

   I've always had a beef with schemes that try to get a Group-Reply to work
   correctly.  The Reply-To header is the most obvious and worst of the bunch.
   The clearest problem of these (beyond the fact that the subscriber may not
   be subscribed to the list) is when a note goes to _two_ mailing lists.
   The Reply-To style scheme will only reply to one of those.  And if you're
   subscribed to both and have a procmailrc that supresses duplicates (don't
   we all), it'll be random which Reply-To you get.

For an alternative, see http://cr.yp.to/proto/replyto.html .

You should see a Mail-Followup-To header in this mail message.

Ian

Re: ezmlm blocking and mailing list idea

[Christopher Faylor]

On Sat, Nov 25, 2000 at 07:26:24PM -0800, Jason Molenda wrote:
>> It is pretty straightforward to do, but then I started thinking that
>> since we have to do this for every submission to the mailing lists
>> anyway it would be sort of interesting to add an additional field like:
>> 
>> List-Poster-Is-Subscribed: yes
>> 
>> to the header.
>
>You'd probably want to prefix it with "X-", and the "List-" headers
>are defined in an RFC, so I'd avoid any resemblance to that namespace.
>"X-Poster-Is-Subscribed:"?

Thanks.  I forgot that List- was in the RFC.

>It won't work when the person is subscribed as cgf@cygnus.com but
>sends mail from cgf@redhat.com or cgf@rtl.cygnus.com, but I bet
>it'll correctly identify the majority of senders.

Yeah.  The "subscribers allowed" scheme will also cause problems for
people who use a From: address that is not the address used to subscribe
to the list, too.

I wonder if I'm opening myself up to a new postmaster@sources.redhat.com
request:

"I understand that you can add my alternate address so that I can send
email to your mailing list?  Would you mind doing that for me?  I can't
change my From address because I use Lotus Notes 1.47." (repeat)

>I don't know if any list users would ever make good use of this
>header, but I've often added little features to sourceware where
>I'm probably the only one using them.  :-)  I say go for it.

I've read the ezmlm documentation and I can't find any way to add a dynamic
header like this, so I'm not 100% certain that it can be done.

>> That would enable you to know when it was necessary to Cc someone when
>> you are responding to the list and when it was not necessary.
>
>I've always had a beef with schemes that try to get a Group-Reply to work
>correctly.  The Reply-To header is the most obvious and worst of the bunch.
>The clearest problem of these (beyond the fact that the subscriber may not
>be subscribed to the list) is when a note goes to _two_ mailing lists.
>The Reply-To style scheme will only reply to one of those.  And if you're
>subscribed to both and have a procmailrc that supresses duplicates (don't
>we all), it'll be random which Reply-To you get.

Yeah, I use the Reply-To header in the cygwin mailing list because otherwise
I'd have 57 messages in my inbox every day rather than in my cygwin folder.
I've been meaning to play with procmail to see if I can come up with a scheme
for forcing anything that goes to a mailing list into its proper folder if it
has been Cc'ed to me.  You're right that the most popular method for eliminating
duplicates doesn't allow you to prioritize things, so this is a problem.

>PS- One downside to appending this header: If you're only checking the
>subscriber list for spam blocking, you only need to get the full
>subscriber list if their MTA is RBL'ed.  If you want to add the header,
>you're going to need to get the subscriber list for every note.  I
>don't think this is such a big deal for our site -- even the busiest
>list is only ~100 mail notes a day, so a few additional seconds to get
>the subscriber list isn't a big deal.  But I thought I'd point it out.

Hmm.  The way I've written this so far is that the subscriber test happens
prior to the rbl check.  I figured that this would be much faster than
the reverse since the rbl check requires name lookups over the internet
via a few hosts.

This will cause more wear and tear on the hard disk but doesn't it make
sense to do things like this, in the interests of speed?

cgf

Re: ezmlm blocking and mailing list idea

[Christopher Faylor]

On Sat, Nov 25, 2000 at 07:58:58PM -0800, Jason Molenda wrote:
>I probably don't understand what you want to do.  Wouldn't a simple
>
>:0
>* ^TOcygwin@sources.redhat.com
>  cygwin-list-mail
>
>do it?  This would cause all mail sent to cygwin@sources to go in
>to the cygwin-list-mail folder in your mail directory (and not into
>your incoming mail box).  You could add another entry to handle
>the @sourceware addr, or you can probably combine both with a
>logical OR expression of some kind.

Yes.  It would.  Right now I am also doing filtering on one of the
"List-" headers.  I guess I could just filter on the TO.  I have
problems with this method when people Bcc things to me.  That is not an
issue for sourceware lists but I was thinking I'd try to come up with a
generic method for dealing with all of the lists that I am subscribed
to.

>I have a bunch of lists where I save copies of all e-mail to a
>dropbox (use ":0 c" instead of ":0" to save a copy to the dropbox),
>and most of them I also have put in my incoming mail folder by
>default.  I play fast and loose with my incoming mail folder, and
>If I ever want to refer to the pristine mailing list archive, I go
>look that over.

I do the same thing.

>>Hmm.  The way I've written this so far is that the subscriber test
>>happens prior to the rbl check.  I figured that this would be much
>>faster than the reverse since the rbl check requires name lookups over
>>the internet via a few hosts.
>
>I hadn't thought of it that way.  Your way makes just as much sense.
>
>>This will cause more wear and tear on the hard disk but doesn't it make
>>sense to do things like this, in the interests of speed?
>
>Yeah, but putting the RBL's first puts the load on _other people_'s
>systems, whereas putting the subscriber check first puts the load on
>sourceware.  :-)
>
>Being realistic, a half a dozen seconds longer delay for either check
>is not going to be noticable for the list users.

Ok.  I think I've gotten everything working.  Now I just have to take a
deep breath and check it in.  Even if I have screwed something up, the
worse that should happen is an increase in spam.  It shouldn't cause
any erroneous bounces.

I think I will change the rblcheck program so that it again defaults to
using relays.orbs.org rather than outputs.orbs.org, too.  We just had
some spam in the cygwin mailing list which would have been caught by
relays but was passed by outputs.

cgf

Re: ezmlm blocking and mailing list idea

[Jason Molenda]

On Sat, Nov 25, 2000 at 10:37:13PM -0500, Christopher Faylor wrote:

> >I don't know if any list users would ever make good use of this
> >header, but I've often added little features to sourceware where
> >I'm probably the only one using them.  :-)  I say go for it.
> 
> I've read the ezmlm documentation and I can't find any way to add a dynamic
> header like this, so I'm not 100% certain that it can be done.


One nice thing about ezmlm is that it's always easy to add on lots of
grody hacks. :-)  You could prefix the one or two critical lines in 
the DIR/editor file, so

|ezmlm-send ....

goes to

|/usr/sourceware/bin/add-user-in-list-header.sh | ezmlm-send ...

Or what have you.  Man that's nasty.  I don't think it'd be worth it, but
it is possible.  Another way would be to modify ezmlm itself to modify
the mail note.  

> Yeah, I use the Reply-To header in the cygwin mailing list because otherwise
> I'd have 57 messages in my inbox every day rather than in my cygwin folder.
> I've been meaning to play with procmail to see if I can come up with a scheme
> for forcing anything that goes to a mailing list into its proper folder if it
> has been Cc'ed to me.  


I probably don't understand what you want to do.  Wouldn't a simple

:0
* ^TOcygwin@sources.redhat.com
  cygwin-list-mail

do it?  This would cause all mail sent to cygwin@sources to go in
to the cygwin-list-mail folder in your mail directory (and not into
your incoming mail box).  You could add another entry to handle
the @sourceware addr, or you can probably combine both with a
logical OR expression of some kind.

I have a bunch of lists where I save copies of all e-mail to a
dropbox (use ":0 c" instead of ":0" to save a copy to the dropbox),
and most of them I also have put in my incoming mail folder by
default.  I play fast and loose with my incoming mail folder, and
If I ever want to refer to the pristine mailing list archive, I go
look that over.

> Hmm.  The way I've written this so far is that the subscriber test happens
> prior to the rbl check.  I figured that this would be much faster than
> the reverse since the rbl check requires name lookups over the internet
> via a few hosts.

I hadn't thought of it that way.  Your way makes just as much sense.

> This will cause more wear and tear on the hard disk but doesn't it make
> sense to do things like this, in the interests of speed?

Yeah, but putting the RBL's first puts the load on _other people_'s systems,
whereas putting the subscriber check first puts the load on sourceware. :-)

Being realistic, a half a dozen seconds longer delay for either
check is not going to be noticable for the list users.

Jason

Re: ezmlm blocking and mailing list idea

[Gerald Pfeifer]

On Sat, 25 Nov 2000, Christopher Faylor wrote:
> I was doing the research today to figure out how to add "if you are
> a subscriber it's ok to post" to the rbl-check.sh blocking.

Looks like a rather nice addition!

> List-Poster-Is-Subscribed: yes
>
> That would enable you to know when it was necessary to Cc someone when
> you are responding to the list and when it was not necessary.

I happen to like the current way, because that way replies to messages of
mine are delivered to my (main) inbox, while the lists themselves are
filtered to a folder of their own that I do not check that often (and do
not read as carefully as my inboxes).

But perhaps that's just me, and everyone else prefers not to receive such
copies!

Gerald
-- 
Gerald "Jerry" pfeifer@dbai.tuwien.ac.at http://www.dbai.tuwien.ac.at/~pfeifer/

Re: ezmlm blocking and mailing list idea

[Ian Lance Taylor]

   Date: Sat, 25 Nov 2000 19:26:24 -0800
   From: Jason Molenda <jason@molenda.com>

   I've always had a beef with schemes that try to get a Group-Reply to work
   correctly.  The Reply-To header is the most obvious and worst of the bunch.
   The clearest problem of these (beyond the fact that the subscriber may not
   be subscribed to the list) is when a note goes to _two_ mailing lists.
   The Reply-To style scheme will only reply to one of those.  And if you're
   subscribed to both and have a procmailrc that supresses duplicates (don't
   we all), it'll be random which Reply-To you get.

For an alternative, see http://cr.yp.to/proto/replyto.html .

You should see a Mail-Followup-To header in this mail message.

Ian

ezmlm blocking and mailing list idea

[Christopher Faylor]

I was doing the research today to figure out how to add "if you are
a subscriber it's ok to post" to the rbl-check.sh blocking.

It is pretty straightforward to do, but then I started thinking that
since we have to do this for every submission to the mailing lists
anyway it would be sort of interesting to add an additional field like:

List-Poster-Is-Subscribed: yes

to the header.

That would enable you to know when it was necessary to Cc someone when
you are responding to the list and when it was not necessary.

I know that I could probably write something in mutt to detect this and
do the right thing when I replied to messages.  Would this be useful to
anyone else?

I'm assuming that this should be relatively easy to do with ezmlm but I
haven't yet researched if this is the case.

cgf

Re: ezmlm blocking and mailing list idea

[Jason Molenda]

On Sat, Nov 25, 2000 at 08:30:26PM -0500, Christopher Faylor wrote:
> I was doing the research today to figure out how to add "if you are
> a subscriber it's ok to post" to the rbl-check.sh blocking.

Cool!

> It is pretty straightforward to do, but then I started thinking that
> since we have to do this for every submission to the mailing lists
> anyway it would be sort of interesting to add an additional field like:
> 
> List-Poster-Is-Subscribed: yes
> 
> to the header.

You'd probably want to prefix it with "X-", and the "List-" headers
are defined in an RFC, so I'd avoid any resemblance to that namespace.
"X-Poster-Is-Subscribed:"?

It won't work when the person is subscribed as cgf@cygnus.com but
sends mail from cgf@redhat.com or cgf@rtl.cygnus.com, but I bet
it'll correctly identify the majority of senders.

I don't know if any list users would ever make good use of this
header, but I've often added little features to sourceware where
I'm probably the only one using them.  :-)  I say go for it.


> That would enable you to know when it was necessary to Cc someone when
> you are responding to the list and when it was not necessary.

I've always had a beef with schemes that try to get a Group-Reply to work
correctly.  The Reply-To header is the most obvious and worst of the bunch.
The clearest problem of these (beyond the fact that the subscriber may not
be subscribed to the list) is when a note goes to _two_ mailing lists.
The Reply-To style scheme will only reply to one of those.  And if you're
subscribed to both and have a procmailrc that supresses duplicates (don't
we all), it'll be random which Reply-To you get.


Anyway, with the header you're proposing, the MUA could send the
reply to the To and Cc listings, and suppress the From header.
And that'd certainly be a good step in the right direction.


Jason

PS-  One downside to appending this header:  If you're only checking
the subscriber list for spam blocking, you only need to get the
full subscriber list if their MTA is RBL'ed.  If you want to add
the header, you're going to need to get the subscriber list for
every note.  I don't think this is such a big deal for our site --
even the busiest list is only ~100 mail notes a day, so a few
additional seconds to get the subscriber list isn't a big deal.
But I thought I'd point it out.