Saturday, 18 August 2007

how did the Debian Games Team got rid of spam

How to get rid of most spam on Alioth mailing lists.

Until some Debian developer implements this idea in the BTS, the mailing list and in PTS, the quantities of spam that people get on the mail addresses they expose in the Debian world, the spam will keep on coming in huge quantities. I don't care that there will always be spam, I care that the amount of spam would be lower or probably nonexistent for one time sending (I sent just one mail to the BTS from two different mail addresses and it was enough to get huge amounts of spam on them).


The Debian Games Team's ML has suffered for a while, back in 2006, from spam bombing.
We thought that the easiest method to get rid of spam is to requires registration in order to be able to send mails to the list.

That was a big mistake because, as some people suspect, mails from the BTS needed to be allowed in and the reporters got an automated reply saying that the mail is waiting for approval, etc, etc - plain rude, people send you bug reports and you say "You are not listed, bla, bla bla". We changed the setting not to send any automatic reply and people were more content.

But still, we had to hand approve or white list valid mails/addresses. That worked for a while, but once the mail ended up on spammer lists, the thing blew out of proportions. We were desperate. We tweaked the settings of the lists in different ways, but in the end we ended up in having such a broken setup that we had to approve white listed addresses.

That did for me. I had to do something. So I started working on the ML setup and basically I did the following things to get rid of spam:
  • anything coming from white listed mail addresses was approved
  • black listed mails (bellsouth is one of those) is rejected (or should be)
  • anything that came from the BTS (there are some nice headers that the BTS puts) and was evaluated as ham with a score of 0 (there is a field for that, too - some field has Spam=No, Score=...) was allowed
  • anything that came from the BTS that was Spam=No with a score above 0 was held for moderation
  • anything else is held for moderation

This solution allowed us to lighten the burden of manually white listing every email address that ever sent valid mails to the BTS and to focus only on real spam.

Try it your self on your lists, you'll be pleasantly surprised.

3 comments:

Anonymous said...

For a better alternative to the mailman moderator interface, you might try maildirman. It exposes the mailman moderator queue as a maildir mailbox, which you can then manipulate via IMAP. So, for instance, you can use a powerful mail client like mutt, with all its filtering, tagging, and mass moving of messages, to manage the moderator queue.

João Carneiro said...

have you tried to hook up greylisting?
it rocks!
brought down spam to it's knees even withou any further filtering!

cheers

João Carneiro

Anonymous said...

yeah, graylisting on Alioth... You'd better ask the Alioth admins.

EddyP