| 1 |
From: whoever sends this
|
| 2 |
Subject: Bits from the listmasters
|
| 3 |
|
| 4 |
Hi everyone,
|
| 5 |
|
| 6 |
As you might have already read from several posts on debian-devel or
|
| 7 |
debian-user, lists.debian.org has a new spam filter setup. This was
|
| 8 |
done during the listmaster@ + owner@bugs meeting in Essen, Germany.
|
| 9 |
But to review everything in a single message:
|
| 10 |
|
| 11 |
Internals
|
| 12 |
---------
|
| 13 |
|
| 14 |
* Amavis-Setup
|
| 15 |
The new spam filter setup of lists.debian.org includes the use of
|
| 16 |
amavisd-new. We are using a feature called policy-banks, where we
|
| 17 |
have grouped all 180 mailing lists into the following policy banks
|
| 18 |
plus a few more administrative ones:
|
| 19 |
|
| 20 |
* bug * lang-greek
|
| 21 |
* en-ht * lang-hu-fi
|
| 22 |
* en-lt * lang-indic
|
| 23 |
* lang-arabic * lang-indonesic
|
| 24 |
* lang-asian * lang-romanic
|
| 25 |
* lang-esperanto * lang-scandinavic
|
| 26 |
* lang-french * lang-slavic
|
| 27 |
* lang-germanic
|
| 28 |
|
| 29 |
Each policy bank has its own spam filtering setup. Most of it can be
|
| 30 |
looked at, as it is checked into svn[1]. To find out to which policy
|
| 31 |
bank a list belongs, look for the X-Virus-Scanned header in the email.
|
| 32 |
|
| 33 |
The advantage of this new setup is that now we can distinguse between
|
| 34 |
different list types, and can set filters and scorings for each
|
| 35 |
list type on its own.
|
| 36 |
|
| 37 |
Mails to each list can be "ham", "maybe-spam" and "spam". For
|
| 38 |
borderline messages (maybe-spam) we are currently implementing a
|
| 39 |
queueing mechanism, which allows us to delay these mails for a while
|
| 40 |
and on recheck them after a defined time has passed.
|
| 41 |
|
| 42 |
Gandalf
|
| 43 |
~~~~~~~
|
| 44 |
Don Amstrong is currently implementing a new greylisting daemon we
|
| 45 |
want to use on lists.debian.org. You now might ask, why another
|
| 46 |
greylisting daemon? We were inspired by the sort of postfix-weight is
|
| 47 |
working, but think it has some design flaws. Also we consider some
|
| 48 |
reporting mechanism back from spamassassin back into the greylisting
|
| 49 |
daemon quite helpful. Stay tuned, as we want this feature going live
|
| 50 |
rather soon.
|
| 51 |
|
| 52 |
Considering of lurker as webfrontend
|
| 53 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
| 54 |
The listmaster team currently evaluating lurker as official
|
| 55 |
additional webfrontend for the mailing list archive. A few
|
| 56 |
show-stoppers have been found and documented in [2]. These have been
|
| 57 |
forwarded to the lurker upstream who is also Debian developer. We
|
| 58 |
hope to have these changes implemented rather soon, so we can also
|
| 59 |
offer lurker as an official web archive.
|
| 60 |
|
| 61 |
SVN on Alioth
|
| 62 |
~~~~~~~~~~~~~
|
| 63 |
We have moved some non-confidential configuration files to an SVN
|
| 64 |
repository on Alioth, including our SpamAssassin and amavisd-new
|
| 65 |
configuration. It can be viewed here[1]. If you want to help us with
|
| 66 |
spam filtering, see if you can improve the SpamAssassin rule files.
|
| 67 |
Send patches to them to listmaster@lists.debian.org.
|
| 68 |
|
| 69 |
Team members
|
| 70 |
~~~~~~~~~~~~
|
| 71 |
There has been quite a bit of restructuring of listmaster team
|
| 72 |
members, since we last sent out an official bits from the
|
| 73 |
listmasters. New members were added, some old members left the team.
|
| 74 |
All of them did tremendous good work as listmasters and we want to
|
| 75 |
thank them. Our thanks goes to Jaakko Niemi, Anand Kumria, Frans
|
| 76 |
Pop, Robert McQueen and Cesar Mendoza.
|
| 77 |
|
| 78 |
Recently we have also added three more team members, Don Armstrong,
|
| 79 |
David Moreno Garza and Thomas Viehmann, last one doing listarchives
|
| 80 |
only.
|
| 81 |
|
| 82 |
Clean up
|
| 83 |
~~~~~~~~
|
| 84 |
We use smartlist[3] for running the lists. Smartlist consists of a
|
| 85 |
series of C programs, procmail and shellscripts. The setup was
|
| 86 |
deployed originally sometime in 1998 (judging from some file
|
| 87 |
timestamps) and since then it has evolved. Currently we have 180
|
| 88 |
lists and each of it has more than 30 files that define how it works
|
| 89 |
(maxsize, moderation, ...). That sums up to more than 6000 files we
|
| 90 |
have to maintain.
|
| 91 |
|
| 92 |
So now we have been cleaning up and linking identical files
|
| 93 |
together, reducing the differing configuration files to ideally one
|
| 94 |
file per list.
|
| 95 |
|
| 96 |
During the listmaster meeting this progress started, and about 1000
|
| 97 |
files have been linked together. The process of simplifying and
|
| 98 |
unifying configurations is still in process.
|
| 99 |
|
| 100 |
There are also some spam filter remnants in these configurations,
|
| 101 |
that are also being moved into the spamassassin-config.
|
| 102 |
|
| 103 |
whitelist
|
| 104 |
~~~~~~~~~
|
| 105 |
While it is possible to post with an address which isn't subscribed
|
| 106 |
to the lists, we recommend that you subscribe to our white-list
|
| 107 |
(http://lists.debian.org/whitelist/) so that our system recognizes
|
| 108 |
you. This will reduce the risk of false positives causing your mail
|
| 109 |
to be dropped.
|
| 110 |
|
| 111 |
Cooperation between bugs and lists
|
| 112 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
| 113 |
During the meeting in Essen Don Amstrong and Robert Blarson from
|
| 114 |
bugs.debian.org team were present. This helped quite a lot, as we were
|
| 115 |
able to ease the configuration of the spamfilters on both sides, so we
|
| 116 |
are now using mostly the same SpamAssassin config, which should
|
| 117 |
improve the spam situation on both sides.
|
| 118 |
|
| 119 |
How to help listmasters against spam
|
| 120 |
------------------------------------
|
| 121 |
|
| 122 |
* If you notice a spam in the list archives, press the 'Report As
|
| 123 |
Spam'-Button.
|
| 124 |
|
| 125 |
* If you run some spam-protecting mechanisms like
|
| 126 |
+ greylisting
|
| 127 |
+ tdma (challenge-response system)
|
| 128 |
+ virus scanner
|
| 129 |
+ spamscanner
|
| 130 |
|
| 131 |
make sure that it doesn't reject mails from murphy.debian.org (and
|
| 132 |
master.debian.org), as our bounce detection software is likely to
|
| 133 |
unsubscribe you. From the figures above you can see that we filter
|
| 134 |
a lot of spam and malware, but as we run a
|
| 135 |
posting-is-open-for-everyone policy, there will always new kinds of
|
| 136 |
junk that will pass our filters.
|
| 137 |
|
| 138 |
* Do not ever report spam received through our lists to third parties
|
| 139 |
services. They are likely to blacklist us or complain to our ISP,
|
| 140 |
both of which result in degraded performance for yourself and
|
| 141 |
others. This is also likely to cause tension between us, our
|
| 142 |
sponsors and their ISPs.
|
| 143 |
|
| 144 |
* Report spam that gets to you through our filters to
|
| 145 |
report-listspam@lists.debian.org. Please leave all the headers
|
| 146 |
untouched. The best method is to bounce (as in mutt) them. There is a
|
| 147 |
plugin for thunder^Wicesomething to do that at
|
| 148 |
http://mailredirect.mozdev.org/ . DO NOT do this automagically. If
|
| 149 |
you want to help us, you must make personally sure that the things
|
| 150 |
you report are REALLY spam.
|
| 151 |
|
| 152 |
* If you receive lots of spam and know how to stop it through
|
| 153 |
procmail or spamassassin, send us (listmaster@lists.debian.org) a
|
| 154 |
note with the recipe, or contact us in OFTC #debian-lists
|
| 155 |
|
| 156 |
* If you really want to use some kind of auto-responder, make sure
|
| 157 |
that it is sane, and interprets the Lists and Precedence-Headers
|
| 158 |
correctly so it ignores our mails. If we find that your mail
|
| 159 |
address issues automatic responses to the list or subscribers,
|
| 160 |
we'll unsubscribe you from all lists.
|
| 161 |
|
| 162 |
* Don't subscribe to our lists with a forwarding mail address, if something
|
| 163 |
goes wrong with the mail address you are forwarding to, it will be harder
|
| 164 |
for us to find out exactly which address we have to drop. Instead, please
|
| 165 |
subscribe with the address on which you will be reading the mail. You are
|
| 166 |
free to send responses with another address, so your receiving
|
| 167 |
address isn't published.
|
| 168 |
|
| 169 |
* Please keep in mind that the Mails to our public lists are
|
| 170 |
publically archived at lists.debian.org and many other services
|
| 171 |
on the net. This means that everything in your mail is public,
|
| 172 |
including your sending mail address.
|
| 173 |
|
| 174 |
[1] http://svn.debian.org/wsvn/pkg-listmaster/
|
| 175 |
[2] http://wiki.debian.org/Teams/ListMaster/LurkerProblems
|
| 176 |
[3] http://packages.debian.org/smartlist
|
| 177 |
|
| 178 |
# vim: tw=72
|
Parent Directory
| 
Revision Log