| 1 |
From: whoever sends this
|
| 2 |
Subject: Bits from the listmasters
|
| 3 |
|
| 4 |
Hi everyone,
|
| 5 |
|
| 6 |
as you might have already read from several posts on debian-devel or
|
| 7 |
debian-user, lists.debian.org has got a new spam filter setup. This was
|
| 8 |
done during the listmaster@ + owner@bugs meeting last week in Essen,
|
| 9 |
Germany. But everything in a row:
|
| 10 |
|
| 11 |
Internals
|
| 12 |
---------
|
| 13 |
|
| 14 |
* Amavis-Setup
|
| 15 |
The new spam filter setup of lists.debian.org includes the use of
|
| 16 |
amavisd-new. We are using a feature called policy-banks, where we
|
| 17 |
grouped all 186 mailing lists into the following policy banks plus a
|
| 18 |
few more administrative once:
|
| 19 |
|
| 20 |
* bug
|
| 21 |
* en-ht
|
| 22 |
* en-lt
|
| 23 |
* lang-arabic
|
| 24 |
* lang-asian
|
| 25 |
* lang-esperanto
|
| 26 |
* lang-french
|
| 27 |
* lang-germanic
|
| 28 |
* lang-greek
|
| 29 |
* lang-hu-fi
|
| 30 |
* lang-indic
|
| 31 |
* lang-indonesic
|
| 32 |
* lang-romanic
|
| 33 |
* lang-scandinavic
|
| 34 |
* lang-slavic
|
| 35 |
|
| 36 |
Each policy bank has it's own spam filtering setup. Most of it can be
|
| 37 |
looked at, as it is checked into svn[1]. To find out to which policy
|
| 38 |
bank a list belongs, look for the X-Virus-Scanned header in the email.
|
| 39 |
|
| 40 |
Advantage of this new setup is now, that we can distinguse between
|
| 41 |
different list types, and can set filters and scorings for each
|
| 42 |
list(type) on it's own.
|
| 43 |
|
| 44 |
Mails to each list can be "ham", "maybe-spam" and "spam". For unsure
|
| 45 |
candidates (maybe-spam) we are currently implementing some queueing
|
| 46 |
mechanism, which allows us to delay these mails for a while and
|
| 47 |
on recheck them after a defined time has passed.
|
| 48 |
|
| 49 |
Gandalf
|
| 50 |
~~~~~~~
|
| 51 |
Don Amstrong is currently implementing a new greylisting daemon we
|
| 52 |
want to use on lists.debian.org. You now might ask, why another
|
| 53 |
greylisting daemon? We were inspired by the sort of postfix-weight is
|
| 54 |
working, but think it has some design flaws. Also we consider some
|
| 55 |
reporting mechanism back from spamassassin back into the greylisting
|
| 56 |
daemon quite helpful. Stay tuned, as we want this feature going live
|
| 57 |
rather soon.
|
| 58 |
|
| 59 |
Considering of lurker as webfrontend
|
| 60 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
| 61 |
The listmaster team currently evaluates lurker as official additional
|
| 62 |
webfrontend for the mailing list archive. A few show-stoppers have
|
| 63 |
been found and documented in [2]. These have been forwarded to lurker
|
| 64 |
upstream who is also Debian developer. We hope to have these changes
|
| 65 |
implemented rather soon, so we can also offer lurker as webarchive
|
| 66 |
officially.
|
| 67 |
|
| 68 |
SVN on Alioth
|
| 69 |
~~~~~~~~~~~~~
|
| 70 |
We moved some uncritical config parts (those which do not contain
|
| 71 |
personal data) to a SVN on Alioth, including our SpamAssassin and
|
| 72 |
amavisd-new configuration. It can be viewed here[1].
|
| 73 |
If you want to help us with spam filtering, see if you can improve the
|
| 74 |
SpamAssassin rule files. Send patches to them to
|
| 75 |
listmaster@lists.debian.org.
|
| 76 |
|
| 77 |
Team members XXXXXX
|
| 78 |
~~~~~~~~~~~~
|
| 79 |
There has been quite a bit of restructuring of team members, since we
|
| 80 |
last sent out an official bits from the listmasters. New members were
|
| 81 |
added, some old members left the team. All of them did tremendous good
|
| 82 |
work as listmasters and we want to thank them. Our thanks goes to
|
| 83 |
liiwi, wildfire, fjp, ...
|
| 84 |
Recently we added three more team members, Don, damog and thom_v
|
| 85 |
|
| 86 |
Clean up
|
| 87 |
~~~~~~~~
|
| 88 |
We use smartlist[2] for running the lists. Smartlist consists of C-programs,
|
| 89 |
procmail and Shell-scripts. The setup was deployed originally sometime in
|
| 90 |
the 1998s (judging from some file timestamps) and since then it has evolved.
|
| 91 |
Currently we have ~180 lists and each of it has more than 30 files that
|
| 92 |
define how it works (maxsize, moderation). That sums up to more than 6000
|
| 93 |
files we have to maintain.
|
| 94 |
|
| 95 |
So now we try to clean up and link identically files together, move
|
| 96 |
differing config-variables ideally to one file per list.
|
| 97 |
|
| 98 |
On the listmaster-meeting this progress has started, and about 1000 files
|
| 99 |
have been linked together. The process of simplifying and unifying
|
| 100 |
configurations is still in process.
|
| 101 |
|
| 102 |
There are also some filters, that should be moved into the
|
| 103 |
spamassassin-config.
|
| 104 |
|
| 105 |
whitelist
|
| 106 |
~~~~~~~~~
|
| 107 |
While it is possible to post with an address which isn't subscribed
|
| 108 |
to the lists, we recommend that you subscribe to our white-list
|
| 109 |
(http://lists.debian.org/whitelist/) so that our system recognizes
|
| 110 |
you. This will make spamassassin and crossassassin like you better.
|
| 111 |
|
| 112 |
Cooperation between bugs and lists
|
| 113 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
| 114 |
During the meeting in Essen Don Amstrong and Robert Blarson from
|
| 115 |
bugs.debian.org team were present. This helped quite a lot, as we were
|
| 116 |
able to ease the configuration of the spamfilters on both sides, so we
|
| 117 |
are now using mostly the same SpamAssassin config, which should
|
| 118 |
improve the spam situation on both sides.
|
| 119 |
|
| 120 |
How to help listmasters against spam
|
| 121 |
------------------------------------
|
| 122 |
|
| 123 |
* If you notice a spam in the list archives, press the 'Report As
|
| 124 |
Spam'-Button.
|
| 125 |
|
| 126 |
* If you run some spam-protecting mechanisms like
|
| 127 |
+ greylisting
|
| 128 |
+ tdma (challenge-response system)
|
| 129 |
+ virus scanner
|
| 130 |
+ spamscanner
|
| 131 |
|
| 132 |
make sure that it doesn't reject mails from murphy.debian.org (and
|
| 133 |
master.debian.org), as our bounce detection software is likely to
|
| 134 |
unsubscribe you. From the figures above you can see that we filter
|
| 135 |
a lot of spam and malware, but as long as we run on the
|
| 136 |
posting-is-open-for-everyone policy, there will always new kinds of
|
| 137 |
junk that will pass our filters.
|
| 138 |
|
| 139 |
* Do not ever report spam received through our lists to third parties
|
| 140 |
services. They are likely to blacklist us or complain to our ISP,
|
| 141 |
both of which result in degraded performance for yourself and
|
| 142 |
others. This is also likely to cause tension between us, our
|
| 143 |
sponsors and their ISPs.
|
| 144 |
|
| 145 |
* Report spam that gets to you through our filters to
|
| 146 |
report-listspam@lists.debian.org. Please leave all the headers
|
| 147 |
untouched. The best method is to bounce (as in mutt) them. There is a
|
| 148 |
plugin for thunder^Wicesomething to do that at
|
| 149 |
http://mailredirect.mozdev.org/ . DON'T do that automagically, if
|
| 150 |
you want to help us, you must make personally sure that the things
|
| 151 |
you report are REALLY spam.
|
| 152 |
|
| 153 |
* If you receive lots of spam and know how to stop it through
|
| 154 |
procmail or spamassassin, send us (listmaster@lists.debian.org) a
|
| 155 |
note with the recipe, or contact us in OFTC #debian-lists
|
| 156 |
|
| 157 |
* If you really want to use some kind of auto-responder, make sure
|
| 158 |
that it is sane, and interprets the Lists and Precedence-Headers
|
| 159 |
correctly so it ignores our mails. If we find that your mail
|
| 160 |
address issues automatic responses to the list or subscribers,
|
| 161 |
we'll unsubscribe you from all lists.
|
| 162 |
|
| 163 |
* Don't subscribe to our lists with a forwarding mail address, if
|
| 164 |
something goes wrong with the mail address you are forwarding to, it
|
| 165 |
will be harder for us to find out exactly which address we should
|
| 166 |
be removing. Instead, please subscribe with the address on which
|
| 167 |
you will be reading the mail.
|
| 168 |
|
| 169 |
[1] http://svn.debian.org/wsvn/pkg-listmaster/
|
| 170 |
[2] http://packages.debian.org/smartlist
|
| 171 |
|
| 172 |
# vim: tw=72
|
Parent Directory
| 
Revision Log