2 title: "Fighting Mailman Subscription Spam: Leveling Up"
6 Last week, I blogged about [my efforts to fight mailman subscription spam]({{
7 site.baseurl }}{% post_url 2018-06-02-mailman-subscription-spam %}). Enabling
8 `SUBSCRIBE_FORM_SECRET` as described there indeed helped to drastically reduce
9 the amount of subscription spam from more than 1000 to less than 10 mails sent
10 per day, but some attackers still got through. My guess is that those machines
11 were just so slow that they managed to wait the required five seconds before
14 So, clearly I had to level up my game. I decided to pull through on my plan to
15 write a simple CAPTCHA for mailman (that doesn't expose your users to Google).
16 This post describes how to configure and install that CAPTCHA.
20 ### CAPTCHA configuration
22 This simple CAPTCHA is based on a list of questions and matching answers that
23 you, the site admin, define. The idea is to use questions that anyone who is
24 interested in your site can easily answer. Since most sites are small enough
25 that they are not to be targeted specifically, the bots (hopefully) will not be
26 able to answer these questions. At least for my sites, that has worked so far
27 (I am running with this patch for a week now).
29 The CAPTCHA requires `SUBSCRIBE_FORM_SECRET` to be enabled. Configuration can
30 look something like this:
32 SUBSCRIBE_FORM_SECRET = "<some random string, generated e.g. by [openssl rand -base64 18]>"
34 # This is a list of questions, each paired with a list of answers.
35 ('What is two times six?', ['12', 'twelve']),
36 ('What is the name of this site's blog', ['Ralf's Ramblings']),
42 Right now, the `CAPTCHAS` part of the configuration will not yet do anything
43 because you still have to install my patch. The patch is losely based on
44 [this blog post](https://warmcat.com/2017/08/12/mailman-captcha.html) and was
45 written against Mailman 2.1.23 on Debian 9 "Stretch". If you are using a
46 different version you may have to adapt it accordingly.
48 First of all, create a new file `/usr/lib/mailman/Mailman/Captcha.py` with the
52 from Mailman import Utils
54 def display(mlist, captchas):
55 """Returns a CAPTCHA question, the HTML for the answer box, and
56 the data to be put into the CSRF token"""
57 idx = random.randrange(len(captchas))
58 question = captchas[idx][0]
59 box_html = mlist.FormatBox('captcha_answer', size=30)
60 return (Utils.websafe(question), box_html, str(idx))
62 def verify(idx, given_answer, captchas):
67 if not idx in range(len(captchas)):
69 # Chec the given answer
70 correct_answers = captchas[idx][1]
71 given_answer = given_answer.strip().lower()
72 return given_answer in map(lambda x: x.strip().lower(), correct_answers)
74 This contains the actual CAPTCHA logic. Now it needs to be wired up with the
75 listinfo page (where the subscription form is shown to the user) and the
76 subscription page (where the subscription form is submitted to).
78 Here is the patch for `/usr/lib/mailman/Mailman/Cgi/listinfo.py`:
80 --- listinfo.py.orig 2018-06-03 19:18:30.089902948 +0200
81 +++ listinfo.py 2018-06-10 19:12:59.381910750 +0200
84 from Mailman import mm_cfg
85 from Mailman import Utils
86 +from Mailman import Captcha
87 from Mailman import MailList
88 from Mailman import Errors
89 from Mailman import i18n
91 # drop one : resulting in an invalid format, but it's only
92 # for our hash so it doesn't matter.
93 remote = remote.rsplit(':', 1)[0]
95 + (captcha_question, captcha_box, captcha_idx) = Captcha.display(mlist, mm_cfg.CAPTCHAS)
96 + replacements['<mm-captcha-question>'] = captcha_question
97 + replacements['<mm-captcha-box>'] = captcha_box
99 replacements['<mm-subscribe-form-start>'] += (
100 - '<input type="hidden" name="sub_form_token" value="%s:%s">\n'
101 - % (now, Utils.sha_new(mm_cfg.SUBSCRIBE_FORM_SECRET +
102 + '<input type="hidden" name="sub_form_token" value="%s:%s:%s">\n'
103 + % (now, captcha_idx, Utils.sha_new(mm_cfg.SUBSCRIBE_FORM_SECRET +
106 mlist.internal_name() +
110 And here the patch for `/usr/lib/mailman/Mailman/Cgi/subscribe.py`:
112 --- subscribe.py.orig 2018-06-03 19:18:35.761813517 +0200
113 +++ subscribe.py 2018-06-03 20:35:00.056454989 +0200
116 from Mailman import mm_cfg
117 from Mailman import Utils
118 +from Mailman import Captcha
119 from Mailman import MailList
120 from Mailman import Errors
121 from Mailman import i18n
122 @@ -144,13 +147,14 @@
123 # for our hash so it doesn't matter.
124 remote1 = remote.rsplit(':', 1)[0]
126 - ftime, fhash = cgidata.getvalue('sub_form_token', '').split(':')
127 + ftime, fcaptcha_idx, fhash = cgidata.getvalue('sub_form_token', '').split(':')
131 + ftime = fcaptcha_idx = fhash = ''
133 token = Utils.sha_new(mm_cfg.SUBSCRIBE_FORM_SECRET +
136 mlist.internal_name() +
138 if ftime and now - then > mm_cfg.FORM_LIFETIME:
141 _('There was no hidden token in your submission or it was corrupted.'))
142 results.append(_('You must GET the form before submitting it.'))
144 + captcha_answer = cgidata.getvalue('captcha_answer', '')
145 + if not Captcha.verify(fcaptcha_idx, captcha_answer, mm_cfg.CAPTCHAS):
146 + results.append(_('This was not the right answer to the CAPTCHA question.'))
147 # Was an attempt made to subscribe the list to itself?
148 if email == mlist.GetListEmail():
149 syslog('mischief', 'Attempt to self subscribe %s: %s', email, remote)
152 Finally, the HTML template for the listinfo page needs to be updated to show the CAPTCHA question and answer box.
153 On Debian, the templates for enabled languages are located in `/etc/mailman/<lang>`.
154 The patch for the English template looks as follows:
156 --- /usr/share/mailman/en/listinfo.html 2018-02-08 07:54:28.000000000 +0100
157 +++ listinfo.html 2018-06-03 20:35:10.680275026 +0200
160 <mm-digest-question-end>
162 + <TD BGCOLOR="#dddddd">Please answer the following question to prove that you are not a bot:
163 + <mm-captcha-question>
165 + <TD><mm-captcha-box></TD>
169 <center><MM-Subscribe-Button></center>
173 If you have other languages enabled, you have to translate this patch
176 That's it! Now bots have to be adapted to your specific questions to be able to
177 successfully subscribe someone. It is still a good idea to monitor the logs
178 (`/var/log/mailman/subscribe` on Debian) to see if any illegitimate requests
179 still make it through, but unless you site is really big I'd be rather surprised
180 to see bots being able to answer site-specific questions.
182 **Update:** With Mailman 2.1.30, this patch is now included upstream.
183 The `CAPTCHAS` format is slightly different than above to support multiple languages; consult the Mailman documentation for further details. **/Update**