Introduction to our Challenge-Response e-mail system
What is a challenge-response e-mail system? It is an anti-spam system which
is designed to shift some of the filtering work from the recipient to the
spammer (or the legitimate sender). The main idea is that spammers
will not take the time to confirm that they want to send you email, but a
legitimate sender will. The system maintains two lists of addresses: a
"blacklist" of senders that will always be blocked, and a "whitelist" of
senders that will never be blocked. If someone sends you email from an
address not listed in either list, they will get an "challenge" (and their
message will be queued temporarily). If they give the correct "response" to
the challenge, they get added to your white list and their queued message(s)
get forwarded to you.
Our implementation
Our implementation of Challenge-Response (C/R) has a number of features.
The two most significant enhancements are the ability to see the list of
queued messages, and a special "warn" mode that the C/R system can operate
in.
The queue display allows you to see what is in the queue, and to approve
(whitelist), reject, deliver, or delete queued messages. This
means you do not have to wonder if a message is stuck in the C/R system.
The "warn" mode allows you to shift the filtering burden back to yourself.
This can be desirable for several reasons. This was conceived of as a way to
allow you to turn on C/R without affecting folks that may have been sending
you mail for years. Some of them might be surprised and ignore the
challenge. In "warn" mode, the system will NOT challenge the sender, but
will instead queue their message and send you an alert. This way you can
build up your whitelist without discouraging folks from e-mailing you. Later
you can turn the system from "warn", to "on" and hopefully forget that there
are spammers out there!
Another conceivable use of "warn" mode would be for screening offensive
mail. If a parent were to control the whitelist and queue functions (which
are password protected) they can consider their child's mailbox to be quite
safe. (Please note: no system can guarantee perfect filtering, see
Weaknesses below.)
Message Queue
Remember that messages from senders not on the white or black lists get queued
while the system waits for the challenge to be delivered and the response to
come back? These messages are stored in the "message queue". As soon as a
challenge is responded to, the messages are delivered and removed from the
queue. Since the point of the system is to filter spam, many challenges will
not get responded to. In time, messages will time out and be deleted from
the queue.
Alternatively, you can look at the message queue and make decisions yourself
instead of waiting for the sender to answer the challenge.
The summary queue display has two buttons: Accept and Reject, and a list of
message senders with a check box beside each one of them. You can whitelist
a group of senders by checking the box next to their email address and
hitting "Accept".
Note that messages which are recognized as being from a mailing list,
and for which the list owner has not been white-listed,
will be queued without being challenged. This is a "good netiquette"
comprimise to save big list owners from being deluged with
challenges.
If you click on the 'detail' button beside each sender, you can get a little
more detail on the queued messages (if there is more than one message), and
you get two more options 'deliver' and 'delete'. The deliver and delete
buttons do not affect the white or black lists. The 'deliver' button will
deliver the checked messages to you and remove them from the message queue.
The 'delete' button simply removes the messages from the queue.
Queue Time-outs
We cannot queue messages forever. The system uses a flexible set of rules
and deletes the oldest queued messages first. The current goal is to queue
messages for 30 days. Messages may get pushed out of the queue early if
there are more than 500 messages queued, or if there are more than 20
megabytes of storage being used by the queued messages. The system may allow
more disk space in attempt to keep messages for a minimum of 7 days. (FIXME,
check this.)
Weaknesses
The fundamental basis of the C/R system is the sender's email address.
Unfortunately senders can trivially forge email addresses, and we have seen
cases where a spammer knew what address to forge in order to be able to send
to a mailing list. The same thing could happen in a C/R system.
Recommendation: If you are using C/R, do not blind cc yourself, and do not
whitelist yourself. It is fairly common for spammers to forge a message that
has the same from address as the to address.
The other weakness is in how complex it is to answer the challenge. We have
chosen to start easy and plan to make it more difficult if and when
required. The complex challenges used in other systems are a challenge
message with image attachments and requiring the sender to go to a web page
and key in the text from the image. (This could be simplified by showing the
image on the web page, we'll do that if needed).
The current challenge we
use is simply a message with a specially formatted from address. Hitting
reply and then send in almost any mail client should generate a successful
response. The challenge is setup in such a way that bounces are ignored
(since they could be a temporary failure report.)
Notes
Regarding looping challenges: this shouldn't happen, the only two addresses
shown in one of our challenges are the bounce address (ignored) and the
acceptance address (which won't generate a response). So, any response to
one of our challenges should _not_ cause another challenge.
And we only generate one challenge per day for each unknown sender, so
we shouldn't be in danger of causing a mail loop.
The system isn't perfect. It would be difficult, awkward, and expensive to
make it perfect. We will continue to improve it, but perfection will have
to wait.
