13
13
schemes, which can be adjusted to trade in speed and memory performance for
14
14
statistical sophistication. <b>dbacl</b>(1) also permits the user to select cost weightings
15
15
for different categories, thereby permitting simple adjustments to the
16
type I and type II errors (a.k.a. false positives, etc.).
16
type I and type II errors (a.k.a. false positives, etc.). Finally,
17
<b>dbacl</b>(1) can print confidence percentages which help decide when
18
a decision is ambiguous.
19
21
is a general purpose text classifier which can understand email message formats.
20
22
The <a href="http://www.lbreyer.com/dbacltut.html">tutorial</a> explains general classification only. It is worth reading (of course :-), but doesn't describe the extra steps necessary
21
to enable the email functionality. This document describes the necessary switches and caveats.
23
to enable the email functionality. This document describes the necessary switches and caveats from first principles.
23
25
You can learn more about the dbacl suite of utilities (e.g <a href="http://www.lbreyer.com/dbaclman.html">dbacl</a>,
24
26
<a href="http://www.lbreyer.com/bayesolman.html">bayesol</a>,
40
42
You must make sure that the <i>$HOME/mail/notspam</i> folder doesn't contain any unwanted messages, and
41
43
similarly <i>$HOME/mail/spam</i> must not contain any wanted messages. If you mix messages in the two folders, <b>dbacl</b>(1) will be somewhat confused, and its classification accuracy will drop.
46
If you've used other Bayesian spam filters, you will find that <b>dbacl</b>(1) is slightly different. While other filters can sometimes learn incrementally, one message at a time,
47
dbacl always learns from scratch all the messages you give it, and only those. dbacl is optimized for learning a large number of messages quickly in one go, and to classify messages as fast as possible afterwards. The author has several good reasons for this choice, which are beyond the scope of this tutorial.
43
50
As time goes by, if you use <b>dbacl</b>(1) for classification, you will probably set up your filing system so that messages identified as <i>spam</i> go automatically into the <i>$HOME/mail/notspam</i> folder,
44
51
and messages identified as <i>notspam</i> go into the <i>$HOME/mail/notspam</i> folder.
45
52
<b>dbacl</b>(1) is far from perfect, and can make mistakes. This will result in messages going to the wrong folder. When <b>dbacl</b>(1) relearns, it will become slightly confused and over time its ability to distinguish <i>spam</i> and <i>notspam</i> will be diminished.
48
55
and if you find messages in the wrong folder, you must move them to the correct folder before relearning. If you keep your mail folders clean for learning, <b>dbacl</b>(1) will eventually make very
49
56
few mistakes, and you will have plenty of time to inspect the folders once in a while. Or so the theory goes...
58
Since <b>dbacl</b>(1) must relearn categories from scratch each time, you will
59
probably want to set up a <b>cron</b>(1) job to relearn your mail folders
60
every day at midnight. This tutorial explains how to do this below.
61
If you like, you don't need to relearn periodically at all. The author relearns
62
his categories once every few months, without noticeable loss. This works
63
as long as your mail folders contain enough representative emails for training.
66
Last but not least, if after reading this tutorial you have trouble to get
67
classifications working, please read <a href="is_it_working.html">is_it_working.html</a>.
51
70
<h2>Basic operation: Learning</h2>
53
72
To learn your <i>spam</i> category, go to the directory containing your
59
78
This reads all the messages in $HOME/mail/notspam one at a time, ignoring certain mail
60
headers and attachments, and also removes HTML markup.
79
headers and attachments, and also removes HTML markup. The result is a binary
80
file called <i>spam</i>, which can be used by dbacl for classifications.
81
This file is a snapshot, it cannot be modified by learning extra mail messages.
82
(unlike other spam filters, which sometimes let you learn incrementally,
83
see below for discussion).
62
85
If you get warning messages about the hash size being too small,
63
86
you need to increase the memory reserved for email tokens. Type:
80
103
parsed. The man page lists them all, but of particular interest are the <i>-T</i>
81
104
and <i>-e</i> options.
83
If your email isn't kept in mbox format, you can list each email separately on the command line.
106
If your email isn't kept in mbox format, <b>dbacl</b>(1) can open one or more directories
107
and read all the files in it.
84
108
For example, if your messages are stored in the directory <i>$HOME/mh/</i>, one file per email,
87
% dbacl -T email -l spam $HOME/mh/*
111
% dbacl -T email -l spam $HOME/mh
90
Note however that you should be certain that only RFC822 messages are contained in this directory,
91
and you might run into shell command line limitations if you have a very large number of emails.
92
Perhaps a better (but not necessarily faster) solution is to temporarily
113
At present, <b>dbacl</b>(1) won't read the subdirectories, or look at the file names
114
to decide whether to read some messages and not others.
115
Another (but not necessarily faster) solution is to temporarily
93
116
convert your mail into an mbox format file and use that for learning:
95
118
% find $HOME/mh -type f | while read f; \
99
122
It is not enough to learn <i>$HOME/mail/notspam</i> emails, you must also learn the
100
<i>$HOME/mail/notspam</i> emails. <b>dbacl</b>(1) can only choose among the categories it learns. It
101
cannot say that an email is unlike <i>spam</i>, only that an
102
email is like <i>spam</i> or like <i>notspam</i>. To learn
123
<i>$HOME/mail/notspam</i> emails. <b>dbacl</b>(1) can only choose among the categories which have been previously learned. It
124
cannot say that an email is unlike <i>spam</i> (that's an open ended statement), only that an
125
email is like <i>spam</i> or like <i>notspam</i> (these are both concrete statements). To learn
103
126
the <i>notspam</i> category, type:
105
128
% dbacl -T email -l notspam $HOME/mail/notspam
125
148
All you get is the name of the best category, the email itself is consumed.
126
If you would like to see scores for each category, type:
149
A variation of particular interest is to replace -v by -U, which gives
150
a percentage representing how sure dbacl is of printing the correct category.
152
% cat email.rfc | dbacl -T email -c spam -c notspam -U
156
A result of 100% means dbacl is very sure of the printed category, while
157
a result of 0% means at least one other category is equally likely.
158
If you would like to see separate scores for each category, type:
128
160
% cat email.rfc | dbacl -T email -c spam -c notspam -n
129
161
spam 232.07 notspam 229.44
132
The winning category always has the score closest to zero. In fact, the numbers returned with
133
the <i>-n</i> switch are practically distances towards each category.
164
The winning category always has the smallest score (closest to zero). In fact, the numbers returned with
165
the <i>-n</i> switch can be interpreted as unnormalized distances towards each category from the input document, in a mathematical space of high dimensions.
134
166
If you prefer a return code,
135
167
<b>dbacl</b>(1) returns a positive integer (1, 2, 3, ...)
136
168
identifying the category by its position on the command line. So if you type:
151
183
<i>STDERR</i>. It is always worth rehearsing the operations you intend to script, as
152
184
<b>dbacl</b>(1) will let you know on
153
185
<i>STDERR</i> if it encounters problems during learning. If
154
you ignore warnings, you will likely end up with suboptimal classifications.
186
you ignore warnings, you will likely end up with suboptimal classifications,
187
because the dbacl system prefers to do what it is told predictably,
188
rather than stop when an error condition occurs.
156
190
Once you are ready for spam filtering, you need to handle two issues.
158
192
The first issue is when and how to learn.
160
You should relearn your categories whenever you've received an appreciable number of emails.
161
A category model normally doesn't change dramatically if you add a single new email (provided the
162
original model depends on more than a handful of emails).
163
The simplest strategy is a <b>cron</b>(1) job run once a day:
194
You should relearn your categories whenever you've received an appreciable number of emails or whenever you like. Unlike other spam filters, dbacl cannot
195
learn new emails incrementally and update its category files. Instead, you
196
must keep your messages organized and <b>dbacl</b>(1) will take a snapshot.
198
<i>This limitation is actually advantageous in the long run, because it forces you to
199
keep usable archives of your mail and gives you control over every message
200
that is learned. By contrast, with incremental learning you must remember
201
which messages have already been learned, how many times, and whether to unlearn
202
them if you change your mind.</i>
204
A dbacl category model normally doesn't change dramatically if you add a single new email (provided the
205
original model depends on more than a handful of emails). Over time, you
206
can even stop learning altogether when your error rate is low enough.
207
The simplest strategy for continual learning is a <b>cron</b>(1) job run once a day:
165
209
% crontab -l > existing_crontab.txt
225
269
<h2>Advanced operation: Costs</h2>
271
<i>This section can be skipped. It is here for completeness, but probably
272
won't be very useful to you, especially if you are a new user.</i>
227
274
The classification performed by <b>dbacl</b>(1) as described above is known as a <i>MAP</i>
228
275
estimate. The optimal category is chosen only by looking at the email contents. What is missing is your input
229
276
as to the costs of misclassifications.
279
This section is by no means necessary for using <b>dbacl</b>(1) for most
280
classification tasks. It is useful for tweaking dbacl's algorithms only.
281
If you want to improve dbacl's accuracy, first try to learn bigger collections
231
285
To understand the idea, imagine that an email being wrongly marked <i>spam</i> is likely to be
232
286
sitting in the <i>$HOME/mail/spam</i> folder until you check through it, while an email wrongly marked <i>notspam</i> will prominently appear among your regular correspondence. For most people, the former case can mean a missed timely communication, while the latter case is merely an annoyance.
287
341
<h2>Advanced operation: Parsing</h2>
343
<i><b>dbacl</b>(1) sets some default switches which should be acceptable
344
for email classification, but probably not optimal. If you like to experiment,
345
then this section should give you enough material to stay occupied, but
346
reading it is not strictly necessary.</i>
289
348
When <b>dbacl</b>(1) inspects an email message, it only looks at certain words/tokens.
290
349
In all examples so far,
291
350
the tokens picked up were purely alphabetic words. No numbers are picked up, or special characters such as $, @, % and punctuation.
332
391
<h2>Cross Validation</h2>
393
<i>This section explains quality control and accuracy testing, but
394
is not needed for daily use.</i>
334
396
If you have time to kill, you might be inspired by the instructions above to write your own learning and filtering shell scripts. For example, you might
335
397
have a script <i>$HOME/mylearner.sh</i> containing
421
483
<a href="http://spambayes.sourceforge.net">spambayes</a>.
422
484
The interface requirements are described in the mailcross(1) manual page.
486
Note that the supplied wrappers can be sometimes out of date for the most
487
popular Bayesian filters, because these projects can change their interfaces
488
frequently. Also, the wrappers may not use the most flattering combinations
489
of switches and options, as only each filter author knows the best way
490
to use his own filter.
424
492
Besides cross validation, you can also test Train On Error and Full Online
425
493
Ordered Training schemes, via the mailtoe(1) and mailfoot(1) commands. Using
426
494
them is very similar to using mailcross(1).
498
The (United States) <a href="http://www.nist.gov">National Institute of Standards and Technology</a> organises an annual conference on text retrieval called
499
<a href="http://trec.nist.gov">TREC</a>, which in 2005 began a new
500
track on spam filtering. A goal of this conference is to develop over
501
several years a set of standard methodologies for evaluating and comparing
502
spam filtering systems.
504
For 2005, the initial goal is to compare spam filters in a laboratory
505
environment, not directly connected to the internet. An identical
506
stream of email messages addressed to a single person
507
is shown in chronological order to all
508
participating filters, which can learn them incrementally and must predict
509
the type of each message as it arrives.
511
The <a href="http://plg.uwaterloo.ca/~trlynam/spamjig/spamfilterjig/">spamjig</a>
512
is the automated system which performs this evaluation. You can download it
513
yourself and run it with your own email collections to test any participating
514
filters. Special <a href="http://www.lbreyer.com/gpl/README.TREC2005.txt">instructions</a> for dbacl can be found in the
515
TREC subdirectory of the source package. Many other open source spam filters
516
can also be tested in this framework.
b'\\ No newline at end of file'
added
removed
doc/email.html
