« back to all changes in this revision
Viewing changes to RELEASE_NOTES
- browse files at revision 4
- compare with another revision
- download diff
- download tarball
- view history from revision 4
- Committer: Bazaar Package Importer
- Author(s): Christian Perrier
- Date: 2007-02-24 19:27:53 UTC
- mfrom: (3.1.7 feisty)
- Revision ID: james.westby@ubuntu.com-20070224192753-fvvima53q1jrp34x
Tags: 1:2.4.2-6.1
* Non-maintainer upload to fix pending l10n issues.
* Debconf translations
- Remove extra debian/po/de.po~
- Convert all translation files to UTF-8
- Russian. Closes: #405243
- Spanish. Closes: #408734
- Italian. Closes: #409831
* Add very simple LSB headers to init scripts
* Debconf translations
- Remove extra debian/po/de.po~
- Convert all translation files to UTF-8
- Russian. Closes: #405243
- Spanish. Closes: #408734
- Italian. Closes: #409831
* Add very simple LSB headers to init scripts
- files added:
- JpegTester.pm
- debian/conf
- debian/etc/conf.d
- debian/po
- files removed:
- debian/etc/amavisd.conf
- debian/etc/de_DE
- debian/etc/it_IT
- debian/etc/pt_BR
- files modified:
- AAAREADME.first
- helper-progs/amavis-milter.c
added
removed
RELEASE_NOTES
1
---------------------------------------------------------------------------
2
June 27, 2006
3
amavisd-new-2.4.2 release notes
4
5
6
SUMMARY OF CHANGES:
7
8
- new feature: "pen pals soft-whitelisting" lowers spam score of received
9
replies to a message previously sent by a local user to this address;
10
- new features: added command line options to override certain configuration
11
settings from a config file, see below;
12
- documentation bug fixes, specially on the use of SQL data type TIMESTAMP;
13
- zoo decoder interface routine can now use utility unzoo(1) or zoo(1);
14
15
16
COMPATIBILITY WITH 2.4.1:
17
18
There are no incompatible changes since 2.4.1, but please notice below
19
the fixes to SQL and to LDAP documentation, which may affect you.
20
21
22
BUG FIXES AND CHANGES since 2.4.1:
23
24
- LDAP.schema: add missing LDAP attribute amavisSpamQuarantineCutoffLevel
25
to the list of allowed attributes in objectclass amavisAccount;
26
pointed out by Paolo Cravero;
27
28
- README_FILES/README.sql: added short for-the-impatient sections:
29
* BRIEF MySQL EXAMPLE of a log/report/quarantine database housekeeping
30
* BRIEF MySQL EQUIVALENT EXAMPLE based on time_iso if its type is TIMESTAMPS
31
* BRIEF PostgreSQL EXAMPLE of a log/report/quarantine database housekeeping
32
33
- README.sql PostgreSQL notes: fixed incorrect advice in README.sql which
34
suggested to declare a field msgs.time_iso as TIMESTAMP WITHOUT TIME ZONE
35
instead of the correct TIMESTAMP WITH TIME ZONE.
36
Previous instructions were also contradictory to suggested data type on
37
ALTER TABLE msgs ALTER time_iso. Using inappropriate WITHOUT TIME ZONE
38
when comparing time_iso to now() (which is aware of a time zone) offsets
39
results by a current time zone offset, which unexpectedly includes too
40
many or too few records in maintenance purging operation. A workaround
41
if WITHOUT TIME ZONE continues to be used is to: SET TIME ZONE 'UTC';
42
before purging.
43
44
If you were misled by previous documentation choosing WITHOUT TIME ZONE
45
for time_iso, and decided now to convert it to WITH TIME ZONE, the following
46
clause can convert time_iso to proper universal time by manually providing
47
appropriate time offset:
48
49
ALTER TABLE msgs ALTER COLUMN time_iso
50
TYPE TIMESTAMP WITH TIME ZONE
51
USING time_iso [-+] INTERVAL '[offset]';
52
53
Thanks to Brian Wong for a problem description and advice.
54
55
- README.sql MySQL notes: fixed incorrect advice in README.sql which suggested
56
to declare a field msgs.time_iso as TIMESTAMP instead of the correct
57
TIMESTAMP NOT NULL DEFAULT 0. The "DEFAULT 0" is mandatory to prevent
58
MySQL from overwriting mail reception timestamp with current local time
59
when other fields are updated at the end of processing of a message.
60
Also not to be forgotten: $timestamp_fmt_mysql *MUST* be set to 1 in
61
amavisd.conf with MySQL when msgs.time_iso data type is TIMESTAMP... !
62
63
- README.sql MySQL notes: if using field msgs.time_iso to select records
64
for purging (instead of msgs.time_num), and its data type is TIMESTAMP...
65
(as opposed to CHAR...), one should use function utc_timestamp() in place
66
of now() in the DELETE clause to make it work correctly regardless of
67
time zone. Alternatively, now() can continue to be used, provided that
68
SQL client time zone is set to UTC in the purging SQL script:
69
SET time_zone='+00:00'; thanks to Gary V for investigation;
70
71
- a message with only a header, without empty separator line and with no body,
72
lost the last line of a header on forwarding or writing to quarantine;
73
observed by Elias Oltmanns, reported through Debian bug tracking;
74
75
- header validity checks inappropriately reported 'header field syntax error'
76
as a 'header field too long' (BadHdrLong) instead of BadHdrSyntax;
77
78
- ensure that notification would not be sent if notification template is empty,
79
solving the following problem: when recipient notifications for bad headers
80
is enabled, and a message is spam with bad headers, recipient would receive
81
an empty notification message (because message contents category is spam
82
and recipient notification template for spam is empty); reported by Alex;
83
84
- changed SMTP status code 550 to 554 when rejecting mail contents,
85
the 550 is not envisioned in RFC 2821 as a valid reply code to a "."
86
after data transfer; thanks to Victor Duchovni;
87
88
- fixed case mismatch when storing e-mail address to SQL table maddr,
89
which wasted one unnecessary failed attempt on INSERT;
90
91
- ignore $timestamp_fmt_mysql if SQL database driver (DBD) is not 'mysql';
92
93
- perl taint workaround in lookup_sql() where SQL select clause
94
could become tainted; problem reported by Christer Borang;
95
96
- fixed amavisd.conf-default which stated incorrect default values of
97
keys 'ins_rcp' and 'ins_quar' in %sql_clause; reported by Glenn Sieb;
98
99
- limit reported boost score to three decimal places; long fractions
100
observed by Gary V;
101
102
- Postfix since version 20060610 uses xtext-encoded (rfc3461) strings in
103
XCLIENT and XFORWARD attribute values, previous versions used plain text
104
with neutered special characters - amavisd-new now xtext-decodes value
105
if it looks xtext encoded, and encodes it on sending; the change could
106
affect exotic host names (e.g. with a plus in host name) from broken
107
mailers or DNS; thanks to Ralf Hildebrandt for pointing out the recent
108
change in Postfix;
109
110
- improve regular expressions in the $map_full_type_to_short_type_re list
111
to cope better with different versions of the file(1) utility regarding
112
recognition of various MS executables; based on a problem report by Misha;
113
114
- use stricter suggested regular expression in amavisd.conf for matching
115
CLSID (Class ID extension); previous expression was loose and too easily
116
matched file names with braces in the name; suggested by Martin Schuster
117
through Debian bug tracking;
118
119
- zoo decoder interface routine (do_zoo) can now use utility unzoo(1) or the
120
traditional zoo(1); the unzoo(1) recognizes some additional parameters
121
which makes it more resilient (but still not watertight) against some
122
attempts to hide archive contents or to extract members to unexpected
123
locations, but unfortunately does not recognize all zoo compression schemes
124
("error, LZD not yet implemented"), and the relative modes "-j ./" or "-j X"
125
do not protect against all malicious cases - so it is a mixed blessing.
126
The way amavisd calls zoo(1) (piping members to stdout, which can be slow)
127
avoids some of the security problems with zoo (writing to arbitrary
128
directories), which were probably the main reason for ClamAV project
129
deciding to switch to unzoo(1);
130
131
- zoo sucks, unzoo (v4.4) sucks more: considered, but decided against changing
132
zoo entry in @decoders to ['unzoo','zoo'] in amavisd.conf, as was suggested
133
by G�bor K�vesd�n. It would not necessarily be an improvement (see previous
134
item, misses extracting members from my test cases), so feel free to choose
135
between the two poor choices, I still prefer zoo(1), partly also because it
136
covers cases which clamd decoding misses;
137
138
- internal: saving recipient addresses to SQL table maddr is now done
139
earlier to make information available to pen pals code;
140
141
- explicitly test if SQL 'prepare' silently fails to return a statement
142
handle, just in case;
143
144
- adjusted list of pre-loaded SA modules to cater for SA 3.1.3;
145
146
147
NEW FEATURES:
148
149
- new feature: added command line options which override some configuration
150
settings from a config file (an option to override pid_file suggested by
151
Paul Murphy and G�bor K�vesd�n):
152
153
-d log_level ... overrides $log_level
154
-m max_servers ... overrides $max_servers
155
-L lock_file ... overrides $lock_file (Net::Server serialization)
156
-P pid_file ... overrides $pid_file
157
-H home_dir ... overrides $MYHOME directory
158
-Q quarantine_dir ... overrides $QUARANTINEDIR directory, empty disables
159
-T tempbase_dir ... overrides $TEMPBASE directory
160
-S helpers_home_dir ... overrides $helpers_home directory (SA workplace)
161
-D db_home_dir ... overrides $db_home, empty arg turns off $enable_db
162
-R chroot_dir ... overrides $daemon_chroot_dir, empty avoids chroot
163
-p listen_port_or_socket ... overrides $inet_socket_port as well as
164
$unix_socketname, argument may be a decimal TCP port
165
number, or an absolute path name of a Unix socket;
166
may be specified multiple times: daemon can listen on
167
multiple inet sockets and/or multiple Unix sockets;
168
example: -p 10024 -p 9998 -p /var/amavis/amavisd.sock
169
-V ... shows version and exits
170
-h ... shows version and command line options, then exits
171
172
For completeness, here are remaining options, unchanged from
173
previous versions:
174
-u user ... overrides $daemon_user
175
-g group ... overrides $daemon_group
176
-c config_file ... config file path, may be specified multiple times
177
178
- new feature: "pen pals soft-whitelisting" lowers spam score of received
179
replies (or followup correspondence) to a message previously sent by a
180
local user to this address;
181
182
Pre-requisites:
183
* both the outgoing and the incoming mail must pass through amavisd
184
(although outgoing mail may have spam checks disabled if desired);
185
* SQL logging must be enabled (@storage_sql_dsn) and records should
186
be kept for at least several days (some statistics: 85% of replied mail
187
(or followups) is sent within a week since previous correspondence,
188
60% within one working day, 40% within 8 hours, 20% within 1 hour);
189
* @mynetworks and @local_domains_maps must reflect reality;
190
* the information about client IP address must be available to amavisd,
191
i.e. Postfix XFORWARD protocol extension must be enabled, or AM.PDP+milter;
192
* configuration variable $penpals_bonus_score must be set to a positive
193
value (such as 1.0, increase to 3 or 4 after seeing that it works),
194
zero disables the feature and is a default;
195
* $sql_clause{'sel_penpals'} must contain a SELECT clause (which by
196
default it does, unless overridden by an old assignment to %sql_clause
197
in amavisd.conf);
198
* sender/recipient address pair must exactly match recipient/sender pair of
199
previous correspondence (except for allowed case-changes in domain part),
200
which means that care must be taken when canonical and/or virtual mapping
201
is performed by MTA (such as mapping between internal and external address
202
forms) - if external address forms of local addresses are to be seen by
203
a content filter then canonical mapping (int->ext) must be done *before*
204
filtering and virtual mapping (ext->int) *after*; alternatively, if
205
internal address forms are to be seen by a content filter, then canonical
206
mapping should be done after filtering, and virtual mapping before;
207
see README.postfix, section "TO DO 'VIRTUAL ALIAS' MAPPING AND OTHER
208
POSTFIX CLEANUP PROCESSING BEFORE OR AFTER CONTENT FILTERING?";
209
210
How it works:
211
* SQL logging stores records about all mail messages processed by amavisd,
212
their sender, recipients, delivery status, mail contents (no changes
213
there, this feature was introduced with amavisd-new-2.3.0); for the
214
purpose of pen pals scheme only records with local-domain senders matter;
215
* when a message is received, a SQL lookup against a SQL logging database
216
is performed, looking for previous messages sent in reverse direction,
217
i.e. from a local user (which is now a recipient of the current mail)
218
to the address that is now the sender of the message being processed;
219
A SELECT clause in $sql_clause{'sel_penpals'} is used, which by default
220
only considers records of previous messages that were actually
221
delivered (not rejected, discarded or bounced), and were not infected.
222
SQL lookup returns a timestamp of the most recent such message (if any),
223
the difference (in seconds) between the current time and the timestamp
224
is an 'age' as used in the following formula;
225
* an exponential decay formula calculates score points to be deducted
226
from the SA score:
227
weight = 1 / 2^(age/penpals_halflife)
228
score_boost = -penpals_bonus_score * weight
229
i.e. penpals_bonus_score is multiplied by 1, 1/2, 1/4, 1/8, 1/16, ...
230
at age 0, 1*halflife, 2*halflife, 3*halflife, 4*halflife ...
231
weight is a continuous function of age (actually, in steps of one second);
232
* main configuration variables, members of policy banks:
233
$penpals_bonus_score ... a maximal (positive) score value by which
234
spam score is lowered when sender is known to have previously received
235
mail from our local user from this mail system. Zero or undef disables
236
pen pals lookups, and is a default.
237
$penpals_halflife ... exponential decay time constant in seconds,
238
defaults to 7 days; pen pal bonus is halved for each halflife
239
period since the last mail sent by a local user to the current
240
message's sender;
241
* auxiliary configuration variables, global settings:
242
$penpals_threshold_low ... SA score below which pen pals lookups are
243
not performed to save time, defaults to 1.0; undef lets the threshold
244
be ignored (useful for testing and statistics gathering);
245
$penpals_threshold_high ...
246
when (SA_score - $penpals_bonus_score > $penpals_threshold_high)
247
pen pals lookup will not be performed to save time, as it could not
248
influence blocking of spam even at maximal penpals bonus (age=0);
249
usual choice for value would be kill level or other reasonably high
250
value; undef lets the threshold be ignored and is a default (useful
251
for testing and statistics gathering);
252
253
Caveats / notes / exceptions with "pen pals soft-whitelisting":
254
* pen pals soft-whitelisting aids incoming mail, and internal-to-internal
255
mail, but has no effect on outgoing mail;
256
* if SQL logging was not used so far and you are considering enabling it
257
for a busy site, you would appreciate PostgreSQL 8.1 compared to MySQL,
258
as purging old records seems to be *much* faster than in MySQL 4.1,
259
which could lock down mail processing for an hour or more during a
260
weekly (or daily) purge, as opposed to minutes or seconds;
261
* infected messages are exempted from pen pals checks;
262
* mail with (unadjusted) SA score below $penpals_threshold_low (1 by default)
263
is exempted from pen pals check to save time and lighten the load on SQL;
264
similarly for high score spam which would not have a chance of being
265
'saved' even by a maximal pen pals bonus score;
266
* non-delivery notifications have null sender address, so can not match
267
previous correspondence and can not receive a pen pal bonus;
268
* sender address matching local domains but coming from outside is
269
not trusted and is exempted from pen pals checks; this implies that
270
authenticated local users posting from outside to this MTA do not
271
benefit from pen pals scheme when sending to local recipient - the
272
cleanest solution is to separate MSA from MTA (which also avoids the
273
SpamAssassin SPF & DUL difficulty with trusted networks for such users);
274
* messages from a local user to self are exempted from pen pals check;
275
* outgoing messages (i.e. to non-local recipients) are exempted from
276
pen pals checks to save some time and simplify reasoning (which reverse
277
mail transaction to trust?); assuming that local users rarely send
278
spammy mail, outgoing mail would rarely need help from pen pals checks;
279
* messages received from mailing list typically use ML bounce or admin
280
address (possibly VERPed) as the sending address, so they would not be
281
considered replies to postings to a mailing list from a local user;
282
* underlying assumption is that a local-domains sender address in mail
283
coming from inside can be trusted not to be faked; if this is not the case,
284
an internal user cooperating with a spammer can widen spam tolerance for
285
another internal user (but it probably does not pay off, too much trouble
286
for too little effect);
287
* if a spammer knows or can guess that a local user is frequently sending
288
mail to some address (e.g. a mailing list unprotected by DK or SPF), he
289
can gain few bonus score points by using such sending address in his spam;
290
* there may be multiple MTA+amavisd servers, but all must use the same
291
logging SQL database;
292
* forwarding is compatible with the pen pals scheme;
293
* broken forwarding like suggested for SPF, where sender address is replaced
294
by a forwarding mailbox address is counterproductive; for example: a local
295
user may also have an external mailbox at some remote provider with poor
296
spam protection; forwarding from the remote to a local mailbox is set up
297
and forwarding MTA misguidedly substitutes original sender address with a
298
mailbox address; spam reaching remote mailbox is forwarded to a local site
299
with a sender address rewritten, making it look like it is coming directly
300
from a user's remote mailbox, and benefiting from pen pals bonus of user's
301
previous correspondence with his remote mailbox;
302
303
Testing:
304
* set $penpals_bonus_score initially to a low value such as 1 or 0.5
305
to avoid surprises;
306
* set $penpals_threshold_low and $penpals_threshold_high to undef
307
to perform pen pals checks regardless of the score;
308
* at log level 2 (or higher) search the log for a string "penpals: "
309
(only shows on incoming mail sent by a non-local sender); the log also
310
shows mail_id of the referenced message (previous communication),
311
and Subject header fields of previous and current message;
312
313
Based on a feature request by Aaron P. Martinez, thanks to Gary V for
314
suggestions and prompting and to Michael Scheidell and Richard Bishop
315
for feedback.
316
317
318
---------------------------------------------------------------------------
319
May 8, 2006
320
amavisd-new-2.4.1 release notes
321
322
323
INCOMPATIBLE CHANGE WITH 2.4.0:
324
325
- notification templates incompatibility with 2.4.0 (but not with versions
326
2.3.3 or older): major contents category numbers are renumbered due to a
327
newly inserted category CC_SPAMMY; it affects the use of macro ccat_maj
328
in templates (one field added), and only affect users which provide
329
non-default templates based on 2.4.0 templates; older templates (2.3.3
330
or earlier) are unaffected as they do not use macro ccat_maj;
331
332
333
OTHER CHANGES:
334
335
- revert a change introduced with 2.4.0, which was adding address extensions
336
at CC_SPAM, i.e. when score exceeds kill level. Previously (2.3.3) address
337
extensions were inserted at tag2 level. Implemented by a new mechanism:
338
a new major contents category CC_SPAMMY is inserted just below the CC_SPAM,
339
where CC_SPAMMY is controlled by tag2_level and CC_SPAM continues to be
340
selected at kill_level. Also spam defanging (if enabled) is now activated
341
at CC_SPAMMY and no longer at CC_SPAM (which was on a TODO list for some
342
time); undesired change in 2.4.0 reported and changes tested by Mario Liehr;
343
344
- fixed old nuisance bug (probably present since 2.3.0) when an external
345
decoder program for self-extracting archives (rar/unrar, lha, arj/unarj)
346
is defined but the program does not exist, which resulted in logged
347
non-fatal errors like:
348
run_command: failed to exec SCALAR(0x8598550) lq ...
349
run_command: failed to exec REF(0x85985c8) v -c- -p- -av- -idcdp -- ...
350
run_command: failed to exec ARRAY(0x89e5f0c) l ...
351
No such file or directory
352
reported by Martin Baertl, Maurizio Marini, boka, and Donald Teed,
353
investigated by Gary V;
354
355
- bug fix in a Courier setup: add a missing reset of per-recipient data
356
to prevent previous message check affecting the next one performed by
357
the same process; fix by Martin Orr, reported by Bowie Bailey;
358
359
- the amavisd-new-courier.patch now requires Net::Server version 0.90 or
360
later (preferably 0.93 or later); to use older version of Net::Server
361
please apply the older amavisd-new-courier-old.patch and follow
362
README.courier-old - both will go away with next version of amavisd-new;
363
364
- updated amavisd-new-qmqpqq.patch patch (qmail interface) to work with
365
the current code, by Martin Solciansky, testing by Nicklas Bondesson;
366
367
- fix error handling when a problem occurs during temporary directory cleanup;
368
369
- when defanging mail make a 'Subject' header field be editable by header
370
edits, so that Subject tags like ***UNCHECKED*** can still apply;
371
372
- modify unquote_rfc2821_local so that it appends an '@' as a domain name
373
only if localpart contains '@', so that read_array() can still be
374
used to read a list of networks in CIDR notation; a change in 2.4.0
375
to properly handle addresses like "aaa@bbb" made read_array unsuitable
376
for reading list of networks; pointed out by Petr Vokac;
377
378
- add another round of local($1) declarations as a workaround for already
379
familiar Perl taint bugs, popping up again on some Perl installations;
380
reported by Jaap Struyk; reported symptoms were:
381
Insecure dependency in chown while running with -T switch at
382
/usr/lib/perl5/site_perl/5.8.7/Net/Server.pm line 488
383
Insecure dependency in eval while running with -T switch at
384
/usr/lib/perl5/site_perl/5.8.7/Mail/SpamAssassin/PluginHandler.pm line 91
385
386
- added config variables: @spam_subject_tag3_maps, @spam_tag3_level_maps
387
(and $sa_tag3_level_deflt), which makes it possible to split contents
388
category CC_SPAMMY into two sublevels (minor categories) and give
389
each its of Subject tag text;
390
the "CC_SPAMMY,0" contents category still corresponds to tag2 level,
391
and "CC_SPAMMY,1" contents category corresponds to tag3 level (if defined).
392
Only static maps are available (also members of policy banks), but
393
no corresponding SQL and LDAP attributes are provided. Example:
394
@spam_tag2_level_maps = (5.5);
395
@spam_tag3_level_maps = (12);
396
@spam_subject_tag2_maps = ('***LIKELY*SPAM*** ');
397
@spam_subject_tag3_maps = ('***BLATANT*SPAM*** ');
398
based on suggestion from Benedict White;
399
400
- add LDAP attributes: amavisSpamSubjectTag, amavisSpamSubjectTag2,
401
amavisSpamDsnCutoffLevel, amavisSpamQuarantineCutoffLevel to match
402
equivalent SQL lookup fields; missing amavisSpamQuarantineCutoffLevel
403
noticed by Paolo Cravero;
404
405
- presence of LDAP attributes is now tested with 'defined', and is
406
no longer testing them as Perl booleans;
407
408
- mail_via_bsmtp: storing mail in BSMTP format now saves DSN information,
409
as permitted by RFC 2442;
410
411
- apply the concept of separate timers $child_timeout and $smtpd_timeout
412
as used in a SMTP session to AM.PDP and AM.CL protocols;
413
414
- apply the concept of separate timers $child_timeout and $smtpd_timeout
415
as used in a SMTP session to Courier patch; by Martin Orr;
416
417
- new macros: remote_mta, smtp_response, remote_mta_smtp_response
418
and score_boost available to log and notification templates;
419
420
- enhanced regexp selector macro [~string|regexp|then|else], which can now
421
capture parenthesized regexp subexpressions and make them available
422
as %1, %2, ... %9 to 'then' and 'else' replacements; a copy of the first
423
argument (a string) is available to replacements as %0;
424
425
- extend the semantics of the regexp selector macro, which can now take
426
more than one pair of regexp+then arguments, catering for a nested
427
'if then elseif then elseif then else' structure:
428
[~string|regexp1|then1|regexp2|then2|...|regexpN|thenN|else]
429
430
- enhanced iterator macro, which can now take a long macro name as its
431
first argument, and imply a %x as iterator name;
432
433
- make use of the new macro remote_mta_smtp_response and add it to
434
a default $log_templ, so that a Postfix queue-id of a forwarded
435
message shows up like 'queued_as: DCF2A17B9E4' in the main log entry,
436
facilitating search for a related log entry in a MTA log.
437
In case of a mail split, all the MTA responses would now be
438
shown, e.g.: queued_as: F3DBD17B847/F3DBD17B847/F3DBD17B847
439
(customizable by the use of macros in $log_templ);
440
441
- sophos_savi_internal (SAVI module): don't include errno ($!) in the
442
error message, it may be misleading; reported by Matthias Ivers;
443
444
- internal - programming style: use $b=1 instead of $b++
445
where variable $b is supposed to be a boolean and not a counter;
446
447
448
---------------------------------------------------------------------------
449
April 3, 2006
450
amavisd-new-2.4.0 release notes
451
452
453
The most important changes since 2.3.3 at a glance:
454
455
Delivery status notifications (DSN) are now supported, both as a SMTP
456
protocol extension and in notifications. Header fields like X-Amavis
457
and X-Spam are now prepended to mail header for DomainKeys compatibility.
458
Configuration variables can be chosen based on mail contents category,
459
which is now represented explicitly. A built-in macro expander is enhanced,
460
providing new macros and call types. Added support for passive operating
461
system fingerprinting with the use of p0f, supplying collected information
462
as a header field to SpamAssassin. Provide compatibility with Net::Server
463
0.91 and later.
464
465
466
INCOMPATIBLE CHANGES SINCE 2.3.3:
467
468
- incompatible change when logging or quarantining to SQL: added field
469
'quar_loc' to table 'msgs' to facilitate quarantine release, and added
470
FOREIGN KEY constraint for data consistency and simplified purging;
471
see below for a simple database modification;
472
473
- inserted header fields like X-Amavis-* and X-Spam-* are now _prepended_ to
474
mail header instead of being appended, and occupy position just above the
475
inserted Received header field; this pairing with Received makes it easier
476
to identify which MTA/content filter inserted them, makes it consistent
477
with position of Resent-* header fields as required by RFC 2822, and avoids
478
the possibility of breaking DomainKeys and similar mail signing schemes.
479
SpamAssassin implemented the same change with 3.1.0.
480
To achieve former behaviour, specify: $append_header_fields_to_bottom=1;
481
482
- trailing whitespace is no longer trimmed by default from SQL fields,
483
from LDAP attribute values and from associative array righthand-sides
484
(hash values) as read by read_hash(); see below if trimming is really
485
still needed;
486
487
- SMTP server side: no longer allow e-mail address without enclosing
488
angle brackets in MAIL FROM and RCPT TO smtp commands; such syntax is
489
illegal according to RFC 2821 and RFC 821, no compliant MTA is using it,
490
so the change should not effect anyone (except perhaps sloppy testers);
491
492
- changed defaults for banned & bad header administrator address to:
493
$banned_admin = undef;
494
$bad_header_admin = undef;
495
@banned_admin_maps = (\$banned_admin, \%virus_admin,\$virus_admin);
496
@bad_header_admin_maps = (\$bad_header_admin);
497
In other words, if $banned_admin is left at a default value (undefined),
498
banned admin falls back to %virus_admin or $virus_admin.
499
If $bad_header_admin is left at a default value (undefined), bad header
500
admin has no default, admin notifications for bad headers are not sent;
501
502
503
MAJOR NEW ENHANCEMENTS:
504
505
- support for DSN (RFC 3461) in the SMTP protocol (parameters NOTIFY and ORCPT
506
in ESMTP RCPT commands, parameters RET and ENVID in ESMTP MAIL command), with
507
corresponding updates to Delivery Status Notifications (RFC 3462, RFC 3464);
508
(about a missing support for option ORCPT in Net::SMTP please see
509
http://rt.cpan.org/Public/Bug/Display.html?id=18456 );
510
511
- represent mail contents category more explicitly internally, and provide
512
new configuration variables:
513
%final_destiny_by_ccat %lovers_maps_by_ccat %defang_by_ccat
514
%quarantine_method_by_ccat %quarantine_to_maps_by_ccat
515
%notify_admin_templ_by_ccat %notify_recips_templ_by_ccat
516
%notify_sender_templ_by_ccat %warnsender_by_ccat
517
%hdrfrom_notify_admin_by_ccat %mailfrom_notify_admin_by_ccat
518
%hdrfrom_notify_recip_by_ccat %mailfrom_notify_recip_by_ccat
519
%hdrfrom_notify_sender_by_ccat
520
%admin_maps_by_ccat %dsn_bcc_by_ccat
521
%warnrecip_maps_by_ccat %addr_extension_maps_by_ccat
522
gradually phasing out separate configuration variables for each category;
523
the change is fully backwards compatible, existing variables are referenced
524
through default values of the new variables, and no longer used directly;
525
526
The chain of lookups adhere to the following evaluation sequence
527
for settings with an associated *_by_ccat mechanism:
528
* policy bank chooses a *_by_ccat associative array (by TCP port or
529
by client's IP address (MYNETS));
530
* the most relevant contents type of the message chooses an entry in
531
a _by_ccat associative array; the entry can be a final settings value,
532
or a ref to an array of by-recipient lookup tables (*_maps);
533
mostly for compatibility reasons an entry can also be a ref to CODE,
534
which allows for delayed evaluation through legacy *_maps settings
535
(which may again be members of policy banks);
536
* the chosen list of lookup tables is queried based on recipient address,
537
producing a final setting;
538
539
Note that currently only settings which are applicable _after_ the mail
540
contents type has already been determined, have their associated _by_ccat
541
associative array. Settings like @bypass_spam_checks_maps which need to be
542
evaluated _before_ mail contents is assessed, do not have their associated
543
_by_ccat array;
544
545
- added ability to explicitly kill externally running decoder process or
546
a command-line virus scanner process if running for too long;
547
548
- enhanced built-in macro expander now allows long macro names (previously
549
limited to one character), neutral and active macro calls, dynamically
550
defining macros, new regexp matching built-in macro, proper bookkeeping
551
of quoting levels and speedups achieved by pre-tokenization;
552
details in see README.customize;
553
554
- improved wrapping of inserted header fields, fields in DSN, and in
555
generated text sections of the new notification templates;
556
557
- improved text of notification templates, taking advantage of new macros;
558
559
- compatible with Net::Server 0.90, 0.91, 0.92 and 0.93 by providing
560
workarounds; thanks to Paul Seamons, the author of Net::Server, for his
561
cooperation, the 0.93 solves problems introduced by a change in 0.91
562
(but it remains incompatible with version of amavisd-new 2.3.3 and older);
563
564
- experimental support for passive operating system fingerprinting with
565
the use of externally running utility p0f, supplying collected information
566
as a header field to SpamAssassin, making possible to add rules to score
567
SMTP client hosts based on educated guess about their operating system
568
type and IP distance; see below for details;
569
570
- make variable $myhostname a dynamic variable, member of policy banks,
571
likewise for syslog parameters facility, priority and ident; details below;
572
573
- added config options to enable quarantining (archiving) of clean mail;
574
575
- lots of cleanups and generalizations in the code;
576
577
578
SECURITY:
579
580
- fix insufficient sender address sanitation when storing quarantined or
581
forwarded files as BSMTP files _and_ having a %s in the corresponding
582
*_method template; potential security vulnerability (with limited scope)
583
in versions of amavisd-new 2.3.1, 2.3.2 and 2.3.3 discovered by Thomas
584
Jarosch;
585
586
- recognize result "ms-windows metafile" (or "ms-windows metafont") from a
587
file(1) utility and provide short type 'wmf' for it; added two example
588
rules to amavisd.conf (and amavisd.conf-sample) to block files containing
589
Windows Metafiles, based on US-CERT Alert TA05-362A;
590
591
592
OTHER CHANGES:
593
594
- incompatible change when logging or quarantining to SQL is enabled
595
(as mentioned above, here is a more detailed description of the change):
596
* added column 'quar_loc' to table 'msgs' to store quarantine file name
597
(the same string as in macro %q, normally seen in the main log entry);
598
based on input from Andrew A. Neuschwander, Brian Wong and Craig Herring;
599
* add constraint FOREIGN KEY ... ON DELETE CASCADE to keep database
600
consistent (no orphaned records) and simplify maintenance deletions
601
and possibly speed them up; suggested by Brian Wong;
602
603
The following clause must be executed for upgrading pre-2.4.0
604
amavisd-new SQL schema to the 2.4.0 schema:
605
606
ALTER TABLE msgs ADD quar_loc varchar(255) DEFAULT '';
607
608
The following clause should preferably be executed to take advantage
609
of the ON DELETE CASCADE:
610
611
ALTER TABLE msgrcpt
612
ADD FOREIGN KEY (mail_id) REFERENCES msgs(mail_id) ON DELETE CASCADE;
613
ALTER TABLE quarantine
614
ADD FOREIGN KEY (mail_id) REFERENCES msgs(mail_id) ON DELETE CASCADE;
615
616
See updated suggested set of DELETE clauses at the end of README.sql.
617
618
The following clause can optionally be used to create an index
619
on field msgs.time_num to speed up deletions;
620
CREATE INDEX msgs_idx_time_num ON msgs (time_num);
621
or if purging is based on field msgs.time_iso instead of of msgs.time_num:
622
CREATE INDEX msgs_idx_time_iso ON msgs (time_iso);
623
624
(compatibility note with pre-releases of 2.4.0: there were added fields
625
msgrcpt.time_num and quarantine.time_num in pre-release versions of 2.4.0,
626
which are now dropped in favor of FOREIGN KEY constraint; these fields
627
are no longer set by the program and should not be relied-on when purging
628
records, they may be removed from tables);
629
630
- solve compatibility issues brought up by changes in file descriptors
631
usage as introduced with Net::Server version 0.91; thanks to Ralph Seichter,
632
Matt Jackson, Jim Knuth and Paul Seamons (the author of Net::Server)
633
for help;
634
635
- bug fix with LDAP lookups: if the LDAP connection to the server drops
636
(i.e. after being idle for some time) amavisd is unable to reconnect;
637
a symptom in the log is: 'do_search: failed again'; a fix by Petr Vokac,
638
and later independently fixed by Matteo Brancaleoni and Mike Hall;
639
problem also reported by Paolo Cravero;
640
641
- bug fix with LDAP amavisBannedRuleNames lookups, failing to looking up a
642
set of banned rules names and referencing them in the users lookup table,
643
like in other lookup tables. The LDAP lookup was returning an array reference
644
for the 'amavisBannedRuleNames' attribute since it was a list (multivalued)
645
and the reference wasn't being dereferenced down the line. The fix is to
646
make the attribute single valued, the value can be a comma-separated list
647
of names. This brings it in-line with SQL lookups which also uses a single
648
field of comma-separated names. The included LDAP.schema and documentation
649
files are fixed accordingly. Also moved the LDAP stuff out of README.lookups
650
into its own README.ldap and updated it accordingly for the banned
651
rules stuff. Fixed by Michael Hall; problem reported by J�r�me Schell,
652
Aury Fink Filho and Brian Wong; thanks also to Jack Stewart and Willi Gruber;
653
654
- bug fix: properly disconnect SMTP session with 421 response if it times out;
655
watchdog timer needs to be nudged during DATA transfer as well; pointed out
656
by Victor Duchovni;
657
658
- introduced new configuration variable $smtpd_timeout (default 8*60 seconds)
659
which controls the amount of time we are willing to wait for slow/idle client
660
during incoming SMTP session before disconnecting a session. Previously the
661
$child_timeout was covering complete elapsed time, both our processing and
662
waiting for client, now the $child_timeout only still limits our processing,
663
and $smtpd_timeout only limits waiting time. With Postfix after-queue setup
664
the $smtpd_timeout should be higher than Postfix setting max_idle (default
665
100s). Some other setups (like a pre-queue setup) may demand substantially
666
higher $smtpd_timeout values; inflexibility pointed out by Martin Schmitt;
667
668
- added ability to kill externally running decoder process or a command-line
669
virus scanner process if running for too long; currently implemented for
670
all command-line virus scanners and for more common and/or more troublesome
671
external decoders: do_unrar, do_unarj, do_uncompress, do_pax_cpio, do_lha,
672
and partly for do_arc, do_zoo; allowed time is calculated as 2/3 of the
673
remaining time (initially at $child_timeout), but at least 10 seconds;
674
675
- use the same timeout calculation as above for calls to SA, taking
676
$sa_timeout instead if that value is bigger than the calculated time;
677
678
- let do_pax_cpio recognize (and ignore) a single character in place of
679
a date in more exotic cases of a pax listing; reported by Ralf Hildebrandt;
680
681
- standards compliance: recognize (and discard) source route in mail address
682
as required by rfc2821;
683
684
- no longer bother to convert addresses like <""@yahoo.com> to <@yahoo.com>,
685
both forms are invalid anyway, and recent versions of Postfix treat them
686
the same. It is probably a good idea to set strict_rfc821_envelopes=yes
687
in main.cf to reject such non-replyable sender addresses straight away,
688
otherwise we end up processing such mail with inability to bounce it when
689
needed, effectively losing it;
690
691
- make address with '@' in the localpart but without a domain (such as
692
<"aaa@bbb"> ) distinguishable from <aaa@bbb> by appending an empty domain
693
('@' only) to the internal (unquoted) address form; also, we used to strip
694
off empty domain on rfc2821-quoting, but this leads Postfix to interpret
695
an address with an '@' in the local part like <"hhh@example.net"> as
696
<hhh@example.net> (subject to the 'resolve_dequoted_address' Postfix
697
setting), which is not what the sender requested (perhaps unintentionally)
698
so we no longer do that. Both measures together, along with the new address
699
parsing code, solve the inconsistency problem reported by Les Ault;
700
701
- fix string_to_mime_entity() to properly split header from body even in some
702
corner cases (empty header or empty body); as a bonus a tiny speedup in
703
template message splitting is gained;
704
705
- mail header of a 'defanged' message should not contain broken original
706
headers (with illegal characters or whitespace lines); now sanitize such
707
header fields; reported by Ivers Matthias;
708
709
- do not fix illegal all-whitespace continuation header lines when writing
710
to quarantine (or when submitting notifications) in order to preserve the
711
original bad header; only fix the header when such message is forwarded or
712
released from a quarantine; masking problem brought up by Michael Scheidell;
713
714
- when quarantining in a Unix-style mbox format, replace null return path
715
in a delimiting 'From ' line with a string 'MAILER-DAEMON', like Postfix
716
and sendmail local delivery agents do, otherwise some mbox-reading
717
clients do not recognize the line as a message delimiter;
718
719
- when quarantining to a mbox file, ">"-escape all /^From / lines, not just
720
the ones following a blank line; this is more universal and does not break
721
on more sloppy mail readers (thunderbird, kmail, mutt and pine); MUAs like
722
elm and mail(1) (the later usually comes with the OS) are more robust,
723
treating as a message delimiter only /^From / lines following a blank line,
724
these did not mind the more compact approach used by amavisd-new so far;
725
726
- new config variable $syslog_ident makes it possible to configure syslog
727
ident string, its value defaults to 'amavis'; suggested by Andrzej Kukula;
728
729
- instead of the old config variable $SYSLOG_LEVEL (default value 'mail.debug')
730
there are now two config variables $syslog_facility and $syslog_priority,
731
defaulting for compatibility to the before-the-dot and after-the-dot
732
substrings of the variable $SYSLOG_LEVEL. The variable $SYSLOG_LEVEL
733
still exists, can still be used, but is considered obsolete;
734
735
- make variables $syslog_ident, $syslog_facility and $syslog_priority
736
dynamic variables, members of policy banks. This makes it possible for each
737
policy bank to use its own specific syslog settings, for example to log
738
to a different file (by using a different syslog facility like 'LOCAL3'
739
for mail originating from inside (policy 'MYNETS')), or to change syslog
740
ident to 'outgoing-amavis' for certain policy bank, or to rise syslog
741
priority for releases from a quarantine. Note that switching syslog_ident
742
or syslog_facility is done only when necessary, and involves closing and
743
reopening syslog connection, which involves some (quite small) cost for
744
each change. Dynamically changing syslog_priority however is for free;
745
746
- make variable $myhostname a dynamic variable, member of policy banks.
747
This makes it possible for each policy bank to exhibit a different identity
748
in notifications, in inserted header fields and in certain log entries.
749
A quick overview of where myhostname may appear: From, Resent-From,
750
Resent-Sender, Message-ID, Resent-Message-ID, X-Amavis-Modified,
751
Reporting-MTA, macro %h, SQL logging in field msgs.host, log identity,
752
and can be incorporated in variables $smtpd_greeting_banner,
753
$smtpd_quit_banner and $hdrfrom_notify_* by specifying a substring
754
${myhostname}, which will be replaced by a current value of $myhostname
755
just before use;
756
757
- new configuration variables for finer control on propagation of DSN options:
758
$propagate_dsn_if_possible and $terminate_dsn_on_notify_success (both part
759
of policy banks). One or the other may be used to hide internal mail routing
760
from outsiders if desired. Although Postfix can be configured to selectively
761
announce or not announce DSN smtp extension based on client address
762
(e.g. announcing it for internal clients but not for the rest of the world)
763
letting DSN options reach a content filter may be desirable because a
764
content filter can provide a more informative delivery status notification,
765
and perhaps more importantly, it can suppress sending a DSN when it suspects
766
the sender address is faked (viruses, high score spam). This means the
767
front-end Postfix smtpd service should not be terminating DSN chain,
768
but unfortunately the second instance of smtpd service at port 10025 can
769
no longer distinguish between internal and external clients, because it
770
only sees an IP address of a content filter. One solution is to turn on
771
the $propagate_dsn_if_possible within policy bank MYNETS, and turn it off
772
globally, e.g.:
773
774
$propagate_dsn_if_possible = 0;
775
776
$policy_bank{'MYNETS'} = {
777
propagate_dsn_if_possible => 1,
778
...
779
};
780
781
Turning off $propagate_dsn_if_possible is exactly equivalent to the case
782
where MTA on the return port (10025) does not announce support for DSN
783
extension to the SMTP protocol. The only difference is that the amavisd-new
784
setting can be controlled more selectively by a policy bank.
785
786
Turning on $terminate_dsn_on_notify_success is similar, but more refined.
787
It tells amavisd not to pass NOTIFY=SUCCESS option when submitting checked
788
mail back to MTA, which lets amavisd behave as if talking to a non-DSN
789
compliant server, so that DSN success notification will be generated
790
by amavisd itself (unless suppressed for other reasons). This is similar
791
to $propagate_dsn_if_possible=0, the difference is that other DSN options
792
(if any) *will* be passed to MTA, so options like NOTIFY=NEVER or RET,
793
ENVID or ORCPT will not be lost. Such behaviour is not strictly by the book
794
(RFC 3461), but is still in its spirit :) Here is the most useful setting:
795
796
$terminate_dsn_on_notify_success = 1;
797
$policy_bank{'MYNETS'} = { terminate_dsn_on_notify_success => 0 };
798
799
or if you prefer this syntax, changing only one key in an existing p.bank:
800
$policy_bank{'MYNETS'}{terminate_dsn_on_notify_success} = 0;
801
802
- new configuration variable $dsn_bcc (also part of policy banks) and a
803
corresponding %dsn_bcc_by_ccat (by-contents-category settings) allows
804
to specify an additional e-mail address to receive a copy (blind-CC)
805
of a delivery status notification (DSN) that is being sent to the
806
envelope sender address; it allows administrator to monitor what
807
DSN messages (and how many) are being sent out by amavisd-new;
808
809
Possible uses:
810
$dsn_bcc = 'admin@example.com'; # applies to all content classes
811
812
or more selectively (overrides $dsn_bcc):
813
%dsn_bcc_by_ccat = (
814
CC_VIRUS, 'admin+infected@example.com',
815
CC_BANNED, 'admin@example.com',
816
CC_BADH, 'admin@example.com',
817
CC_SPAM, undef,
818
);
819
or perhaps:
820
%dsn_bcc_by_ccat = (
821
CC_SPAM, undef,
822
CC_CATCHALL, 'admin@example.com',
823
);
824
825
Note the use of comma as a delimiter (instead of '=>'), as _values_ of
826
CC_* constants must be used as hash keys, not constant _names_ as strings;
827
(the Perl '=>' operator implicitly quotes its left operand);
828
829
- passive operating-system fingerprinting (p0f) support lets SA gain
830
information about SMTP client's operating system and estimated IP distance,
831
and can reduce the number of bounces:
832
833
* find and install the p0f utility: http://lcamtuf.coredump.cx/p0f.shtml
834
or in FreeBSD ports collection as 'net-mgmt/p0f';
835
836
* start a p0f process on the same host where MTA (MX) is running, making
837
it listen only to incoming TCP sessions (to reduce its workload) to the
838
IP address and TCP port (25) where MTA is accepting incoming mail from
839
outside (it doesn't hurt to let it see other traffic too, it just isn't
840
needed); after testing p0f alone and seeing that it works, you may start
841
it up, feeding its output to program p0f-analyzer.pl that comes with
842
amavisd-new package, e.g.:
843
844
p0f -l 'tcp dst port 25' 2>&1 | p0f-analyzer.pl 2345 &
845
846
on multi-homed boxes one may need to specify interface and IP address
847
where MTA is listening, the filter syntax is the same as in tcpdump, e.g.:
848
849
p0f -l -i bge0 'dst host 192.0.2.66 and tcp dst port 25' 2>&1 \
850
| p0f-analyzer.pl 2345 &
851
852
* the program p0f-analyzer.pl reads p0f reports on stdin, keeps a cache
853
for a limited time (10 minutes, configurable) of data about incoming TCP
854
sessions organized by remote IP address, and listens on UDP port 2345
855
(specified as its command line argument) for queries; only queries from
856
allowed IP addresses are accepted and responded to, other queries are
857
silently ignored - configure @inet_acl accordingly, defaults to 127.0.0.1;
858
859
* adding the following line to amavisd.conf, matching the chosen port
860
number to the one specified on the command line to the p0f-analyzer.pl:
861
862
$os_fingerprint_method = 'p0f:127.0.0.1:2345';
863
864
makes amavisd send queries to p0f-analyzer.pl (on the supplied IP address
865
and UDP port number) to collect information about remote SMTP client's OS;
866
collected response is then supplied as a header field when SpamAssassin
867
is invoked; query/response is very quick and imposes no burden on amavisd
868
process nor does its extend its processing time. The $os_fingerprint_method
869
setting is also a member of policy banks to make it more flexible to
870
disable fingerprinting for mail from site's own SMTP clients, e.g:
871
872
$policy_bank{'MYNETS'}{os_fingerprint_method} = undef;
873
874
* one may now add scoring rules to SA local.cf file, e.g.:
875
876
header L_P0F_WXP X-Amavis-OS-Fingerprint =~ /^Windows XP/
877
score L_P0F_WXP 3.5
878
header L_P0F_W X-Amavis-OS-Fingerprint =~ /^Windows(?! XP)/
879
score L_P0F_W 1.7
880
header L_P0F_UNKN X-Amavis-OS-Fingerprint =~ /^UNKNOWN/
881
score L_P0F_UNKN 0.8
882
header L_P0F_Unix X-Amavis-OS-Fingerprint =~ /^((Free|Open|Net)BSD)|Solaris|HP-UX|Tru64/
883
score L_P0F_Unix -1.0
884
885
It is also possible to add score based on estimated IP distance, for
886
example to slightly favorize nearer hosts (this is probably good for Europe
887
or academic/university networks, and possibly less useful elsewhere):
888
889
header L_P0F_D1234 X-Amavis-OS-Fingerprint =~ /\bdistance [1-4](?![0-9])/
890
header L_P0F_D5 X-Amavis-OS-Fingerprint =~ /\bdistance 5(?![0-9])/
891
header L_P0F_D6 X-Amavis-OS-Fingerprint =~ /\bdistance 6(?![0-9])/
892
header L_P0F_D7 X-Amavis-OS-Fingerprint =~ /\bdistance 7(?![0-9])/
893
header L_P0F_D8 X-Amavis-OS-Fingerprint =~ /\bdistance 8(?![0-9])/
894
header L_P0F_D9 X-Amavis-OS-Fingerprint =~ /\bdistance 9(?![0-9])/
895
header L_P0F_D10 X-Amavis-OS-Fingerprint =~ /\bdistance 10(?![0-9])/
896
header L_P0F_D11 X-Amavis-OS-Fingerprint =~ /\bdistance 11(?![0-9])/
897
score L_P0F_D1234 -0.5
898
score L_P0F_D5 -0.5
899
score L_P0F_D6 -0.5
900
score L_P0F_D7 -0.5
901
score L_P0F_D8 -0.5
902
score L_P0F_D9 -0.4
903
score L_P0F_D10 -0.3
904
score L_P0F_D11 -0.3
905
906
* make sure the @mynetworks is configured correctly, otherwise you will be
907
inappropriately penalizing mail from internal hosts running Windows!
908
Other methods to turn off fingerprinting for our own SMTP client hosts
909
is to put $os_fingerprint_method in policy banks, and/or to specify
910
more selective packet filter on the p0f command line;
911
912
* based on statistics, less than 0.7 % of mail coming from external
913
Windows XP -based hosts is ham, yet 20 % of all spam is coming from
914
external Windows XP hosts; amavisd-new suppresses bounces to external
915
Windows XP hosts, reducing bounce pollution. The amavisd-agent utility
916
now provides some additional statistics based on p0f information.
917
918
Some statistics collected from our logs in February 2006:
919
p0f OS guess ham : spam
920
-----------------------------
921
Windows-XP 0.7 % : 99.3 %
922
Windows-2000 5.8 % : 94.2 %
923
UNKNOWN 16.5 % : 83.5 %
924
Linux 58.8 % : 41.2 %
925
Unix 80.3 % : 19.7 %
926
(Unix+Linux 66.5 % : 33.5 %)
927
(ham: mail with score below 3, spam: score above 6)
928
929
930
- new configuration variable $allow_fixing_improper_header_folding (also
931
a member of policy banks) controls fixing improperly folded header fields
932
made up entirely of whitespace (prohibited by RFC 2822) by removing
933
all-whitespace continuation lines; defaults to true for backwards
934
compatibility; fixing such header fields is desirable and can protect
935
less careful mail readers, but may break DomainKeys validation of messages
936
with illegal header, so if this is of concern, one has a choice of turning
937
it off (if local MUAs can't be fixed);
938
939
- make config variable $insert_received_line a member of policy banks;
940
941
- removed mail header (macro %H) from the default template of the plain text
942
part of the virus or banned sender notifications; these headers are available
943
in the third MIME part of the DSN, so having them twice was redundant;
944
945
- new macros: date_unix_utc, date_iso8601_utc, date_iso8601_local,
946
date_rfc2822_local, x-mailer, header_field, ccat_name, ccat_maj, ccat_min,
947
wrap, lc, uc, substr, index, len, incr, decr and a couple of SpamAssassin
948
lookalike macros - see README.customize for details;
949
950
- new macro ccat_min expands to a minor contents category; this makes it
951
possible for a notification template to distinguish (for example) between
952
cases of bad message header: 1=bad MIME, 2=8-bit char, 3=NUL/CR, 4=empty,
953
5=long, 6=syntax error; See also README.customize for the list of currently
954
available macros; see %ccat_display_names for minor ccat numbers currently
955
in use, feel free to add new ones;
956
957
- edited text of default notification templates to make them tidier and
958
more informative; also make a good use of new macros; older templates
959
are still compatible with this version of amavisd-new;
960
961
- new global variable $timestamp_fmt_mysql, defaults to false;
962
specific to MySQL, when logging to SQL is enabled and field msgs.time_iso
963
is declared as TIMESTAMP instead of char(16); setting $timestamp_fmt_mysql=1
964
changes the timestamp format written to field msgs.time_iso to avoid ISO 8601
965
standard delimiter 'T' and trailing timezone 'Z', which MySQL does not like.
966
Don't turn on $timestamp_fmt_mysql when using PostgreSQL!
967
Should not turn on $timestamp_fmt_mysql with MySQL unless msgs.time_iso is
968
of type TIMESTAMP.
969
970
- new config variable $trim_trailing_space_in_lookup_result_fields controls
971
trimming of trailing whitespace from SQL fields, from LDAP attribute values
972
and associative array righthand-sides (hash values) as read by read_hash();
973
disabled by default; turn it on for compatibility with pre-2.4.0 versions.
974
975
Note that trailing spaces may still be trimmed by SQL itself (contrary to
976
SQL-99 specification). Trailing spaces in MySQL version up to and including
977
4.1 are removed from values when stored in a VARCHAR column; this also means
978
that spaces are absent from retrieved values. Starting with MySQL V5.1,
979
trailing spaces are retained when values are stored and retrieved.
980
Investigated by Gary V;
981
982
- treat exit status 2 as a warning when returned by bzip2, gzip and other
983
decompressors handled by do_uncompress; problem with decoding of corrupted
984
bzip2 file reported by Kim Leandersson;
985
986
- when determining file short type, match "Microsoft Cabinet file"
987
result from a file(1) command case-insensitively; reported and
988
patch provided by ap at zip com au via Debian bug tracking system;
989
also recognize "Microsoft Cabinet archive data" as .cab;
990
991
- do_unzip: set attribute 'U' (undecodable) if zip archive fails to be
992
decoded; based on a patch by Oliver Geisen; -- well, on a second thought,
993
perhaps not, this line is now commented out; it flagged too many bounces
994
containing chopped-off ZIP attachments as ***UNCHECKED***;
995
996
- added a minimalistic decoder interface routine to call a command line
997
unpacker from stuffit.com to decode Macintosh StuffIt archives. Not tested
998
extensively, program source is not available for inspection, use at YOUR
999
OWN RISK (and the risk is non-negligible!). If using non-default assignment
1000
to @decoders, the following entry can be added to the list of decoders:
1001
['sit',\&do_unstuff,'unstuff'],
1002
thanks to Oliver Geisen for the suggestion;
1003
1004
- keep X-Spam-Level bar empty if sender is whitelisted;
1005
1006
- untaint recip_score_boost when writing a log report entry to SQL;
1007
'Insecure dependency in parameter 7' could have been reported when
1008
SQL-based score_sender lookup table is used; reported by Jim Knuth;
1009
1010
- treat undefined spam level as 0 for the purpose of comparing it to
1011
tag/tag2/kill levels, e.g. when spam scanning is skipped due to large
1012
message size; this now allows score_sender_maps to push non-checked
1013
messages over a tag2/kill limit by its score boost if desired;
1014
1015
- header fields X-Spam-Status, X-Spam-Score, X-Spam-Level and X-Spam-Flag
1016
in a quarantined message now take into account also the score_sender boost
1017
and white- and blacklisting (using 'any' and 'max' to summarize in case
1018
of multiple recipients);
1019
1020
- in passed and quarantined mail a header field X-Spam-Status now shows
1021
score as an explicit sum of SA score and a by-recipient score_sender boost
1022
(when the boost is nonzero); the X-Spam-Score header field still shows
1023
a sum of both as a single number so as not to confuse MUA filters which
1024
may operate on that header field;
1025
1026
- insert X-Quarantine-ID header field into passed mail if a passed message
1027
was also quarantined (e.g. *_lover or final_*_destiny=D_PASS), suggested
1028
by Pavel Urban;
1029
1030
- when folding is needed for long new or edited header fields, use TAB
1031
instead of a space on fold points; also: take into account the apparent
1032
display size of expanded TABs instead of the actual character count,
1033
to make header look prettier; use of TABs suggested by Debian community;
1034
1035
- improved wrapping of inserted header fields and in generated text sections
1036
of notification templates;
1037
1038
- automatic wrapping of long header lines from notification templates;
1039
1040
- standards compliance: wrap Diagnostic-Code field in message/delivery-status
1041
section of delivery status notifications according to rfc3461 section 9.2;
1042
1043
- added config options to enable quarantining (archiving) of clean mail;
1044
defaults settings are:
1045
$clean_quarantine_method = undef;
1046
$clean_quarantine_to = 'clean-quarantine';
1047
quarantining clean messages is disabled by default; to enable:
1048
$clean_quarantine_method = 'local:clean-%m';
1049
1050
- reports "Blocked TEMPFAIL" instead of "Blocked CLEAN" in case of a
1051
temporary 4xx failure;
1052
1053
- in generated MIME parts (notifications, defanging) replace suggested
1054
file names like 'message.txt', 'dsn_status.txt' and 'header.txt' with
1055
names without an extension, i.e. 'message', 'dsn_status' and 'header';
1056
Reportedly Outlook Express 6.0 (but not Outlook) determines how to present
1057
a MIME part based on its name extension, instead of based on MIME type.
1058
Depriving it of name extensions makes it obey a MIME type. This solution
1059
has been successfully tested with Outlook (5 & 6), Outlook Express (2k, 2k3),
1060
Thunderbird, JavaMail and Squirrelmail. Investigation and suggestion by
1061
Ivers Matthias;
1062
1063
- clamscan AV entry: change test for status 1 by a test for /:.*\sFOUND$/
1064
to prevent system errors like failed load (e.g.: 'ld-elf.so.1: Shared object
1065
"libgmp.so.6" not found', which also produce exit status 1) from causing
1066
all mail to be treated as infected; suggested by Tomasz Kojm in response
1067
to my problem report;
1068
1069
- add three AV entries for avast products (FreeBSD and Linux):
1070
'avast! Antivirus daemon', 'Client/Server Version' and the command-line
1071
avastcmd; kindly provided by Frantisek Mensik, ALWIL Software;
1072
later refined based on testing and feedback from Bill Landry;
1073
1074
- updated regexp in McAfee uvscan entry to cope with spaces in virus name;
1075
reported by and output samples provided by Andreas Schulze;
1076
1077
- updated bdc AV entry: newer versions of BitDefender don't use option --all
1078
any longer; thanks to Max Matslofva, Andreas, Gary V and Bill Landry;
1079
1080
- updated NOD32 AV entry based on NOD32 documentation and advise
1081
from Willi Gruber about adding status code 3 to the list of success values;
1082
1083
- better handle failed decoder attempts when checking an executable file
1084
for self-extracting archive (SFX), avoid decoder 'dry runs' where possible;
1085
1086
- require minimal version 0.32 of Net::LDAP, a subroutine
1087
Net::LDAP::Util::escape_filter_value() is needed; reported by Harry Hoffman;
1088
1089
- allow to specify option 'deref' in calls to Net::LDAP->search to control
1090
dereferencing of aliases to locate the base object for the search; the
1091
default remains 'find' as before. Use a key 'deref' in the default_ldap
1092
hash to specify a different value. The values are those documented
1093
in the Net::LDAP manpage in the search function under the deref section;
1094
(btw, dereferencing in LDAP is supported in both Postfix and Courier);
1095
a patch provided by John Allman;
1096
1097
- added a timing report entry fwd-data-cmd for time it takes MTA to respond
1098
with a status to a DATA command;
1099
1100
- read_hash: trim whitespace off the right-hand side data only if followed by
1101
a comment (#), otherwise keep the rhs as it is;
1102
1103
- include a currently chosen dataset name (dsn) in the log entry
1104
when quarantining to SQL to be able to retrieve a quarantined message
1105
from the correct SQL server; suggested by Cami Sardinha;
1106
1107
- dwell in $tempdir (chdir to) instead of in $TEMPBASE most of the time;
1108
1109
- edit_header() may now be called multiple times for the same header field
1110
to provide iterative header edits - provided mostly for completeness;
1111
1112
- Courier code (the patch) now supports D_BOUNCE and D_DISCARD message
1113
destinies and adding and deleting recipients (due to differing per-recipient
1114
configuration or addr_extension_*_maps); it also supports DSN (RFC 3461);
1115
by Martin Orr;
1116
1117
- prepend (!!) or (!) to log messages at levels below 0 to facilitate
1118
log parsing and make critical messages stand out; based on a patch
1119
by Henrique de Moraes Holschuh (Debian port maintainer);
1120
1121
- silence logging of AM.PDP commands delivery_care_of, tempdir_removed_by
1122
and tempdir when using amavisd-milter; a patch by Petr Rehor;
1123
1124
- silence Perl warnings about uninitialized values;
1125
1126
- optimization: about 15% speedup in macro expansion due to pre-tokenization
1127
of template strings at startup (affects preparing main log entry and each
1128
notification message);
1129
1130
- modified old amavis.c client program to return status 0 (success) even when
1131
LDA command-line arguments are not specified (i.e. when amavisd daemon is
1132
in charge of delivery), and amavisd daemon returns status 99 (=discard);
1133
simplified code in amavisd/check_amcl_policy() to take advantage of it;
1134
1135
- internal/coding: a much needed and appreciated hefty patch from Martin Orr
1136
introduces class Amavisd::TempDir and collects there existing code from
1137
various places dealing with maintaining a temporary directory and files
1138
within; it also makes possible for the Courier interface code to use
1139
this module and avoid duplicating code;
1140
1141
- internal/coding: the Amavis::In::Courier package has been tidied up and split
1142
into multiple methods, of a hopefully manageable length. Several new features
1143
in amavisd-new core code, which required changes to Amavis::In::Courier to
1144
take advantage of, have also been dealt with (in particular MYNETS and
1145
MYUSERS policy banks and SNMP counters); by Martin Orr;
1146
1147
- require minimal version 1.43 of DBI, working last_insert_id is needed;
1148
(actually last_insert_id is no longer needed, but the requirement stays :)
1149
1150
- internal/coding: when logging to SQL avoid a need for last_insert_id()
1151
by doing SELECT after INSERT when adding a new e-mail address to table maddr;
1152
this also avoids a rollback/retry when more than one process tries to insert
1153
the same new address into a database;
1154
1155
- internal/coding: merge subroutines do_spam and do_virus into a single
1156
do_notify_and_quarantine, and use the same code to prepare spam-describing
1157
headers as in subroutine add_forwarding_header_edits_per_recip;
1158
1159
- internal/coding: new $msginfo data object: contents_category, holding
1160
a reference to a sorted (descending order) array of entries, each one
1161
corresponding to one contents category under which a message can be
1162
classified, e.g. it can be both a CC_VIRUS and a CC_BANNED. Supporting
1163
subroutines are: add_contents_category, main_contents_category,
1164
is_in_contents_category, setting_by_contents_category;
1165
thanks for suggestions and feedback to G�rald Macinenti;
1166
1167
The contents_category list is a sorted list of strings, each of the form
1168
"major" or "major,minor", where major and minor are numbers, representing
1169
major and minor category type. Sort order is descending by numeric values,
1170
major first, and subordered by a minor value. When an entry "major,minor"
1171
is added, an entry "major" is added automatically (minor implied to be 0).
1172
A string "major" means the same as "major,0". See CC_* constants for major
1173
category types. Minor category types semantics is specific to each major
1174
category, higher number represent more important finding than a lower number;
1175
1176
- internal/coding: merge sections in sub check_mail dealing with different
1177
content types one-at-a-time into a single section 'decide_mail_destiny',
1178
taking advantage of the new information on contents category, improving
1179
consistency and simplifying code;
1180
1181
- internal/coding: new $msginfo data objects: spam_level, spam_status,
1182
spam_report, autolearn_status, avoiding ugly global variables;
1183
1184
- internal/coding: separate SpamAssassin-specific code from general anti-spam
1185
code - new module Amavis::SpamControl::SpamAssassin; based on suggestion
1186
and patch by Felix Schwarz;
1187
1188
- internal/coding: use File::Spec::catfile to splice full file name from its
1189
components (in mail_to_local_mailbox, more needed); a patch by Felix Schwarz.
1190
Actually, on a second thought, comment that out and revert to previous code:
1191
there are so many other similar cases which were not generalized, that
1192
it makes no sense to generalize (through File::Spec) one percent of them
1193
(and drag-in yet another Perl module), and leave the rest hard-coded;
1194
1195
- collect the most commonly needed header fields into an associative array
1196
$msginfo->orig_header_fields, removing the need to call ensure_mime_entity()
1197
in certain cases, saving on mime decoding when it is not really needed;
1198
1199
- internal/coding: do_log now takes optional arguments, and if they are
1200
present, the message text is treated as a format string to snprintf.
1201
Take advantage of this in most calls to do_log. The message (format) argument
1202
should not be tainted (not enforced), but arguments may be. In the absence
1203
of additional arguments, do_log behaves as before;
1204
1205
- updated README.postfix to explicitly override (just in case) two newer
1206
Postfix options: smtpd_data_restrictions and smtpd_end_of_data_restrictions,
1207
thanks to Noel Jones for the suggestion;
1208
1209
- documentation: updated README.sendmail-dual - added custom rules to reject
1210
unknown users outright; provided by Matej Vela, thanks to Simone Marx;
1211
added a reference to the 'milter-ahead' project info; thanks to Adam Gibson;
1212
1213
- documentation: fixed README.exim_v4 (don't let messages with null
1214
return path get through unchecked), by Igor D'Astolfo;
1215
1216
- documentation: updated README.customize, describing new features of a
1217
built-in macro expander, and describing new macros;
1218
1219
- documentation: updated README.sql, describing new SQL log purging
1220
recommendations, improved PostgreSQL instructions, and the (optional) use
1221
of data type TIMESTAMP in field msgs.time_iso;
1222
1223
1224
---------------------------------------------------------------------------
1225
August 22, 2005
1226
amavisd-new-2.3.3 release notes
1227
1228
1229
Version 2.3.3 is a maintenance release over 2.3.2. Besides fixing known
1230
problems and providing some optimizations, no new features were added.
1231
If using SpamAssassin older than 3.1, an upgrade of either SA to 3.1,
1232
or an upgrade of amavisd-new to 2.3.3 is recommended.
1233
1234
1235
- privacy: add a safety fuse / workaround around calls to SA to detect
1236
SA's failure (in SA versions before 3.1) to catch a failed exec() in a
1237
forked process, which could produce runaway process clones. See SA bug
1238
report #4370. An incident of a mail copy being delivered to unrelated
1239
recipient reported by Joel Nimety;
1240
1241
- privacy: turn warning into a fatal error when a quarantine ID of a message
1242
requested for a quarantine release does not match the requested mail_id;
1243
1244
- security: require minimal version 1.35 of Compress::Zlib to avoid
1245
vulnerability in the zlib compression library;
1246
1247
- the dsn_cutoff_level should have been ignored if undefined according to
1248
documentation, but was not, causing DSN to be suppressed regardless of
1249
spam level; discovered by Gary V;
1250
1251
- ensure the banned check is not performed if all recipients agree
1252
it is not needed, even in presence of $banned_namepath_re;
1253
undesired behaviour (not strictly incorrect) reported by Joel Nimety;
1254
1255
- missing import of lookup_ip_acl in module Amavis::In::AMCL caused
1256
failure in sendmail milter setup when using the new AM.PDP protocol;
1257
reported by Mic And;
1258
1259
- document and explicitly define handling of syntactically invalid IP address
1260
in lookup_ip_acl: it matches a zero-length-mask net, a constant lookup table,
1261
or a hash entry with an undef key, but no other entries in IP lookup tables;
1262
syntactically invalid IP addresses are now logged;
1263
1264
- fix parsing of IPv6 address in $notify_method and $forward_method in case
1265
of dynamic destination override (the use of '*' in method fields);
1266
1267
- check during startup that $myhostname is a fully qualified domain name
1268
(or 'localhost', if you must), and abort if it isn't, otherwise a non-FQDN
1269
can end up in places where RFC 2822 does not allow it; if uname(3) does not
1270
provide a FQDN, then an assignment to $myhostname must be done explicitly
1271
in amavisd.conf;
1272
1273
- when quarantining to a single file in mbox format the 'From ...' line
1274
needs an English date, regardless of current locale; fixed by globally
1275
setting locale LC_TIME to "C";
1276
1277
- pass on the parameter BODY=8BITMIME on MAIL FROM when submitting to MTA
1278
when original message reception indicated it is needed (RFC 1652).
1279
Note that mail forwarding may now fail if the feeding MTA requests
1280
BODY=8BITMIME SMTP service extension (or just passes data with msb set),
1281
but the MTA on the output side does not allow the use of the BODY parameter
1282
in SMTP. In case of Postfix this may only happen when receiving service
1283
on port 10025 is misconfigured and does not announce ESMTP capability
1284
and support for the SMTP service extension 8BITMIME;
1285
1286
- RFC 2554 requires auth_param to be xtext-encoded addr-spec (no angle
1287
brackets) or "<>", not the xtext-encoded addr-spec enclosed in angle
1288
brackets (when specifying submitter during authentication); fixed;
1289
1290
- apply some sanity limit on collected bad-header samples to ensure that
1291
a grossly broken mail does not unnecessarily fill up memory;
1292
1293
- when sending recipient warnings for viruses, banned files, or bad headers,
1294
recipient address must not be rfc2822-quoted twice; fixed;
1295
1296
- fix interpretation of $defang_all to really imply all; previously it only
1297
affected clean messages;
1298
1299
- in quarantined mail the reported spam score in X-Spam-Status header field
1300
now includes maximum of all by-recipient score boosts (less surprising
1301
when soft-whitelisting through @score_sender_maps is in use); suggested
1302
by Mike Cappella and Gary V;
1303
1304
- when a policy delegation protocol attribute "request" is not "AM.PDP"
1305
(perhaps it is a Postfix policy delegation request) don't attempt to find
1306
and open a mail file;
1307
1308
- do_ascii and do_unarj: set environment variable TMPDIR or a command line
1309
temporary directory option to "$tempdir/parts" instead of $TEMPBASE
1310
to minimize possible pollution of top level directory;
1311
1312
- don't abort even if amavisd.conf returns undef as a final value,
1313
as long as there are no errors reading or interpreting it;
1314
1315
- if during 'amavisd stop' or 'amavisd reload' the old running daemon does
1316
not go away for one minute after sending it a SIGTERM, use a bigger
1317
hammer and send it a SIGKILL; suggested by Sven Riedel;
1318
1319
- extend LDAP lookups to allow multiple search attributes (multiple
1320
occurrences of %m in a query); a patch by Michael Hall (and a similar
1321
one by Matthias Bandemer);
1322
1323
- LDAP lookup on an empty envelope address (e.g. a null return path)
1324
adds another lookup key "<>", as it is difficult if not impossible
1325
to have LDAP attributes with empty string as a value; by Michael Hall;
1326
1327
- LDAP.schema: drop "MUST ( mail )" from objectclass 'amavisAccount';
1328
suggested by Michael Hall;
1329
1330
- updated comments and documentation, most notably the README.chroot;
1331
1332
- contributed file Macintosh.tar.gz updated by Dale Walsh;
1333
1334
1335
COMPATIBILITY
1336
1337
- replaced 'hits=' with 'score=' in inserted X-Spam-Status header field
1338
(and in some internal log entries) for compatibility with a changed
1339
default in SpamAssassin 3.1;
1340
1341
- insert X-Spam-Score header field for compatibility with SA (previously
1342
insertion of this header field was commented-out because the information
1343
is redundant, as the score already appears in X-Spam-Status);
1344
1345
1346
OPTIMIZATION
1347
1348
- speed up sending a mail header or full defanged (rewritten) mail over SMTP
1349
back to MTA by a factor of 4 by buffering header fields into large chunks
1350
to avoid bottleneck in Net::Cmd::datasend, which has lots of overhead for
1351
line-by-line writes. Previously slow writes mostly affected mail messages
1352
with extreme header lengths (such as results of a broken mail loop), or
1353
when delivering defanged messages, particularly at sites with large MTA
1354
mail size limits, sometimes to a point of exceeding timeout limits;
1355
reported by Dominik Weber and Ralf Hildebrandt;
1356
1357
- move subroutine lookup_ip_acl() and associated ip_to_vec() into its own
1358
dedicated new package Amavis::Lookup::IP; provide a constructor to pre-parse
1359
IP lookup tables to speed up IP lookups in lookup_ip_acl; prepare pre-parsed
1360
commonly used IP lookup tables (@mynetworks_maps, @publicnetworks_maps,
1361
@inet_acl);
1362
1363
- optimized reading loop in SMTP DATA state, receiving data is now about
1364
35% faster when mail size limit is not enforced (which is a default);
1365
no speedup when mail size limit _is_ enforced;
1366
1367
- cache results of evaluated macros during a single call to expand(),
1368
as macro calls often come in pairs, like: [?%e||\[%e\] ]
1369
or [? %#T ||, Tests: [%T|,]]; together with the above optimization in
1370
pre-parsed IP lookups it shaves off 25% of time in preparing main log entry;
1371
1372
- set locale LC_TIME to "C" globally, avoid changing and restoring locale
1373
for every log write and when generating RFC2822 timestamps;
1374
1375
- added an optimization note in README.sql about indexes and about
1376
SELECT count(*) in MySQL with InnoDB; investigation by Paolo Cravero;
1377
1378
1379
---------------------------------------------------------------------------
1380
June 29, 2005
1381
amavisd-new-2.3.2 release notes
1382
1383
1384
INCOMPATIBILITY with 2.3.1 and earlier versions:
1385
1386
If running amavisd daemon in chroot please note:
1387
1388
Each child process now opens its own syslog connection or a file descriptor
1389
to a log file, and no longer inherits a connection from its parent.
1390
When running in chroot jail and logging to syslog, the syslog client
1391
routines need syslogd socket to be present in the chroot subtree to be
1392
able to establish a connection with syslogd, otherwise logging output
1393
may be lost. Additional syslogd sockets (to be made available in the
1394
jail) may be requested from the syslogd daemon, see its documentation.
1395
This requirement is equivalent to the requirement of chrooted Postfix
1396
services (see Postfix documentation file BASIC_CONFIGURATION_README).
1397
1398
1399
BUG FIXES since 2.3.1:
1400
1401
- do not enforce $MAXFILES limit during top-level MIME decoding to avoid
1402
tempfailing mail; MIME parts are still counted, so a limit exceeded may
1403
still be reported during subsequent decoding, but this is handled more
1404
gracefully and does not cause preserved temporary directories to be left
1405
behind; reported by Marcin Lemanski; suggested by Stephane Lentz and
1406
Robert LeBlanc (noted in the 2.0 release notes);
1407
1408
- use recv() instead of read() to get results from daemonized virus scanners
1409
in an attempt to avoid a bogus Perl I/O status on some Linux installations
1410
(reported by Sander Steffann); we now get a meaningful status codes like
1411
ECONNRESET instead of a bogus EBADF (Bad file descriptor);
1412
1413
- ignore status ECONNRESET when reading results of a daemonized virus scanner
1414
from a socket, specific to some Linux versions; thanks to Sander Steffann
1415
for the initial report and extensive help in debugging the Perl problem;
1416
1417
- run_av and other similar code sections: replace line-by-line reads by
1418
block-by-block reads wherever possible to avoid inappropriate status report
1419
EBADF (Bad file descriptor) caused by Perl I/O bug when last line is not
1420
terminated by a newline. The problem was affecting reading response from
1421
some command line virus checkers; reported by Sander Steffann;
1422
Perl bug tracking: #39060: readline of a not NL-terminated last line
1423
results in Bad file descriptor;
1424
1425
- ignore status EAGAIN when reading results on a pipe from a forked process;
1426
the status EAGAIN seems to be an artifact of Perl I/O on some installations;
1427
reported by several people to cause problems on FreeBSD with Perl 5.8.7
1428
(but Perl 5.8.6 is fine); thanks to Bart Matterne for testing and feedback;
1429
1430
- allow one level of indirection when collecting %needed_protocols;
1431
global setting $protocol='COURIER' did not work, a workaround was needed
1432
with previous version, e.g.: $policy_bank{'QMQPqq'}={protocol=>'QMQPqq'};
1433
reported by Nicklas Bondesson and Martin Orr;
1434
1435
- fix a bug (introduced with 2.3.0) in Courier and QMQPqq setups, where global
1436
information about processed message wasn't always reset and could leak
1437
into processing of a subsequent message; reported by Nicklas Bondesson;
1438
1439
- SQL: fix arguments in calls to last_insert_id(), failing under PostgreSQL
1440
(MySQL didn't mind); pointed out by Henrik Krohns;
1441
1442
- if module SAVI is loaded, insist it is version 0.30 or later;
1443
incompatibility with earlier versions reported by Andrzej Kukula;
1444
1445
- make use of the new Net::Server 0.88 hook run_n_children_hook() to
1446
reload SAVI database; removes a need to apply SAVI patch to Net::Server;
1447
the Net::Server hook was suggested by Paul B. Henson and others,
1448
and incorporated into Net::Server 0.88 by Paul Seamons;
1449
1450
- reopen log file or syslog connection in each child process to make it use
1451
its own file descriptor; also minimizes transients when syslogd is restarted
1452
and its socket re-created, as reported by Les Ault. When running in chroot
1453
please make sure a syslogd socket is also available in the chroot jail,
1454
see README.chroot for syslogd options (and BASIC_CONFIGURATION_README
1455
in Postfix documentation for the Postfix equivalent);
1456
1457
- close log file or syslog in forked process before exec, just to play nicely;
1458
1459
- do_lha: fix extracting archive member filename in case of broken archive
1460
or empty name (avoid interpreting creation date as a file name);
1461
do not increment OpsDecByLha counter for empty archives, which are
1462
most likely not lha archives at all;
1463
1464
- obey $final_bad_header_destiny D_DISCARD or D_REJECT even for messages
1465
with bad headers from mailing lists or with a null envelope sender (DSN);
1466
previously such messages were passed; undesired behaviour reported
1467
by Cami Sardinha.
1468
1469
Such messages are still let through with $final_bad_header_destiny set to
1470
D_BOUNCE, as otherwise they will be lost because a bounce is suppressed
1471
for null sender messages and for mail from mailing list. This behaviour
1472
is retained for backwards compatibility, but may need to be reconsidered.
1473
1474
- fix regexp for extracting am_id from amavis-milter helper program requests;
1475
1476
- if fork/exec fails, try to commit suicide in forked process with
1477
POSIX::_exit(1) first, before trying kill('KILL',$$) as a last resort;
1478
1479
- updated $log_templ example in amavisd.conf-sample to match the default;
1480
pointed out by Gary V;
1481
1482
- further reduce a couple of more frequent Perl warnings about the use of
1483
uninitialized values in expressions;
1484
1485
- pre-load additional Perl modules required by SA 3.1 plugins;
1486
1487
- require minimal versions of modules: Time::HiRes 1.49, Archive::Zip 1.14;
1488
1489
- replaced nonexistent variable @sa_spam_modifies_subj_maps by
1490
@spam_modifies_subj_maps in commented-out example in amavisd.conf-sample;
1491
noticed by Joachim Schoenberg;
1492
1493
1494
LDAP CHANGES by Michael Hall:
1495
1496
All the LDAP changes are transparent to the user.
1497
1498
- rewritten some of the code similar to the restructuring of the SQL code
1499
in version amavisd-new-2.3.0. A new package Amavisd::LDAP::Connection was
1500
added which is a LDAP connection object, and the old connection-related code
1501
in Amavis::Lookup::LDAP has been moved to the new package. Amavisd-new will
1502
now try to reconnect (once) while processing a message, similar to SQL;
1503
1504
- added the ability to specify a '%d' (domain) token in the LDAP base DN;
1505
based on idea from Alexander Wittig;
1506
1507
- updated default LDAP port based on whether SSL/TLS is being used or not;
1508
based on idea from Timo Veith;
1509
1510
- updated the search code to query for multiple records and return the results
1511
sorted in 'make_query_keys' order versus doing a query for each key.
1512
As a result performance is enhanced, and the tweaks 'ldap_get_all', and
1513
'use_query_keys' (recently added) are no longer applicable or needed
1514
and have been removed;
1515
1516
- improved LDAP error reporting and misc changes to multivalued attributes;
1517
1518
- documentation changes (amavisd.conf-default, README.lookups);
1519
1520
1521
MINOR IMPROVEMENTS:
1522
1523
- macro %c (commonly used in a log template) reports spam score no longer
1524
as a single number, but as an explicit sum of a SA score and a by-sender
1525
boost score (from @score_sender_maps) when boost score is nonzero;
1526
suggested by Ed Walker;
1527
1528
- enhancement to amavisd-release: if its only command line argument is '-',
1529
then read arguments from stdin, one release request per line, ignoring empty
1530
lines; input lines have the same format as command line arguments, i.e.:
1531
mail_file
1532
mail_file secret_id
1533
mail_file secret_id alt_recip1 alt_recip2 ...
1534
1535
- better handle cases where a persistent temporary file email.txt
1536
as prepared by the SMTP server module gets replaced as a result
1537
of some user program modification (e.g. when invoking altermime);
1538
problems reported by Dinesh Shah and Leonardo Rodrigues;
1539
1540
1541
---------------------------------------------------------------------------
1542
May 9, 2005
1543
amavisd-new-2.3.1 release notes
1544
1545
1546
INCOMPATIBILITY with 2.3.0 and older versions:
1547
1548
- command line option 'foreground' no longer automatically redirects logging
1549
to STDERR; to request logging to STDERR turn off the $DO_SYSLOG and $LOGFILE;
1550
the improved flexibility suggested by Matthias Andree and Ralf Hildebrandt;
1551
1552
1553
BUG FIXES since 2.3.0:
1554
1555
- don't test errno when line-by-line reading loop is exited before eof,
1556
it was inappropriate and Perl on some versions of Linux does not like it,
1557
possibly reporting "Error reading mail header: Bad file descriptor";
1558
Besides fixing the loops that needed the fix, modified also all remaining
1559
reading loops for consistency. Reported by Ralf Hildebrandt;
1560
1561
- don't call $per_msg_status->get_autolearn_status with SA older than 3.0;
1562
reported by Ian Abbott;
1563
1564
- pre-load some additional SpamAssassin modules, needed by SA 3.1 (CVS);
1565
1566
- reading from dspam process used wrong variable, producing empty lines
1567
for SA checking; reported by Chris Lewicki;
1568
1569
- SAVI-Perl: libsavi option for turning on mime parsing is 'Mime', not 'MIME';
1570
libsavi is case-sensitive since version 3.93.0 and was rejecting incorrect
1571
option name; thanks to Paul B. Henson;
1572
1573
- fixed disabling of SQL wblist ($sql_select_white_black_list=undef);
1574
bug reported by Tom Sommer;
1575
1576
- do_tnef: extract data from attribute 'Attachment' in addition to data from
1577
a more common attribute 'AttachData'; example data provided by Goetz Rieger;
1578
1579
- avoid some more frequent Perl warnings on the use of uninitialized variables;
1580
1581
- add prototypes for decoding subroutines; prototype mismatch warnings
1582
reported by Michael Muenz;
1583
1584
- fixed prototype for add_forwarding_header_edits_common(),
1585
thanks to Ian Abbott;
1586
1587
- replace a simple-minded logic for loading input protocol modules
1588
by a slightly more sophisticated code which takes into consideration
1589
field 'protocol' in all defined policy banks; reported by Brian Wong;
1590
1591
- when replacing existing address extensions don't treat the whole
1592
localpart as an extension if the address happens to start with
1593
a $recipient_delimiter; pointed out by Kaj J. Niemi;
1594
1595
1596
MINOR IMPROVEMENTS:
1597
1598
- unfold obsolete-syntax folded header fields composed entirely of
1599
white space (RFC 2822 section 4.2); suggested by Ian Abbott and others.
1600
Note that such unfolding breaks DomainKeys "simple" canonicalization
1601
algorithm (but is transparent to "nofws") if the affected header is
1602
included in the signature;
1603
1604
- do_tnef_ext: add support for decoding TNEF containers by an external
1605
program 'tnef' if available; selectable by an entry in the @decoders list;
1606
1607
- mail_via_bsmtp enhancement: substitute %s in quarantine filename template
1608
by a defanged sender name; based on a patch by Thomas Jarosch;
1609
1610
- lookup_ip_acl enhancement: when a supplied lookup table is an associative
1611
array (a hash) and the looked-up address is an IPv4 address, allow simple
1612
classful subnet specification by repeatedly truncating the trailing byte
1613
from the looked up IPv4 address until a match is found or until further
1614
truncation is not possible. Note that this does not apply to IPv6 addresses.
1615
For more flexible CIDR subnet specifications please use lookup arrays.
1616
1617
- provide a routine read_array, which is modelled after read_hash, but stores
1618
lines read from a file into an array lookup table, instead of a hash.
1619
1620
1621
---------------------------------------------------------------------------
1622
April 24, 2005
1623
amavisd-new-2.3.0 release notes
1624
1625
1626
QUICK OVERVIEW:
1627
1628
1629
Provides more flexible configuration of decoders. Allows recipients to have
1630
individual banning rules. Assigns a long-term unique id to each message,
1631
reducing clashes and facilitating retrieval of information. The daemon can
1632
store information to a SQL database for logging, reporting and quarantine
1633
retrieval, optionally storing entire message to a SQL database. File-based
1634
quarantine can disperse files to 62 subdirectories. Provides a quarantine
1635
release mechanism. Reconnects to SQL if connection is broken. Can skip
1636
quarantining high-score spam. Compatibility with IPv6-enabled Postfix
1637
is improved.
1638
1639
1640
SECURITY:
1641
1642
- require minimal version 1.05 of Convert::UUlib to avoid a known security
1643
problem in the underlying uulib (likely to be exploitable);
1644
1645
1646
INCOMPATIBILITY with 2.2.1 and older versions:
1647
1648
- structure of @banned_filename_maps config variable has changed in
1649
incompatible way to allow per-recipient banned rules: it is now a
1650
two level map indexed by recipient address (similar to the structure
1651
of @score_sender_maps). See further down for details.
1652
1653
The change will not affect existing installations which either:
1654
* leave @banned_filename_maps at its default value and use the traditional
1655
$banned_filename_re configuration variable to specify banned rules,
1656
which most installations do and remains the most commonly used method;
1657
* or, assign to @banned_filename_maps an empty list to disable it;
1658
* or, use the alternative mechanism $banned_namepath_re and disable
1659
$banned_filename_re or @banned_filename_maps.
1660
1661
- macro %i (used in log and other templates) now always shows mail_id
1662
(see below) which is often but not necessarily also the name of a
1663
quarantined file (like before, available as a macro %q); previously the
1664
%i reflected the actual file name, which was longer/different than now;
1665
1666
- default values of $virus_quarantine_method, $spam_quarantine_method,
1667
$banned_files_quarantine_method and $bad_header_quarantine_method
1668
now specify shorter file names based on the newly-introduced mail_id:
1669
virus-%m, spam-%m.gz, banned-%m and badh-%m respectively;
1670
1671
- a config variable $file now defaults to a string 'file' instead of being
1672
undefined; this makes it unnecessary to be explicitly assigned to in the
1673
configuration file;
1674
1675
- SQL fields virus_lover, bypass_virus_checks and bypass_spam_checks
1676
in table 'policy' are now optional, and if missing their value is treated
1677
as undef (same as if the field is present but is NULL) - instead of being
1678
interpreted as false. This makes it consistent with the interpretation
1679
of other missing fields. The change is unlikely to affect existing
1680
installations, because these fields were considered non-optional in
1681
previous versions.
1682
1683
1684
BUG FIXES or missing functionality:
1685
1686
- avoid modifying directory which is being read by readdir, it can cause
1687
premature termination of the directory traversal; reported by Dale Walsh;
1688
1689
- minimize deep recursion in regexp evaluation while parsing some degenerate
1690
cases of Received header subfields via/with/id/for, which could result
1691
in a process crash;
1692
1693
- turned loops in banned checks inside out to make 'allow' rules useful and
1694
let them behave as one may expect: when checking mail parts against lookup
1695
tables in @banned_filename_maps the evaluation order of checking part's
1696
attributes against rules has changed: on each rule in a list, all attributes
1697
of a part are now checked together; previously all rules were evaluated
1698
(inner loop) for each attribute (outer loop), which made 'allow' rules
1699
hardly ever useful;
1700
1701
- ignore dsn_cutoff_level if undefined, instead of treating it as 0;
1702
1703
- fix generating the positive delivery notifications (requested by
1704
$warn*sender settings): the DSN was missing entirely in case of Courier or
1705
sendmail-milter setups and the chosen template was not the most appropriate;
1706
Courier problem reported by Sander Holthaus;
1707
1708
- fix the text in notification templates 'neutral' and 'virus/banned sender'
1709
to claim non-delivery when the message was truly not delivered, and to
1710
claim delivery for delivered messages;
1711
1712
- when per-recipient subject tag strings are different for each user, the
1713
mail forwarding clustering algorithm should have split the forwarded mail
1714
into separate deliveries but did not do so, resulting in all recipients
1715
of a multi-recipient mail to get the same string inserted into Subject;
1716
reported by Michael Goth;
1717
1718
- at last: when mail is received through LMTP protocol, gracefully handle
1719
a temporary failure 4xx reply from MTA to a RCPT TO command and pass it
1720
back to a LMTP client for tempfailed recipients only, instead of returning
1721
450 for _all_ recipients (needed the sending routine to be aware of the
1722
receiving side capabilities, which was previously not available);
1723
1724
- stricter and more consistent error checking and better error reporting
1725
on Perl read, sysread and getline operations;
1726
1727
- use O_CREAT|O_EXCL when creating files that are not supposed to pre-exist,
1728
to be able to detect a potential race condition and other programming errors;
1729
1730
- fixed reporting of virus names for av scanner Perl-SAVI;
1731
1732
- with Sophos Perl-SAVI module, avoid setting 'MaxRecursionDepth'
1733
if $MAXLEVELS is undefined or zero, matching its semantics to other
1734
uses in the program; debugging and a fix by Paul B. Henson;
1735
1736
- fixed sloppy regexps when parsing SMTP commands;
1737
1738
- fixed a typo in README.lookups leading to confusion between fields
1739
spam_subject_tag and spam_subject_tag2, making one to believe there is
1740
only one such field; pointed out by Max Clark and others;
1741
1742
- handle special case: Postfix hates ""@domain but does not mind @domain;
1743
1744
1745
1746
NEW FEATURES:
1747
1748
- new config variable @decoders (with its policy banks counterpart) makes
1749
it possible to enable/disable decoding of individual file content types
1750
from the configuration file, and to adjust the external decoders paths and
1751
search order, all in one place. This list now makes the following config
1752
variables obsolete: $arc, $gzip, $bzip2, $lzop, $uncompress, $unfreeze, $lha,
1753
$unarj, $unrar, $zoo, $pax, $cpio, $ar, $rpm2cpio, $cabextract, $ripole;
1754
although they are still observed for compatibility reasons if the @decoders
1755
list is left at its default value;
1756
1757
- banned filename/filetype rules can now be specified on a per-recipient basis
1758
1759
Structure of @banned_filename_maps config variable has changed in
1760
incompatible way to allow per-recipient banned rules: it is now a
1761
two level map, similar to the structure of @score_sender_maps.
1762
1763
Lookup keys used at the first level are recipient addresses, results from
1764
this lookup can be either a ref to a list of second-level lookup tables, or
1765
a string which is interpreted as a comma-separated (or whitespace-separated)
1766
list of _names_ of second-level lookup tables. These names are mapped to
1767
actual second-level lookup tables through an associative array %banned_rules.
1768
The indirection by names through %banned_rules is primarily intended for
1769
SQL and LDAP lookups, which can not return complex data structures and
1770
Perl code (and should not do so for security reasons), but names may just
1771
as well be used by static lookups.
1772
1773
The resulting list of second-level lookup tables (which in most cases is
1774
a list containing a single element $banned_filename_re, which makes it
1775
compatible with existing setups) is looked up by using keys such as part
1776
file name and part file type, exactly like in previous versions.
1777
1778
The previous default assignment:
1779
@banned_filename_maps = ( \$banned_filename_re );
1780
is now incorrect, it can be rewritten (if explicit assignment is desired) as:
1781
@banned_filename_maps = ( {'.' => [$banned_filename_re]} );
1782
or (by giving a name to a lookup table):
1783
@banned_filename_maps = ( {'.' => 'MYRULES'} );
1784
%banned_rules = ('MYRULES' => $banned_filename_re);
1785
which is equivalent to the default setting of both variables:
1786
@banned_filename_maps = ( {'.' => 'DEFAULT'} ); # proper two-level struct.
1787
@banned_filename_maps = ( 'DEFAULT' ); # same as previous, but shorter
1788
%banned_rules = ('DEFAULT' => $banned_filename_re);
1789
1790
The SQL table 'policy' may now contain an optional field 'banned_rulenames',
1791
which is a comma-separated (or whitespace-separated) list of _names_ of
1792
second-level lookup tables, with semantics as described above for static
1793
lookups.
1794
1795
The configuration variable @banned_filename_maps is a member of policy banks
1796
as before. The associative array %banned_rules is global and is not a member
1797
of policy banks.
1798
1799
The alternative 'banned file' mechanism $banned_namepath_re hasn't changed
1800
and is still not merged into @banned_filename_maps, which means it can not
1801
be used when per-recipient banned rules are needed. Perhaps in the
1802
next version...
1803
1804
- introduce a concept of 'mail_id', which is similar to the am_id as reported
1805
in the log and elsewhere (e.g. 92598-11-5), but has much stronger long-term
1806
uniqueness property and can be used for the purpose of uniquely identifying
1807
a quarantined mail, or for other uses. The mail_id is a 12-character string
1808
consisting of characters [A-Za-z0-9+-] (like base64, except for a '/' being
1809
substituted by a '+'), guaranteed to start and to end with an alphanumeric
1810
character (i.e. not with '+' or '-'). It is derived by cryptographically
1811
strong method (MD5), cumulatively collecting entropy during the life of
1812
child processes, folding-in entropy from processed mail and other cheaply
1813
accessible sources, collected when an opportunity arises (e.g. file system
1814
file-IDs, SA results etc), without placing a burden on system sources of
1815
randomness (see RFC 4086). Note that MD5 has been demonstrated to have some
1816
weaknesses, but we are not talking about cryptographic attacks here, but
1817
rather about spreading messages which have no inherent intention of causing
1818
collisions.
1819
1820
The mail_id carries 71.9 bits of information (subject to the quality of
1821
sources). For a high-end system that keeps a year's worth of mail messages
1822
in evidence (e.g. in quarantine) and receives 10e6 messages each day
1823
(20..200 TB of yearly mail contents), the probability of a mail_id
1824
collision happening during one year (while gradually displacing an entire
1825
collection with a new set of IDs) is n^2/m = 0.3 %
1826
(10e6 * 365)^2 / (62 * (64^(12-2)) * 62) = (10e6 * 365)^2 / 2^71.9 = 0.003
1827
Eventual clash is still possible and needs to be detected, but a re-tried
1828
mail delivery attempt is acceptable if its probability is low, as each mail
1829
processing rolls a dice again. On a smallish system receiving 10.000 mail
1830
messages daily an 8-character mail_id would suffice, but the savings are
1831
not worth the trouble of providing configuration flexibility.
1832
1833
Paired with a mail_id there is its companion secret_id generated for each
1834
message, such that mail_id can be derived from secret_id and pairing checked,
1835
but not the other way around. The purpose of secret_id is not yet fully
1836
developed, but can serve as a 'ticket', granting user a right to release
1837
a quarantined message addressed to him.
1838
1839
- SQL: can store information about every processed mail to SQL; the information
1840
is similar to level 0 log entries, but more detailed; a SQL database can be
1841
used as a basis for searching for a particular mail, for preparing reports
1842
and to facilitate quarantine management (searching and releasing).
1843
Enabled by configuring the @storage_sql_dsn list which contains information
1844
about a SQL server and dataset name, just like the @lookup_sql_dsn does
1845
for the SQL lookups. If @storage_sql_dsn is the same as the @lookup_sql_dsn,
1846
a single connection to SQL database will be used, otherwise separate and
1847
independent connections are established, possibly to different SQL servers.
1848
Loosely based on ideas from Maia Mailguard by Robert LeBlanc and a patch
1849
by Brian Wong. Thanks to Brian Wong for testing and valuable feedback.
1850
See README.lookups for a SQL schema. See new file amavisd-sql-maintain
1851
(incorporated into README.sql in later versions) for ideas on database
1852
housekeeping (expiring old entries).
1853
1854
- SQL: can quarantine to a SQL database; selected by setting config variables
1855
$*_quarantine_method to 'sql:' The @storage_sql_dsn list of dataset names
1856
is used to choose SQL server and dataset name, and must be nonempty when
1857
$*_quarantine_method is 'sql:'; When $*_quarantine_method is set to 'sql:'
1858
the SQL logging must be enabled as well;
1859
1860
- SQL: clean the mess of needing SQL lookup objects to be aware of each other,
1861
by separating SQL connection information (Amavis::Out::SQL::Connection
1862
object) from objects holding SQL statement handles. Statement handles
1863
are invalidated on reconnect, and are dynamically 'prepared' as needed.
1864
1865
- SQL: thanks to a reorganization of SQL objects an automatic reconnect
1866
to a SQL server is done without temporary failing a processed message;
1867
1868
- SQL: new configuration variable (an associative array) %sql_clause, also
1869
part of policy maps, allows SQL clauses to be switched with policy banks.
1870
The components of %sql_clause under keys 'sel_policy' and 'sel_wblist'
1871
perform the duty of legacy configuration variables $sql_select_policy and
1872
$sql_select_white_black_list. Compatibility with older configuration files
1873
is maintained when %sql_clause is left at its default value;
1874
1875
- can add one layer of 62 subdirectories to the quarantine directory for
1876
more efficient file system use by uniformly distributing quarantined mail;
1877
enabled by setting a new config variable $quarantine_subdir_levels to 1;
1878
1879
- choosing mail_id as a quarantine file name now greatly reduces the likelihood
1880
of two quarantined messages processes by the same child process shortly one
1881
after another from attempting to get quarantined under the same filename,
1882
leading to a temporary failure ("File...already exists, refuse to overwrite")
1883
and leaving behind a preserved temporary directory; reported by
1884
Martin Svensson;
1885
1886
- release from quarantine functionality is now a built-in feature;
1887
a message release can be requested via enhanced AM.PDP protocol. There is
1888
a new utility program 'amavisd-release', which currently mostly serves to
1889
demonstrate how to request releasing of a quarantined file. Currently the
1890
supported quarantine types are: plain file, gzipped plain file with a name
1891
ending in .gz, and a SQL-based quarantine. Currently not supported is a
1892
release from a BSMTP-encoded plain file or from a mbox (Unix-style) mailbox
1893
quarantine file.
1894
1895
Example use:
1896
$ amavisd-release spam/V/V5htXBh0y0Hr.gz H2huh4wfrSyC
1897
1898
or providing a replacement list of recipients:
1899
$ amavisd-release spam/V/V5htXBh0y0Hr.gz H2huh4wfrSyC user@example.com
1900
1901
The first argument is a (relative) quarantine file name, as reported in
1902
the log. It must include a 12-character mail_id which is automatically
1903
recognized. The second argument is a secret_id, which can be fetched from
1904
a SQL database if @storage_sql_dsn is enabled (see README.sql),
1905
for example by the command:
1906
$ mysql amavis -e 'SELECT secret_id FROM msgs WHERE mail_id="V5htXBh0y0Hr"'
1907
or (preferably) by some other more advanced utility program.
1908
1909
Current simple-minded heuristics in the amavisd-release program is to assume
1910
a message is stored in a SQL database when the file name (first argument)
1911
consists only of a 12-character mail_id. Please adjust the program if this
1912
assumption is not true, e.g. when $virus_quarantine_method='local:%m'
1913
instead of a default $virus_quarantine_method='local:virus-%m';
1914
1915
If secret_id is not available, administrator may choose to skip checking
1916
of secret_id in the amavisd daemon by setting a configuration variable
1917
$auth_required_release to false (it defaults to true), and supplying an
1918
empty secret_id or not supplying it at all in the AM.PDP release request.
1919
The variable $auth_required_release is also part of policy banks.
1920
1921
If the release client program specifies a nonempty secret_id in the request,
1922
the secret_id will be validated and a request will fail if not valid,
1923
regardless of the setting of $auth_required_release.
1924
1925
Release requests contend for the same amavisd child processes as regular
1926
mail processing. This may cause delays in responses to release requests,
1927
especially when Postfix feeds mail to amavisd-new via LMTP which is more
1928
persistent in keeping connections open than the Postfix SMTP client service.
1929
To ensure one child process is always available for processing extra
1930
requests such as release requests, one may choose to set $max_servers
1931
larger (by one) than MTA's maxproc setting.
1932
1933
To enable recognition and processing of AM.PDP protocol requests in amavisd
1934
daemon, a dedicated Unix socket or a TCP port needs to be opened, for example
1935
by the following assignments in amavisd.conf:
1936
1937
$unix_socketname = "$MYHOME/amavisd.sock";
1938
$interface_policy{'SOCK'} = 'AM.PDP';
1939
$policy_bank{'AM.PDP'} = { protocol => 'AM.PDP' };
1940
1941
or similarly for connections through a dedicated TCP port 9998, and
1942
restricts it to accepting IP connections from local IP address only:
1943
1944
$inet_socket_port = [10024,9998];
1945
$interface_policy{'9998'} = 'AM.PDP';
1946
$policy_bank{'AM.PDP'} = {
1947
protocol => 'AM.PDP', # Amavis policy delegation protocol
1948
inet_acl => [qw( 127.0.0.1 )],
1949
# log_level => 3,
1950
};
1951
1952
- new configuration variable @spam_quarantine_cutoff_level_maps turns off
1953
quarantining if spam score is at or above this level; suggested by
1954
Gary Verchick, MrC and others;
1955
1956
- more informative logging and SMTP status generation in smtp client code;
1957
1958
- IPv6: allow optional brackets around IPv6 address in lookup tables and
1959
configuration variables, e.g. [FE80::]/10 is treated the same as FE80::/10
1960
Allow (and ignore) link-local scoped address in extended numeric IPv6 syntax
1961
(interface specification) when parsing link-level IPv6 addresses,
1962
e.g. fe80::1%lo0;
1963
1964
- IPv6: adjust the default @mynetworks to include link-local and site-local
1965
address ranges [FE80::]/10 and [FEC0::]/10, and add (optional) brackets
1966
around [::1]; (although the use of site-local address ranges seems to be
1967
deprecated nowadays);
1968
1969
- IPv6: tested sending mail via slightly modified Net::SMTP through
1970
an IPv6 socket to an IPv6-enabled Postfix; updated README.postfix;
1971
1972
To experiment, replace the: @ISA = qw(Net::Cmd IO::Socket::INET);
1973
with: use IO::Socket::INET6; @ISA = qw(Net::Cmd IO::Socket::INET6);
1974
in Net/SMTP.pm, to make use of the: $notify_method='smtp:[::1]:10025';
1975
Don't forget to add [::1] to mynetworks at MTA smtpd service on port 10025;
1976
1977
- bring earlier the initialization of %local_delivery_aliases and %builtins
1978
so that the config file can override the defaults;
1979
1980
- add SA autolearn status to the logged spam status (log line: "SPAM, ..."),
1981
as well as full TESTSCORES info to the list of SA tests (including score
1982
points for each SA test); contributed by John Sivak;
1983
1984
- new small wrapper module Amavis::IO::Zlib around Compress::Zlib allows for
1985
reading back compressed quarantine files for a mail release, and allows for
1986
writing compressed quarantine files without having to fork a gzip process.
1987
1988
This makes the utility program gzip(1) optional, which may be appreciated
1989
when running in a chroot environment;
1990
1991
- modified do_gunzip to use the new Amavis::IO::Zlib module;
1992
1993
- added LDAP lookups for the following attributes:
1994
amavisVirusAdmin, amavisNewVirusAdmin, amavisSpamAdmin,
1995
amavisBannedAdmin, and amavisBadHeaderAdmin; by Michael Hall;
1996
added attribute amavisBannedRuleNames by Brian Wong and Michael Hall;
1997
1998
- a policy bank name 'MYUSERS' now has a special semantics: this policy bank
1999
gets loaded whenever the sender matches @local_domains_maps. This only makes
2000
sense if local sender addresses can be trusted -- for example by requiring
2001
authentication before letting users send with their local address;
2002
the feature requested and a patch provided by Steffen Hansen;
2003
2004
- add cumulative percents to the TIMING report to make it easier to locate
2005
large contributors to elapsed time;
2006
2007
- updated interface code to Sophos Perl-SAVI module, taking advantage of its
2008
new ability to reload virus data: amavisd-new will initialize the SAVI
2009
object in the parent, which will be inherited by the children. The children
2010
will detect whenever the virus data is stale and automatically exit
2011
(reducing the number of messages processed with out-of-date protection),
2012
and the parent will reload the virus data before spawning new children;
2013
update provided by Paul B. Henson. To have this feature fully functional
2014
a small patch to Net::Server is needed:
2015
http://www.csupomona.edu/~henson/www/projects/SAVI-Perl/dist/
2016
Net-Server-0.87-parent_fork_hook-1.diff.gz
2017
(patch is no longer necessary since Net::Server 0.88 and amavisd-new 2.3.2)
2018
2019
- provide a routine dump_hash for debugging purposes: given a reference
2020
to an associative array (a Perl hash) it writes its contents to a log.
2021
Note: if called within amavisd.conf the log is still directed to STDERR.
2022
For example:
2023
@local_domains_maps = ( read_hash("$MYHOME/local_domains") );
2024
dump_hash($local_domains_maps[0]);
2025
2026
2027
OTHER CHANGES:
2028
2029
- new documentation file README.sql;
2030
2031
- tightened up a sample regular expression in amavisd.conf for catching
2032
Class ID (CLSID) extensions in file names;
2033
2034
- restrict the 'double extension' banning rule to require at least one letter
2035
in the next-to-last dot-delimited field; this allows filenames such as
2036
prog.33.22.01.exe not to be blocked by this rule;
2037
2038
- change hash lookups code to stop a hash search when a matching key
2039
exists, even if the matching result (corresponding hash value) is undef;
2040
this reverts the change made in a release 2.1.0 to a previous behaviour which
2041
is consistent with regexp lookups (but leaves SQL lookups to be different,
2042
continuing the scan to more general entries on a NULL field value;
2043
SQL lookup is specific because all attributes live in the same record);
2044
inconsistency reported by Gary Verchick;
2045
2046
- as a workaround for some versions of unrar (sparc64?) which are unable to
2047
create a subdirectory parts/rar by themselves, do_unrar now prepares the
2048
subdirectory explicitly; suggested by Andrzej Kukula;
2049
2050
- provide a new CentralCommand Vexira 'vascan' av entry (distinct from
2051
the 'Antivirus', which remains H+BEDV AntiVir -based) to work with
2052
the new Vexira scanner, thanks to Brian Wong and Norman C Rice;
2053
2054
- Vexira vascan: added status codes 3 (password protected) and
2055
9 (unsupported format), recognize "sequence found"; thanks to Brian Wong;
2056
2057
- F-Prot Antivirus: enhance the pattern to capture virus names;
2058
2059
- change the default $sql_select_policy and $sql_select_white_black_list to
2060
use the LEFT JOIN ... ON instead of WHERE for the 'join' relation; should
2061
be functionally equivalent, but makes the join operation explicit;
2062
2063
- changes to LDAP Schema to make it import and play nicely with Novell NDS,
2064
by Michael Tracey, SONOPRESS USA, LLC April 07 2005
2065
(changes are included in LDAP.schema but commented out by Mark)
2066
2067
- remove special handling in unmangle_sender() for ancient viruses
2068
Magistr and Badtrans, leave decisions on suppressing DSN entirely to
2069
@viruses_that_fake_sender_maps;
2070
2071
- rise a limit on the number of logged matching SA tests from 10 to 50
2072
(still some arbitrary sanity limit); based on observation of Bojan Zdrnja;
2073
2074
- add a minimal version requirement 2.22 for Digest::MD5, we need
2075
the new 'clone' method; reported by Thomas Jarosch;
2076
2077
- a command line option 'debug-sa' now sets variable $sa_debug to a string
2078
'1,all' instead of 1, in anticipation of the next version (3.1) of SA
2079
which changed interpretation of its debug option; the '1,all' seems to be
2080
compatible both with the old and the new interpretation, despite producing
2081
a warning in pre-3.1 versions of SA;
2082
2083
- documentation note: Macintosh.tar.gz installation instructions
2084
for Macintosh are not recent, they apply to OSX 10.2.0-10.3.9
2085
2086
2087
---------------------------------------------------------------------------
2088
December 22, 2004
2089
amavisd-new-2.2.1 release notes
2090
2091
2092
SECURITY:
2093
2094
- add support for the pax(1) archive decoder, which can handle tar/cpio/pax
2095
archives (including legacy format variants). Due to limitations in cpio
2096
(and in Archive::Tar), for security reasons it is preferred to decode
2097
such archives with pax and no longer with cpio; please add a line:
2098
$pax = 'pax';
2099
to amavisd.conf and verify that the program pax is installed on the system
2100
(and in the jail if running in chroot); problem reported by Ron Ogle;
2101
2102
- perform additional tests at startup time on proper protection of the
2103
configuration file;
2104
2105
- add file name extensions wmf, emf and grp to the example list of
2106
banned extension, according to recent Microsoft security bulletins;
2107
suggested by Stephane Lentz;
2108
2109
- introduces 'clean but inconclusive' av scanner result to avoid a specialized
2110
or quick partial av scanner like jpeg checker to claim mail is clean
2111
when all other general purpose av scanners fail (see below);
2112
2113
2114
INCOMPATIBILITY:
2115
2116
- removed some legacy $*_ldap variables, as they are no longer needed;
2117
2118
These variables were still declared but ignored in 2.2.0 for compatibility
2119
with older amavisd.conf files. Such variables need to be removed from
2120
the amavisd.conf if they are still present there from older versions,
2121
otherwise Perl will complain with 'Global symbol ... requires explicit
2122
package name";
2123
2124
2125
OTHER FIXES:
2126
2127
- files_to_scan and decompose_mail are now able to remove unexpected
2128
directories which may have been left behind by some failed decoding
2129
and were causing temporary failures and mail delivery retries;
2130
error recovery problem after failed unarj reported by Ralf Hildebrandt,
2131
and a related problem with tar, reported by Ron Ogle;
2132
2133
- error recovery code in files_to_scan and rmdir_recursively now tries to
2134
change protection on directories and files, and retry if the first attempt
2135
to access them fails because of denied permission;
2136
2137
- pre-load some additional Perl modules needed by SA when running in chroot;
2138
2139
- add module Net::LDAP::Search to a list of pre-fetched modules;
2140
omission pointed out by Paul Jacobson;
2141
2142
- when quarantining is disabled by keeping $QUARANTINEDIR undefined,
2143
the log entry and administrator notification message inappropriately
2144
suggested that mail was quarantined, which in fact (appropriately)
2145
it was not. Setting $QUARANTINEDIR='' did work as expected.
2146
Reported by Sascha Lucas;
2147
2148
- avoid the use of Encode::is_utf8 due to a Perl bug (still present in
2149
5.8.8, Encode::is_utf8 on tainted utf8 character string produces false);
2150
Perl bug tracking: #32687: Encode::is_utf8 on tainted UTF8 string
2151
returns false;
2152
2153
- modify safe_encode() to guarantee the result is a string of octets,
2154
not a string of UTF-8 characters; it saves some unnecessary work in
2155
further processing and keeps MIME::Entity from UTF swamp when running
2156
in chroot; problem pointed out by Branko F. Gracnar;
2157
2158
- avoid braindead Perl default where an empty regexp implies the last
2159
successfully matched regexp, which (if not being very careful) brings in
2160
some completely unrelated last-executed regular expression;
2161
2162
- change kill 'TERM' into kill 'KILL' when a forked process within run_command
2163
and run_command_consumer gets into deep trouble, to avoid exit handlers
2164
being invoked in the subprocess (which could lead to two processes trying
2165
to clean the same set of temporary files);
2166
2167
- in an old sendmail setup using the amavis(.c) helper program without
2168
LDA arguments, avoid inappropriate warning:
2169
"WARN: no recips left (forgot to set $forward_method=undef using milter?)
2170
and return status 0 instead of 99 when message is to be blocked, as the
2171
helper program amavis(.c) does not recognize status 99 in this situation
2172
and inappropriately passed it on to sendmail; reported by The Mindflayer;
2173
2174
- the @bypass_header_checks_maps is now able to also bypass the bad header
2175
checks as provided by MIME::Parser; inconsitency reported by CRivera;
2176
2177
- avoid some Perl warning messages; thanks to Bill Landry;
2178
2179
2180
CHANGES AND MINOR NEW FEATURES:
2181
2182
- add configuration variable @newvirus_admin_maps (and $newvirus_admin,
2183
along with corresponding SQL field 'newvirus_admin') which works like
2184
the existing @virus_admin_maps (and $virus_admin), except that it sends
2185
virus administrator notification to specified e-mail address only for newly
2186
encountered viruses which have not yet been encountered since the amavisd
2187
startup. It makes use of by-virusname counters in the SNMP counters
2188
database. If more than one child process starts working on infected
2189
message containing a not-yet-accounted-for virus, there might be more
2190
than one 'first time' notification, this is not a malfunction. Both
2191
the @newvirus_admin_maps and the @virus_admin_maps may be enabled,
2192
each (possibly both) would receive their notifications as appropriate.
2193
2194
A useful setting is to globally enable only the new virus notifications,
2195
and additionally enable _all_ administrator notifications for internally
2196
originating mail only (by the use of policy banks);
2197
2198
- provide separate configuration variables @banned_admin_maps and
2199
@bad_header_admin_maps, along with corresponding SQL fields
2200
'banned_admin' and 'bad_header_admin'; their function was previously
2201
covered by @virus_admin_maps, which now only still controls administrator
2202
notifications in case of viruses;
2203
2204
- introduces 'clean but inconclusive' av scanner result to avoid a specialized
2205
or quick partial av scanner like jpeg checker to claim mail is clean
2206
when all other general purpose av scanners fail:
2207
2208
in av scanner entries (lists @av_scanners and @av_scanners_backup) give
2209
an extended meaning to undefined fourth argument (the 'match for clean'
2210
list or regexp). The interpretation of the fourth argument is now:
2211
2212
4. an array ref of av scanner exit status values, or a regexp (to be
2213
matched against scanner output), indicating NO VIRUSES found;
2214
a special case is a value undef, which does not claim file to be clean
2215
(i.e. it never matches, similar to []), but suppresses a failure warning;
2216
to be used when the result is inconclusive (useful for specialized and
2217
quick partial scanners such as jpeg checker);
2218
2219
Also modified example jpeg checker entry in amavisd.conf accordingly.
2220
2221
- NOD32 av scanner: changed @av_scanners entry to match the new version
2222
of the scanner; thanks to Nejc Skoberne;
2223
2224
- added @av_scanners entry for File::Scan;
2225
2226
- when preparing a SQL SELECT clause for white/blacklisting lookup,
2227
take into account a relative position of ? and %k in the
2228
$sql_select_white_black_list template to improve flexibility
2229
of specifying the clause; suggested by Matt Petteys;
2230
2231
- reduce the log level of some more common and harmless log messages;
2232
2233
- macro %p and the log entry now reports full policy bank path,
2234
not just the last loaded policy bank name;
2235
2236
- added LDAP attributes amavisWarnVirusRecip, amavisWarnBannedRecip,
2237
and amavisWarnBadHeaderRecip; by Joel Nimety and Michael Hall;
2238
2239
- renamed LDAP attribute name amavisSpamModifiesSubject to
2240
amavisSpamModifiesSubj in order to match the documented LDAP schema;
2241
noticed by Kees Bos, patch by Michael Hall;
2242
2243
- add support for ripOLE decoder, which attempts to extract embedded documents
2244
from MS OLE documents (MS Office) (http://www.pldaniels.com/ripole/,
2245
by Paul L Daniels)); ripOLE is still experimental/alpha code;
2246
To make amavisd-new find the installed program 'ripole', add the:
2247
$ripole = 'ripole';
2248
to the amavisd.conf. Suggested by David Wilson and Noel Jones;
2249
2250
- allow multiple occurrences of command line option: -c config_file
2251
and execute the provided configuration files one after the other;
2252
based on a subset of functionality provided as a patch by Davor Ocelic;
2253
2254
- a slight improvement in classifying mpeg and some other multimedia files
2255
(in the default $map_full_type_to_short_type_re);
2256
2257
- several minor code cleanups;
2258
2259
- add a recommendation by Daniel J McDonald to a documentation file INSTALL:
2260
If different UID is preferred for an AV scanner, a solution for
2261
ClamAV is to add user clamav to the amavis group, and then add
2262
AllowSupplementaryGroups to clamd.conf;
2263
2264
- enclosed a simple demonstrational Perl program amavis.pl, which is
2265
functionally much like the amavis.c helper program, but talks the new
2266
AM.PDP protocol with the amavisd daemon. See README.protocol for the
2267
description of AM.PDP protocol. To be placed in amavisd.conf:
2268
$protocol='AM.PDP'; $unix_socketname='/var/amavis/amavisd.sock';
2269
Usage: amavis.pl sender recip1 recip2 ... < message.txt
2270
2271
- documentation updates;
2272
2273
2274
---------------------------------------------------------------------------
2275
November 2, 2004
2276
amavisd-new-2.2.0 release notes
2277
2278
2279
SECURITY:
2280
2281
- modified MIME entity traversal to include MIME container parts
2282
(e.g. multipart/*) as pseudo parts. Such pseudo-parts do not carry
2283
any body contents but do show up (with their MIME content type only)
2284
in the tree structure as seen by banned names checks. This makes it
2285
possible to specify more complex banned rules based on the placement
2286
of leaf nodes within or outside of mime multipart containers.
2287
2288
This also re-enables the possibility to check such MIME wrappers for
2289
banned MIME Content-Types (most notably for the message/partial and
2290
message/external-body), which was lost with the change of internal
2291
representation of mail parts in version 2.0 (amavisd-new-20040701);
2292
2293
- preserve original zip archive for virus scanners to see, if the archive
2294
contains any zero-length members (Archive::Zip module chooses not to
2295
extract members with declared zero size, even if the size does not
2296
correspond to the actual size); based on a patch by Dirk Datzert;
2297
2298
- add tests to mime_traverse to verify that the files produced by MIME::Parser
2299
really do exist and are readable; (and sent a patch adding I/O checks
2300
for MIME::Tools to its maintainer; please use MIME::Tools 5.414 or later
2301
from CPAN to avoid possibility of full /tmp partition causing infected
2302
or spam mail to pass through);
2303
2304
- changed recommendation in INSTALL to choose the latest version of
2305
MIME-Tools from CPAN - the 5.415 at the time of this writing;
2306
2307
- do_unrar: recognize encrypted entire archives (not just their individual
2308
members), and flag mail as undecipherable;
2309
2310
- recognize file(1) report/^MS Windows\b.*\bDLL\b/ as short type 'dll';
2311
add 'dll' to example patterns in amavisd.conf and amavisd.conf-sample;
2312
add name extension '.cpl' to the list of basic banned names;
2313
2314
2315
INCOMPATIBILITY:
2316
2317
- incompatible change: the default value for $recipient_delimiter is
2318
now undef and no longer '+'; adding address extensions must now be
2319
explicitly enabled;
2320
2321
- minor change in the default X-Virus-Scanned: header field, see below;
2322
2323
2324
OTHER CHANGES SINCE 2.1.2:
2325
2326
- rewritten LDAP modules, by Michael Hall;
2327
2328
- improvements in handling of double errors (errors that occur while
2329
handling consequences of some previous error); be permissive on failures
2330
in DESTROY methods; prevent some cases for child process not being able
2331
to sign off from the nanny database;
2332
2333
- enforce $max_requests more strictly, dropping SMTP session after task
2334
count is exceeded by one; this is in response to the new smtp session
2335
caching behaviour in Postfix, which is now much more persistent in keeping
2336
the session open on a busy mailer; although dropping session at the server
2337
side is discouraged by RFC 2821, this change was recommended on the
2338
postfix-users mailing list;
2339
2340
- added a site-wide mail size limit $smtpd_message_size_limit, and a
2341
per-recipient mail size limit lookups @message_size_limit_maps along with
2342
SQL and LDAP fields, making it possible to reject mail based on its size.
2343
The list of lookup tables maps recipients to mail size in bytes, e.g.:
2344
2345
@message_size_limit_maps = ({'user1@example.com' => 20_000_000,
2346
'user2@example.com' => 15*1024*1024,
2347
'user3@example.com' => 0, # uses global limit
2348
'.example.com' => 10*1024*1024 });
2349
$smtpd_message_size_limit = 20*1024*1024; # global limit if nonzero
2350
2351
A value of 0 or undef disables the check and is a default. A per-recipient
2352
limit is bound on the high side by the $smtpd_message_size_limit, and on the
2353
low side by 64kB, which is a minimal allowed size limit imposed by RFC 2821.
2354
2355
This limiting really belongs to MTA and is only partially supported here
2356
(no admin notifications, no quarantine, no final_*_destiny configurability).
2357
It is mostly provided here to be able to place some sanity limit on runaway
2358
or malicious clients, or if someone insists on using amavisd-new in a
2359
pre-queue filtering setup; suggested by Tuomo Soini;
2360
2361
- internal: add new object $msginfo->mail_text_fn to hold the file name
2362
of the original mail, decouple the location of file email.txt from the
2363
temporary directory, which was implied until now. This is presently needed
2364
for the Courier interface. Add optional attribute 'mail_file' to the
2365
AM.PDP protocol, see README.protocol;
2366
2367
- in banned parts descriptor strings which are used in banned name checking,
2368
provide a 'T=empty' short type for empty mail parts, including for empty
2369
MIME parts (instead of omitting T=... altogether). This can be used in
2370
banned rules to test for empty parts, generally or restricted to empty
2371
MIME parts; suggested by Ricardo Stella and Stephane Lentz;
2372
2373
- a banned lookup result (which is interpreted as boolean for most purposes:
2374
zero or empty for false, anything else for true) may give a result 'DISCARD'
2375
(which is true as well), which will disable DSN if it turns out the mail
2376
was blocked by such banned rule (akin to viruses_that_fake_sender and
2377
spam_dsn_cutoff_level). Here is an example rule in $banned_namepath_re:
2378
[ qr'(?# BLOCK EMPTY MIME PART APPLICATION/OCTET-STREAM )
2379
^ (.*\t)? M=application/octet-stream \t(.*\t)* T=empty (\t.*)? $'xmi
2380
=> 'DISCARD' ],
2381
Suggested by Ricardo Stella and Stephane Lentz;
2382
2383
- fix 'Insecure dependency in unlink' in sub files_to_scan which could
2384
happen when some decoder left non-regular files in the directory;
2385
reported by Andrzej Kukula;
2386
2387
- bug fix: only insert LDAP and SQL lookups objects into lists of maps at the
2388
first task of a child process, later tasks should not insert duplicates;
2389
2390
- new subroutine do_ar and new configuration variable $ar to handle
2391
standard Unix archives, including Debian binary packages;
2392
2393
- recognize a Unix archive and give it a short type .a;
2394
- recognize a Unix relocatable binary and give it a short type .o;
2395
2396
- do not penalize SMTP status "554 5.1.1 Error (DATA): no valid recipients",
2397
the situation arises regularly when pipelining is in effect and all
2398
recipients happen to be rejected;
2399
2400
- protect spaces and newlines when logging broken Message-ID and
2401
Resent-Message-ID header fields (macros %m and %r) to facilitate log
2402
parsing; protect newlines in logged Subject header field (macro %j);
2403
parsing difficulty pointed out by Chris Lee;
2404
2405
The present solution uses =XX (hex) encoding and is a quick and dirty
2406
fix. A cleaner solution to avoid double sanitation of special characters
2407
is needed, but would involve a deeper reorganization;
2408
2409
- updated example list of "banned extensions - long" in amavisd.conf
2410
and amavisd.conf-sample;
2411
2412
- change the default lock file name from "$MYHOME/amavisd-$$.lock" to
2413
"$MYHOME/amavisd.lock", to avoid inventing a new name at every restart
2414
and leaving old files behind; pointed out by Dale Walsh and Martin Orr;
2415
2416
- updated av entry for nvcc (Norman Virus Control v5 / Linux) to include
2417
statuses 10,11, and 2,14 to the status lists according to documentation;
2418
password protected or corrupted archive (status 11) was not recognized
2419
as non-infected status; thanks to Michael Ramke of Norman Data Defense
2420
Systems GmbH;
2421
2422
- updated DrWebD entry to recognize and ignore flag DERR_SKIPPED;
2423
2424
- support Mail::ClamAV 0.12 and 0.13 or later, which is incompatible with
2425
0.11 due to change of constant names in the underlying ClamAV library;
2426
2427
- added 'check-jpeg' example entry to the @av_scanners list and provide the
2428
associated module JpegTester.pm; it offers a fully-fledged check for jpeg
2429
comment field buffer overflow attempts; should serve mainly as an example
2430
for adding similar quick responses to new threats;
2431
2432
- added 'check-jpeg-simple' example entry to the @av_scanners list (only in
2433
amavisd.conf-sample); it offers a quick check (and not very exact one:
2434
checks only the first 32kB) for jpeg comment field buffer overflow attempts;
2435
should serve mainly as an example for adding similar quick responses
2436
to new threats;
2437
2438
- relax too restrictive sanity check on temporary directory name
2439
when accepting requests from a helper program or via AM.PDP protocol
2440
(e.g. with sendmail milter setup); reported by Babu Kanagala;
2441
2442
- relax allowed set of characters when receiving XFORWARD attribute values,
2443
it turns out that characters like '=' and '+' are allowed;
2444
2445
- when using "bsmtp:" delivery method suppress X-Envelope-From and
2446
X-Envelope-To header fields, as the addresses are already available
2447
in the envelope;
2448
2449
- when using the "bsmtp:" quarantine method the *_quarantine_to was
2450
completely ignored, which made it impossible to turn off quarantining
2451
selectively for certain users by specifying an empty or undef value.
2452
Since 2.2.0 an empty *_quarantine_to turns off quarantine for a recipient
2453
regardless of the quarantine method. A nonempty string in *_quarantine_to
2454
(the exact value is still ignored) must now be used even with "bsmtp:"
2455
to enable quarantining. Inconsistency discovered by Sean Doherty;
2456
2457
- suppress leading $QUARANTINEDIR string from the value of macro %q,
2458
thus hiding the absolute file path from notifications;
2459
2460
- add configuration variable $local_client_bind_address (and equivalent
2461
policy bank key), to allow for explicitly binding local socket address
2462
to a specific interface in SMTP client; suggested by Wouter de Jong;
2463
2464
- keep whatever (if any) file results from gunzip and family (do_uncompress)
2465
even if the decompressor's exit status is nonzero; reason: gzip returns
2466
status 2 when decompressing file with trailing garbage; reported by
2467
Tobias Reckhard;
2468
2469
- collect declared original file name from gzipped (and friends) files
2470
if reported by file(1), making them available to banned name checking;
2471
2472
- avoid unnecessarily checking white/black lists if spam checks will not
2473
be used (e.g. infected mail);
2474
2475
- use qquote_rfc2821_local to properly quote e-mail addresses in the
2476
most visible log entries;
2477
2478
- if there is more than a single (or less then one) occurrence of %k in the
2479
SQL SELECT clause template, multiplicate the set of query keys accordingly,
2480
making possible more complicated custom SELECT clauses;
2481
2482
- don't forget to load amavis policy delegation protocol support code
2483
if AM.PDP is explicitly requested in $protocol, even if not listening
2484
on a Unix socket;
2485
2486
- add 'queue_id' attribute to the AM.PDP protocol; equivalent to a change
2487
in the Courier support code by Martin Orr;
2488
2489
- include the declared (faked) sender address in the virus recipient
2490
notification template, in addition to the originator IP address;
2491
2492
- add macro %Q and method Amavis::In::Message::queue_id, holding a
2493
MTA queue ID if available (in Courier and milter/AM.PDP setup);
2494
by Michael Musikhin (through Martin Orr);
2495
2496
- add macro %y to show elapsed processing time; suggested by Ed Walker;
2497
2498
- sanitize newlines and spaces (and some other characters) when moving
2499
syntactically invalid Message-ID and Resent-Message-ID to macros %m and %r
2500
for logging purposes; resulting wrapped main log entry reported by CRivera;
2501
2502
- bring up syslog priority to LOG_NOTICE when debug or debug_oneshot
2503
is in effect;
2504
2505
- make a product name, version ID and version date available as separate
2506
variables to avoid the need to parse $myversion for the purpose of
2507
customizing e.g. the setting of $X_HEADER_LINE; based on suggestion
2508
by Dale Walsh; the re-introduction of a date ($myversion_date) also
2509
suggested by Stephane Lentz.
2510
2511
Added variables: $myproduct_name, $myversion_id, $myversion_id_numeric,
2512
and $myversion_date, which serve to assemble the $myversion. Modified the
2513
default templates of $smtpd_greeting_banner and $smtpd_quit_banner to
2514
take advantage of the new variables. Changed $X_HEADER_LINE default
2515
from "by amavisd-new at $mydomain" to "$myproduct_name at $mydomain" and
2516
added an example of a $X_HEADER_LINE setting with version number included
2517
to the amavisd.conf-sample;
2518
2519
- added SQL fields 'virus_admin' and 'spam_admin' to lookup lists
2520
@virus_admin_maps and @spam_admin_maps; patch by Robert Collier;
2521
2522
- add a log message 'SPAM-KILL, ...' (at log level 3) for not-passed mail,
2523
to complement the 'SPAM-TAG' log message for passed mail;
2524
2525
- add Mail::SpamAssassin::Plugin::SPF to a list of modules that SA fails
2526
to load at init time;
2527
2528
- prevent sending the same SMTP response more than once, if the first
2529
attempt failed due to disconnected SMTP session;
2530
2531
- fix a double-@ formatting buglet in the log message
2532
"adding address extension _spam to user@@domain", reported by Vincenzo;
2533
2534
- add kill('TERM',$$) to the arsenal of attempts to get rid of unwanted
2535
forked process;
2536
2537
- includes rpm spec file, including the init script, contributed by
2538
Marius Andreiana, based on previous work by Dag Wieers;
2539
2540
- document the localization template directory contents
2541
(in file amavisd.conf-sample) when read_l10n_templates is used;
2542
thanks to Jo�l Brogniart;
2543
2544
- includes file Macintosh.tar.gz, which contains auto-startup scripts
2545
and installation instructions for Mac OS X, contributed by Dale Walsh
2546
of the Dale Enterprise L.L.C.
2547
2548
2549
---------------------------------------------------------------------------
2550
September 6, 2004
2551
amavisd-new-2.1.2 release notes
2552
2553
- fixed (hard)black- and white-listing on static lookup tables
2554
which failed to match any sender; reported by Derck Floor;
2555
2556
- use $hdrfrom_notify_recip address in the From: field for recipient
2557
notifications, instead of $hdrfrom_notify_admin; inconsistency
2558
pointed out by Ekkehard Burkon;
2559
2560
- the 'neutral' sender notification template was joining the Subject
2561
and the Message-ID header fields into one longer Subject when it was
2562
reporting a nondelivery other than the 'invalid characters in header'.
2563
Likewise the first body line of this same DSN was eaten up:
2564
"This nondelivery report was generated by the amavisd-new program"
2565
(the problem was introduced in amavisd-new-20030616 and never reported);
2566
2567
- in amavisd-agent, amavisd-nanny, amavisd: extend the signal and error
2568
handling in code sections holding bdb locks from just ignoring the SIGINT,
2569
to controlled catching and re-signaling several signals and error conditions;
2570
problem reported by Tom Mulder;
2571
2572
- suppress duplicate names from the list of virus names in macro %V;
2573
by Gregor Weiss;
2574
2575
- fine-adjusted log level of some log messages;
2576
2577
- discard leading and trailing whitespace from the macro %t (Received trace);
2578
2579
- extend the search for IP in the Received trace from 4 to a maximum of
2580
6 entries;
2581
2582
- ignore private IPv6 addresses (RFC 3513: link-local, site-local, multicast)
2583
when searching through Received trace for the origin of mail;
2584
2585
- place mail header field X-Envelope-From in front of the X-Envelope-To in
2586
quarantined mail; also changed case of X-Quarantine-id into X-Quarantine-Id
2587
for consistency with other header fields;
2588
2589
- provide new macro %e which evaluates to our best guess of the originator
2590
IP address collected from the Received trace, complementing similar macros
2591
%t, %a and %g; suggested by Gregor Weiss;
2592
2593
- add the result of macro %e to the default 0-level log entry;
2594
2595
- provide new macros %u and %U to evaluate to a timestamp of the message
2596
reception similar to an existing macro %d (RFC 2822 local date-time);
2597
the (%u) as Unix time (seconds since 1970-01-01T00:00Z as a decimal integer,
2598
suggested by Gregor Weiss), and (%U) as ISO 8601 (EN 28601) UTC date-time;
2599
2600
- avoid some empty lines in default DSN templates and fix some inconsistencies
2601
in their formatting;
2602
2603
- internal: collect existing common code for time formatting as new
2604
subroutines iso8601_timestamp and iso8601_utc_timestamp; collect existing
2605
common code to find IP address in the Received trace as a new subroutine
2606
best_try_originator_ip;
2607
2608
- bump up the version number in $myversion - the 2.1.1 still presented
2609
itself as 2.1.0;
2610
2611
- add a note about a data structure difference between @score_sender_maps and
2612
$per_recip_blacklist_sender_lookup_tables (amavisd.conf-sample, amavisd)
2613
2614
2615
---------------------------------------------------------------------------
2616
August 24, 2004
2617
amavisd-new-2.1.1 (amavisd-new-20040824) release notes
2618
2619
- unconditionally initialize @banned_filename_maps to (\$banned_filename_re),
2620
otherwise $banned_filename_re is ignored by default (unless amavisd.conf
2621
explicitly assigns to @banned_filename_maps); a patch by Thomas Jarosch;
2622
2623
- fixed inappropriate log entry in SQL whitelisting:
2624
wbl: (SQL) recip <xxxx@yyyy.com> whitelisted sender <...>,
2625
unexpected wb field value: "1";
2626
reported by Carlos Horowicz;
2627
2628
- added missing import of &ca to the amavisd-new-courier.patch; by Martin Orr;
2629
2630
- produce a warning when there is an unknown field in the policy bank
2631
to be loaded;
2632
2633
- with delivery method 'bsmtp:' prepend a directory $QUARANTINEDIR to the
2634
file path if not explicitly specified, to behave like the 'local:' delivery
2635
method, making it possible to hide full path from the X-Quarantine-Id
2636
and notifications; a patch by Thomas Jarosch;
2637
2638
- pre-load SA 3.0.0 module Mail::SpamAssassin::Plugin::Hashcash
2639
to make it available in the chroot jail;
2640
2641
- pre-load modules Mail::SpamAssassin::SpamCopURI and URI::*
2642
for SA older than 3.0.0;
2643
2644
- enhancement to amavisd-nanny: when terminating a process and SIGTERM
2645
produces no result for some time, try SIGKILL; contributed by Philip Engdahl;
2646
2647
2648
---------------------------------------------------------------------------
2649
August 15, 2004
2650
amavisd-new-2.1.0 (amavisd-new-20040815) release notes
2651
2652
The 2.1.0 release is mostly a maintenance release over 2.0, with
2653
only a handful of smaller features added. Based on a manual code audit
2654
the number of smaller internal code changes is rather extensive,
2655
some changes dealing with long-standing known deficiencies, minor bugs,
2656
documentation problems and typos. Only a few fixes are for new bugs
2657
introduced in 2.0.
2658
2659
The files amavisd.conf, amavisd.conf-default and amavisd.conf-sample
2660
have been extensively reworked, with the hope to suit better the
2661
new installations, while possibly causing some head-scratching for
2662
existing installations when looking at a diff. The file amavisd.conf
2663
is the one that should serve as a sound base for the initial config file,
2664
while keeping an eye on the list of all variables and their defaults
2665
in amavisd.conf-default. The amavisd.conf-sample is being phased out
2666
of active maintenance, and should serve mostly as a set of examples and
2667
the source of documentation until better documentation is available.
2668
2669
Two nice features are available:
2670
2671
- the use of BerkeleyDB and libdb is now optional;
2672
controlled by variables $enable_db and $enable_global_cache;
2673
2674
- a new program 'amavisd-nanny', with the accompanying instrumentation
2675
in amavisd, displays the general health of all amavisd child processes,
2676
reports crashed ones and attempts to kill long overdue processes;
2677
It is still experimental and minimalistic, problem reporting is currently
2678
only to stdout.
2679
2680
2681
Other changes:
2682
2683
- incompatible change since 2.0: the use of BerkeleyDB is now off by default;
2684
2685
The use of BerkeleyDB and libdb is made optional, controlled by
2686
variables $enable_db and $enable_global_cache, both false by default.
2687
2688
$enable_db:
2689
enables the use of BerkeleyDB/libdb
2690
(for SNMP counters database and nanny, and optionally for cache);
2691
2692
$enable_global_cache:
2693
enables the use of libdb-based cache when $enable_db is also true;
2694
2695
If either the $enable_global_cache or $enable_db are false,
2696
cache of mail body MD5 digests is kept in child-local memory
2697
as in pre-2.0 versions, and is therefore local and short-lived,
2698
with lower expected cache hit rate;
2699
2700
- incompatible change: DSPAM 3.0 or better is needed (if $dspam is enabled),
2701
no longer works with 2.x;
2702
2703
- incompatible change: changed name of the (hardly ever needed) configuration
2704
variable auth_supported_out to auth_required_out, to better reflect its
2705
semantics (should be true if MTA server to which amavisd is sending
2706
notifications and forwarding mail requires authentication (AUTH smtp
2707
command));
2708
2709
- a new small program included: amavisd-nanny is a program to show the status
2710
and keep an eye on the health of amavisd child processes (experimental);
2711
2712
- fixed a bug in lookup_acl where a "." did not act as a catchall;
2713
thanks to JP;
2714
2715
- fixed a problem in SQL lookup which could return undef even when
2716
not all the matched records had NULL in the field;
2717
2718
- fixed compatibility with old 'amavis' helper program
2719
('delivery_care_of' defaulted to 'client', instead of depending on
2720
the presence of ldaargs), reported by Charlie Schluting and Christer;
2721
2722
- fixed long standing problem in do_ascii, which could return without calling
2723
Convert::UUlib::CleanUp, occasionally spilling state into subsequent
2724
mail checks within the same process;
2725
2726
- fixed macros %D, %O and %N when log_recip_templ is being expanded;
2727
a patch by Ed Walker;
2728
2729
- fixed recognition of separators in a nested call during macro expansion;
2730
2731
- pre-load missing modules Net::LDAP, Net::LDAP::Schema, Net::LDAP::Search,
2732
and Net::LDAP::RootDSE; suggested by Paul Jacobson;
2733
2734
- fix locale-related bug in rfc2822 date generation, where we were restoring
2735
the saved LC_TIME value to LC_CTYPE (!);
2736
a patch by Henrique de Moraes Holschuh / Debian support team;
2737
2738
- protect from signals while bdb cursor holds a lock;
2739
2740
- new subroutine inherit_header_edits() and slight code restructuring
2741
makes possible for spam_scan() and other code before the final delivery
2742
to start submitting common header edits into $msginfo->header_edits,
2743
avoiding the need for passing them through global variables;
2744
2745
- now a loglevel-based automatic syslog priority assignment can no longer
2746
lower a message syslog priority below the syslog priority specified
2747
in the $SYSLOG_LEVEL, it can only increase it; the violation of the
2748
least-surprise principle pointed out by Andy Dills;
2749
2750
- a small optimization in logging: a new subroutine ll allows to save time
2751
in preparing complex log entries when we know their log level exceeds
2752
the current log level and won't be logged;
2753
2754
- in default macro templates $log_templ and $log_recip_templ:
2755
* placed 'spam' condition before 'bad header' for consistency
2756
with program behavior;
2757
* added reporting of tag/tag2/kill levels in $log_recip_templ
2758
(experimental: macro names may change in future versions);
2759
* rewritten templates using negation (i.e. [?x|1] ) to avoid unsightly
2760
selector nesting;
2761
2762
- MIME decoding is now allowed to exceed the decoding quota, avoiding
2763
the problem when a small quota settings might not allow even a plain
2764
mail through;
2765
2766
- override LC_TIME to "C" on every log message, to work around issues
2767
with Unix::Syslog, which would log stuff with the date stamp localized,
2768
which syslog would dislike and add its own, and the resulting mess
2769
is not recognized by amavis log processors;
2770
a patch by Henrique de Moraes Holschuh / Debian support;
2771
2772
- changed dspam command line options to work with dspam 3.0 (no longer
2773
with 2.x), a patch by Ron Ohmer, Nabil Sefrioui, and Reech;
2774
2775
- dspam header fields are now inserted into passed mail if all recipients
2776
are local;
2777
2778
- supply better defaults for $hdrfrom_notify_sender, $hdrfrom_notify_recip,
2779
$hdrfrom_notify_admin and $hdrfrom_notify_spamadmin, similar to defaults
2780
from amavisd-new-20030616-p10;
2781
2782
- when parsing output from the 'AVG grisoft' virus scanner,
2783
don't include CR in virus name; reported by Vernon A. Fort;
2784
2785
- new file 'amavisd-new-qmqpqq.patch' provided by Martin Solciansky,
2786
(similar to fixes by Christopher Odenbach) making it work again
2787
with the current version of amavisd-new;
2788
2789
- use lstat instead of stat, and test for soft links wherever appropriate;
2790
2791
- remember inode and device number when creating temporary directory
2792
and temporary file, and test for change before removing them;
2793
2794
- enhanced security: certain tainted values are allowed to enter deeper
2795
into program, untainting them only where and when really necessary;
2796
2797
- avoid a taint problem in Mail::ClamAV;
2798
2799
- added AV entry for CAI eTrust Antivirus; by Stephane Lentz
2800
(requires a suid shell wrapper around inocmd32);
2801
2802
- added status 9 to the set of infected statuses for the
2803
drweb command line scanner (DrWeb Antivirus);
2804
2805
- use our subroutine q_encode to Q-encode header fields from the
2806
notification templates, instead of MIME::Words::encode_mimeword
2807
(a similar fix in 2.0 applied only to encoding of modified headers
2808
in passing mail);
2809
2810
- add attribute 'x-spam-type=original' to the Content-Type header field
2811
(the SpamAssassin's code to recognize an original email) when defanging
2812
spam, facilitating reporting spam via SA to other spam fighting tools;
2813
a patch by Brian May;
2814
2815
- add a note to amavisd.conf that $sa_auto_whitelist has no effect on SA
2816
since 3.0.0 - SA has now a configuration file option 'use_auto_whitelist';
2817
2818
- turn off timer in post_process_request_hook() to avoid periodically
2819
recreating child processes on an idle machine;
2820
2821
- added @mynetworks_maps and enhanced lookup_ip_acl() to take a list
2822
of lookup tables: currently members can be an array ref (as before),
2823
or a hash ref (new) or a plain constant (new);
2824
2825
- generalized @debug_sender_acl into @debug_sender_maps along the lines of
2826
other lookup tables, and make it part of policy banks;
2827
2828
- added @warnvirusrecip_maps, @warnbannedrecip_maps, @warnbadhrecip_maps;
2829
2830
- added @spam_subject_tag_maps and @spam_subject_tag2_maps,
2831
to allow per-recipient spam tags string; suggested by Ed Walker;
2832
2833
Note, there is an inconsistency in names of legacy variables and the
2834
new @*_maps, in an attempt to rectify an unfortunate choice of name
2835
for seldomly used variable $sa_spam_subject_tag1:
2836
@spam_subject_tag_maps = (\$sa_spam_subject_tag1); # exotic
2837
@spam_subject_tag2_maps = (\$sa_spam_subject_tag); # in common use
2838
2839
Note also that corresponding SQL fields are 'spam_subject_tag' and
2840
'spam_subject_tag2'; usually only the 'spam_subject_tag2' would be used;
2841
2842
- added configuration variable $auth_reauthenticate_forwarded, which
2843
directs amavisd to apply its own credentials ($amavis_auth_user and
2844
$amavis_auth_pass) to unauthenticated forwarded (passed) mail, besides
2845
using them for submitting original messages (notifications and quarantine).
2846
This is similar to how mailing list managers are allowed to work (rfc2554).
2847
Note that the Perl module Net::SMTP in its current version is unable to
2848
specify the 'submitter' in its 'MAIL FROM' command, this should be rectified;
2849
2850
---------------------------------------------------------------------------
2851
July 1, 2004
2852
amavisd-new-20040701 / amavisd-new-2.0 release notes
2853
2854
2855
MAJOR NEW FEATURES (since amavisd-new-20030616-p10):
2856
2857
- security improvements: no shell required in chroot jail, checks performed
2858
to see if dropping privileges was successful, can drop privileges before
2859
reading config file;
2860
- the 'amavisd reload' command reimplemented, it now works even
2861
if running chroot-ed;
2862
- new feature: policy banks hold sets of configuration variables that may
2863
be switched with another predefined set based on incoming port number or
2864
original SMTP client IP address, avoiding the need to run more than one
2865
instance of amavisd daemon;
2866
- new feature: @score_sender_maps is a soft variant of black- and whitelisting;
2867
- extended semantics of SQL field wblist.wb for soft black/white-listing;
2868
- redesigned mail structure representation allows better control
2869
over 'banned' names and types;
2870
- MIME defanging can wrap the entire original mail in a MIME container;
2871
- more flexible control on lookups: configurable list of lookup tables
2872
observes the specified order of tables and permits arbitrary number
2873
of tables of any available type;
2874
- level-0 logging either by-message or by-recipient;
2875
- syslog priorities are now dynamically derived from the log level;
2876
- constantly updates a small database of SNMP-style counters, providing
2877
real-time measurements for status monitoring and statistics reporting,
2878
reducing the need for analyzing a log file; includes a sample/demo
2879
program 'amavisd-agent';
2880
- new 'policy delegation protocol' between helper program and the daemon
2881
can pass more information to the daemon and allows the daemon to
2882
pass more instructions back to MTA (useful for sendmail milter setup);
2883
- persistent cache of recent virus and spam checks, common to all child
2884
processes, improves the cache hit rate;
2885
- support for IPv6 address formats;
2886
- provisional/experimental support for DSPAM spam checker;
2887
- support for ClamAV virus scanner via Perl module Mail::ClamAV;
2888
- cleaned amavisd.conf :
2889
amavisd.conf configuration file with the essentials
2890
amavisd.conf-default lists all configuration variables with their defaults
2891
amavisd.conf-sample traditional-style commented amavisd.conf with examples
2892
2893
----
2894
2895
INCOMPATIBLE CHANGES since amavisd-new-20030616 (any patch level)
2896
2897
- requires Perl module BerkeleyDB with libdb version 3.1 or later
2898
(tested with db 4.1); This requirement will be made optional
2899
at a later date (in amavisd-new-2.1.0);
2900
2901
- a directory at $db_home (default /var/amavis/db) must be manually created
2902
to store cache and snmp DB files. It should be writable by user running
2903
amavisd. The db files within are removed and re-created at each restart
2904
to avoid having to deal with db recovery (but need not be, as far as the
2905
program logic is concerned);
2906
2907
- sending signal HUP in order to restart amavisd no longer works (previously
2908
it only worked in non-chrooted environment and relied on guessing amavisd
2909
absolute path); please use 'amavisd reload', or 'amavisd stop' and restart;
2910
2911
If the HUP method is really still needed, please replace the line
2912
commandline => [], # disable
2913
by:
2914
commandline => ['/usr/local/sbin/amavisd','-c',$config_file],
2915
in file amavisd, adjusting the path if necessary.
2916
2917
- due to changed names of temporary files, the old 'DrWebD' av entry needs
2918
to be replaced with the current one;
2919
2920
- changed $final_virus_destiny default from D_BOUNCE to D_DISCARD
2921
- changed $final_spam_destiny default from D_REJECT to D_BOUNCE
2922
2923
- changed defaults for variables $virus_quarantine_to, $banned_quarantine_to,
2924
$bad_header_quarantine_to and $spam_quarantine_to from undef (no quarantine)
2925
to values 'virus-quarantine', 'banned-quarantine', 'bad-header-quarantine'
2926
and 'spam-quarantine' respectively. Set them to undef or '' (empty string)
2927
to disable quarantine;
2928
2929
- add address extension at tag2 level, not at kill level as before;
2930
suggested by Jacob Elder and others;
2931
2932
- because of the reorganization of lookup tables, a new way
2933
of quickly disabling virus or spam checks in amavisd.conf is used.
2934
Instead of:
2935
# @bypass_virus_checks_acl= qw( . ); # uncomment to DISABLE anti-virus code
2936
# @bypass_spam_checks_acl = qw( . ); # uncomment to DISABLE anti-spam code
2937
the new recipe is:
2938
# @bypass_virus_checks_maps = (1); # uncomment to DISABLE anti-virus code
2939
# @bypass_spam_checks_maps = (1); # uncomment to DISABLE anti-spam code
2940
2941
- @virus_admin_maps and @spam_admin_maps now take as lookup keys recipient
2942
addresses, not sender address as before. The new semantics was often
2943
requested, the old semantics was not useful because modern viruses and
2944
spam fake sender address, so the choice was made to incompatibly change
2945
semantics and use the same config variable names, rather than introduce
2946
new names and leave behind useless variables;
2947
2948
An implication is that with multiple-recipient mail it is now possible
2949
to have more than one admin notification generated when recipients have
2950
different admin addresses associated. Still, each distinct admin address
2951
receives only one admin notification;
2952
2953
- removed old compatibility variable $mailto. Use $virus_admin and $spam_admin
2954
variables instead, as suggested in amavisd-new-20021116 release notes,
2955
or the more recent inventions @virus_admin_maps and @spam_admin_maps,
2956
or their equivalents in policy banks;
2957
2958
- removed support for old configuration variable $mailfrom. Use variables
2959
$mailfrom_notify_admin, $mailfrom_notify_sender, $mailfrom_notify_recip,
2960
and $mailfrom_notify_spamadmin as introduced in amavisd-new-20020630
2961
for the purpose;
2962
2963
- removed ancient variable @local_domains, use @local_domains_acl instead,
2964
or the more general @local_domains_maps
2965
2966
- removed old compatibility code which allowed for a couple of traditional
2967
variables to treat value "no" as false. The use of "no" for false has been
2968
deprecated since amavisd-new-20021116.
2969
2970
- revoke an old compatibility measure where a missing (undef) tag2 level
2971
would fall back to the kill level value;
2972
2973
- lookup_hash incompatible change (but hardly anyone will notice):
2974
a key presence in the Perl hash but with undefined hash value
2975
used to be interpreted as true, but now it is treated as undef,
2976
causing search to continue with remaining lookup tables (if any).
2977
This is more general and more in spirit with other lookup mechanisms;
2978
2979
- changed a default to initialize SAVI-Perl every time a child process is
2980
started, no longer at master process startup time only. This is to avoid
2981
the need to restart amavisd every time the Sophos IDE database is changed.
2982
One can revert to the previous behavior by uncommenting a call to
2983
Amavis::AV::sophos_savi_init in subroutine fetch_modules_extra;
2984
2985
2986
SECURITY
2987
2988
- no longer invoke shell to call gzip for compressing quarantined files
2989
or to call the sendmail command for submitting messages; the most important
2990
consequence is that a shell is no longer needed in a chroot environment
2991
and should preferably be removed;
2992
2993
- not to forget what was introduced in -p10: inserted a security check
2994
for a missing Net::Server patch, and abort if vulnerable;
2995
2996
- new command line options '-u user' and '-g group' are available.
2997
These are pretty much equivalent to doing a su(1) to the specified user
2998
first (in which case the use of these options is redundant).
2999
3000
By doing 'su' or by specifying a command-line option '-u username'
3001
one can prevent a potential security risk on misconfigured sites where
3002
amavisd.conf is writable by UID running amavisd (e.g. not owned by root).
3003
If a (non-root) username or UID is specified, privileges are now dropped
3004
_before_ opening and evaluating a configuration file. The consequence
3005
is that the configuration variables $daemon_user and $daemon_group
3006
(in amavisd.conf) can not have an after-effect (a warning is issued
3007
if different).
3008
3009
If -u is not specified, the behaviour is as before, i.e. the config file
3010
is read and evaluated under the current UID (as root unless 'su' was done),
3011
and the values of $daemon_user and $daemon_group from the config file
3012
are passed to Net::Server, which changes UID during its startup after
3013
chroot-ing (if requested).
3014
3015
If chroot is desired, the -u must not be used: the root privilege is
3016
required to do chroot, and the config file must be read _before_ doing
3017
chroot. A case of Catch-22. Be doubly careful of who can modify the
3018
configuration file.
3019
3020
Another consequence of specifying -u is that any external files (e.g.
3021
templates, lookup hashes) as possibly read from amavisd.conf, are now
3022
accessed as unprivileged user and no longer as root. The same goes for
3023
opening the log file when not logging via syslog.
3024
3025
3026
MAJOR NEW FEATURES
3027
3028
- policy banks hold sets of configuration variables controlling most of
3029
per-message settings, including: static lookup tables, IP interface
3030
access rules, forwarding address, log level, templates, administrator
3031
addresses, spam trigger levels, quarantine rules, lists of anti-virus
3032
scanner entries (or just a subset), banned names rules, defang settings,
3033
etc. The whole set of these settings may be replaced with another
3034
predefined set based on incoming port number, making it possible for
3035
one amavisd daemon to cope with more diverse needs of served user
3036
communities which could so far only be implemented by running more than
3037
one instance of the amavisd daemon, each with its own configuration file;
3038
3039
This mechanism brings new potentials for the future: in principle policy
3040
banks could be swapped not only based on port number or SMTP client
3041
IP address, but on any characteristics pertaining to a mail message as
3042
a whole (not specific to each of its recipients), or to characteristics
3043
of a connection from a mailer (e.g. the interface address or protocol);
3044
3045
Until a better mechanism is available, a policy bank named 'MYNETS' has
3046
special semantics: this policy bank gets loaded whenever MTA supplies a
3047
SMTP client IP address (Postfix XFORWARD extension or a new AM.PDP protocol)
3048
and that address matches the @mynetworks list.
3049
3050
A hash %$interface_policy is a current mechanism of assigning a
3051
policy bank to an incoming TCP port number (port must be in the list
3052
@$inet_socket_port). Whenever the connection from MTA is received,
3053
first a built-in policy bank with an empty name - the $policy_bank{''}
3054
gets loaded, which bringings in all the global/legacy settings.
3055
Then it is overlaid by whatever configuration settings are in the bank
3056
named in the $interface_policy{$port} if any, and finally the bank 'MYNETS'
3057
is overlaid if it exists and the SMTP client IP address is known
3058
(by XFORWARD SMTP extension command from MTA) and it matches @mynetworks.
3059
See amavisd.conf-sample for examples.
3060
3061
When a new policy bank is overlaid over an existing set of configuration
3062
variables, variables not present in the new policy bank retain their value.
3063
3064
The built-in policy bank (with empty name) is predefined, and includes
3065
references to most other variables (the dynamic config variables),
3066
which are accessed only indirectly through the currently installed
3067
policy bank. Overlaying a policy bank with another policy bank may
3068
bring in references to entirely different variables, possibly unnamed.
3069
3070
Configuration variables are referenced from a built-in policy bank (which
3071
is implemented as a perl hash, i.e. an associative array) by keys of the
3072
same name, e.g. { log_level => \$log_level, inet_acl => \@inet_acl, ...}.
3073
For scalars one level of indirection is allowed, e.g.
3074
a policy bank { log_level => \$log_level }; $log_level=2;
3075
is equivalent to { log_level => $log_level } or { log_level => 2 },
3076
but in the first example with an indirect reference, the $log_level
3077
may be assigned to even _after_ the policy bank has already been formed.
3078
3079
A word of caution: the syntax of entries within a policy bank hash
3080
is slightly different from assignments to configuration variables.
3081
This is because entries within policy bank are not asssignments, but
3082
key=>value pairs as in any Perl hash. And these pairs are delimited by
3083
commas, unlike statements, which are delimited by semicolons.
3084
Value is separated from its key by '=>' (or by a comma), whereas the
3085
assignment operator is '='. Keys of a policy bank are without leading
3086
$ or @ or %, unlike variable names. Values of a hash can only be scalars
3087
(e.g. strings or references).
3088
3089
Compare:
3090
- value of a policy bank is a reference to a Perl hash, e.g.:
3091
{ log_level => 3,
3092
forward_method => 'smtp:[127.0.0.1]:10025',
3093
spam_admin_maps => ["spamalert\@$mydomain"],
3094
}
3095
- normal assignments look like:
3096
$log_level = 3;
3097
$forward_method = 'smtp:[127.0.0.1]:10025';
3098
@spam_admin_maps = ("spamalert\@$mydomain");
3099
3100
And a final note: Perl can detect and report typing mistakes in variable
3101
names, but mistyped key is just some unused hash entry lurking in a hash,
3102
never used and never reported as mistyped/useless.
3103
3104
- @score_sender_maps is a soft variant of black- and white-listing,
3105
contributing positive or negative score points based on sender
3106
e-mail address. Btw, the 'score' in the name '@score_sender_maps' is meant
3107
as a verb, recipient is scoring a sender (= to grade, to determine the
3108
merit of);
3109
3110
Whitelisting is becoming deprecated because it is often and easily abused
3111
(but blacklisting can still be useful);
3112
3113
# ENVELOPE SENDER SOFT-WHITELISTING / SOFT-BLACKLISTING
3114
# Instead of hard black- or whitelisting, a softer approach is to add
3115
# score points (penalties) to the SA score for mail from certain senders.
3116
# Positive points lean towards blacklisting, negative towards whitelisting.
3117
# This is much like adding SA rules or using its white/blacklisting, except
3118
# that here only envelope sender addresses are considered (not addresses
3119
# in a mail header), and that score points can be assigned per-recipient
3120
# (or globally), and the assigned penalties are customarily much lower
3121
# than the default SA white/blacklisting score.
3122
#
3123
# The table structure is similar to $per_recip_blacklist_sender_lookup_tables
3124
# i.e. the first level key is recipient, pointing to by-sender lookup tables.
3125
# The essential difference is that scores from _all_ matching by-recipient
3126
# lookups (not just the first that matches) are summed to give the final
3127
# score boost. That means that both the site and domain administrators,
3128
# as well as the recipient can have a say on the final score.
3129
3130
- the MIME defanging (defang: to make harmless or less powerful)
3131
wraps the entire original mail in a MIME container of type
3132
'Content-type: multipart/mixed', where the first part is a text/plain
3133
with a short explanation, and the second part is a complete original
3134
mail, enclosed in a 'Content-type: message/rfc822' MIME part.
3135
Defanging is only done when enabled (selectively by malware type)
3136
and the malware is allowed to pass (*_lovers or *_destiny=D_PASS).
3137
The feature is global, i.e. not available on a per-recipient basis.
3138
3139
Conventional mail header fields are retained, and header fields
3140
Resent-{From,Date,Message-ID} are added. A header field X-Amavis-Modified
3141
is inserted to indicate that the mail body has been modified.
3142
3143
Note that defanging changes mail body and makes subsequent DomainKeys
3144
verifications to fail. The DomainKey-Signature header field is not
3145
retained in defanged mail (but is retained in the attached original mail).
3146
3147
It is an experimental feature (disabled by default, except for mail bombs),
3148
and not very efficient for large mail.
3149
3150
MIME defanging is unconditionally done for mail bombs, i.e. when the
3151
X-Amavis-Hold header field is added. The text in the first MIME part
3152
describes the reason, e.g.:
3153
3154
WARNING, possible mail bomb, NOT CHECKED FOR VIRUSES:
3155
Exceeded storage quota 29089500 bytes by do_unzip; ...
3156
3157
When MIME defanging is enabled for passed spam, the first MIME part
3158
contains the full SpamAssassin report.
3159
3160
The MIME defanging feature is not available in the sendmail milter setup.
3161
3162
- new parameter: @mynetworks
3163
It is an IP access list which determines if the original SMTP client
3164
IP address belongs to our internal networks. It is much like the Postfix
3165
parameter 'mynetworks' in semantics and similar in syntax, and its value
3166
should normally match its Postfix counterpart. It affects the value
3167
of a macro %l (=sender-is-local), and the loading of policy 'MYNETS'
3168
(if present). Note that '-o smtp_send_xforward_command=yes' (or its lmtp
3169
counterpart) must be enabled in the Postfix service that feeds amavisd,
3170
otherwise client IP address is not available to amavisd-new and new
3171
features based on @mynetworks do not work (the %l macro works as before
3172
and bases its decision on sender e-mail address matching local_domains);
3173
3174
3175
LOGGING
3176
3177
- syslog priorities are now dynamically derived from the log level of each
3178
log message (the first argument of sub do_log). The priority as specified
3179
in the $SYSLOG_LEVEL configuration variable is ignored in 2.0 (no longer
3180
in 2.1), but the 'facility' is not ignored. This makes it possible
3181
to influence the log verbosity by syslog.conf settings. Here is an
3182
example of a useful syslogd.conf setting (some levels may be left out;
3183
assumes the $log_level is high enough, e.g. 2, to produce any low-priority
3184
log messages at all):
3185
3186
mail.err /var/log/messages
3187
mail.notice /var/log/amavisd.log
3188
mail.info /var/log/amavisd-info.log
3189
mail.debug /var/log/amavisd-debug.log
3190
3191
(On linux do not forget to prefix filenames in syslog.conf with
3192
a '-' to disable sync, which has much impact on syslog performance!)
3193
3194
At the moment the mapping of message log levels to syslog priorities
3195
is hardwired:
3196
level <= -3: LOG_CRIT
3197
level <= -2: LOG_ERR
3198
level <= -1: LOG_WARNING
3199
level <= 0: LOG_NOTICE
3200
level <= 1: LOG_INFO (in version 2.0)
3201
level <= 2: LOG_INFO (since 2.1)
3202
else: LOG_DEBUG
3203
3204
Not to be confused with the $log_level configuration variable setting,
3205
which still works as before, suppressing generation of all log messages
3206
with log levels above $log_level.
3207
3208
For efficiency reasons one should not specify unnecessarily high $log_level
3209
and then discard low syslog priority messages in the syslogd.
3210
3211
- added $log_recip_templ variable, which is similar to $log_templ,
3212
but gets called for every recipient (the $log_templ is evaluated only once
3213
per message). Normally one or the other log template should be disabled
3214
by assigning undef or an empty string to the corresponding variable;
3215
3216
- the default $log_templ no longer shows quarantine ID if quarantining is
3217
disabled;
3218
3219
- added a macro %. (a dot), which might be useful in the $log_recip_templ;
3220
Its value is empty when $log_templ is expanded, and is a recipient counter
3221
(starting by 1) when $log_recip_templ template is expanded. Based on
3222
this macro one can provide a single template for both the $log_templ
3223
and the $log_recip_templ if needed, or perhaps let the log entry for
3224
the first recipient be more verbose that the rest;
3225
3226
- added a macro %T which expands to a list of triggered SA tests, but only
3227
when $log_templ and $log_recip_templ are expanded. In notifications
3228
the %T is still a list of To: addresses. An overlaid semantics, but we
3229
are running out of letters and a macro expander rewrite would be needed;
3230
3231
- new macros %k, %1, %2, %O, please see README.customize
3232
3233
3234
LOOKUPS
3235
3236
- new configuration variables make it more flexible to specify arbitrary
3237
list of lookup tables. Legacy configuration variables are still available
3238
and are referenced from the default values of @*_maps lists. If these lists
3239
are redefined, legacy variables are not used.
3240
3241
The new variables (lists) are:
3242
3243
@local_domains_maps
3244
@bypass_virus_checks_maps @bypass_spam_checks_maps
3245
@bypass_banned_checks_maps @bypass_header_checks_maps
3246
@virus_lovers_maps @spam_lovers_maps
3247
@banned_files_lovers_maps @bad_header_lovers_maps
3248
@virus_admin_maps @spam_admin_maps
3249
@virus_quarantine_to_maps
3250
@banned_quarantine_to_maps @bad_header_quarantine_to_maps
3251
@spam_quarantine_to_maps @spam_quarantine_bysender_to_maps
3252
@banned_filename_maps
3253
@spam_tag_level_maps @spam_tag2_level_maps @spam_kill_level_maps
3254
@spam_dsn_cutoff_level_maps
3255
@spam_modifies_subj_maps
3256
@whitelist_sender_maps @blacklist_sender_maps @score_sender_maps
3257
@addr_extension_virus_maps @addr_extension_spam_maps
3258
@addr_extension_banned_maps @addr_extension_bad_header_maps
3259
3260
@keep_decoded_original_maps
3261
@map_full_type_to_short_type_maps
3262
@viruses_that_fake_sender_maps
3263
3264
- for more informative logging of lookup operations, a new object type
3265
Amavis::Lookup::Label can be inserted to lists of lookup tables
3266
for the purpose of labeling the main purpose of the list;
3267
3268
- all lookup* subroutines can now return matching key when called in a
3269
list context;
3270
3271
- lookups can now return a list of all matching entries (not just the first
3272
match); used for the new soft- white/blacklists (@score_sender_maps);
3273
3274
- sub lookup() now allows for one level of list elements dereferencing,
3275
which makes possible the construction of the argument list and later
3276
still be able to modify its members (e.g. creation of regexp lookup table
3277
objects in the configuration file); It facilitates transition from old
3278
hard-wired lists of lookup tables to new @*_maps list variables which
3279
permits specifying an arbitrary number of lookup tables and to specify
3280
their search order;
3281
3282
- simplify and unify calls to lookup() by collecting arguments (references
3283
to lookup tables) in lists, e.g. @local_domains_maps, @virus_lovers_maps,
3284
@virus_admin_maps. These array variables default to lists of legacy
3285
variables, which are now never directly used by the program. Either
3286
the individual legacy variables may be assigned to, or the entire list
3287
replaced, in which case the legacy variables no longer have any effect.
3288
3289
- lookup_acl: respect $localpart_is_case_sensitive setting;
3290
3291
- lookup_hash and lookup_sql: rewritten lookup_hash and factored out the
3292
common code from lookup_hash and lookup_sql to make_query_keys();
3293
3294
- lookup_hash bug fix: avoid splitting address literal as if it were
3295
a domain name; (a bug with key '.' not being tried for address literals
3296
fixed thanks to Uwe S. Fuerst);
3297
3298
3299
SQL, LDAP LOOKUPS
3300
3301
- extended semantics of SQL field wblist.wb, which can hold a score value
3302
boost, which is interpreted as soft black/white-listing (the same semantics
3303
as the value in @score_sender_maps);
3304
3305
- recognize SQL server error 2013/"Lost connection to" and treat it
3306
the same as 2006/"MySQL server has gone away"; by Max Kalika;
3307
3308
- full domain stripping: @.sub.example.com @.example.com @.com @.
3309
3310
- lookup_hash and lookup_sql: limit the list of subdomain
3311
search keys to 10 levels as a sanity measure; e.g. for address
3312
user@14.13.12.11.10.9.8.7.6.5.4.3.2.com the subdomains keys search list
3313
starts at .9.8.7.6.5.4.3.2.com; (domain names are limited by standard
3314
to 127 levels);
3315
3316
- prepare SELECT statements on demand;
3317
3318
- recognize all-zero and all-null boolean fields as false;
3319
3320
- recognize new (optional) fields in the table 'policy':
3321
spam_dsn_cutoff_level
3322
virus_quarantine_to, banned_quarantine_to, bad_header_quarantine_to
3323
addr_extension_virus, addr_extension_spam,
3324
addr_extension_banned, addr_extension_bad_header
3325
3326
(the addition of virus_quarantine_to was suggested independently
3327
by Harald Kapper and by Dipl.Ing. Martin Boeck);
3328
3329
- consider the SQL user id a string (no longer required to be numeric);
3330
thanks to Max Kalika / Gentoo support;
3331
3332
- LDAP white/black list support by Jacques Supcik (similar to contribution
3333
from Scott Dier and Eric Dorland, which I forgot about, sorry);
3334
3335
- added amavisSpamQuarantineTo to the LDAP schema;
3336
new version of LDAP schema; by Jacques Supcik:
3337
3338
In the previous schema, the tag levels have been defined as integer.
3339
This is too restrictive, and have now been changed to strings
3340
(there is no float type in LDAP);
3341
3342
- added LDAP attributes for completeness: amavisBadHeaderLover,
3343
amavisBypassBannedChecks, amavisBypassHeaderChecks, amavisVirusQuarantineTo,
3344
amavisBannedQuarantineTo, amavisBadHeaderQuarantineTo; by J�r�me Schell;
3345
3346
3347
DECODING / DEARCHIVING
3348
3349
- provide optional ability to retain complete email message in its
3350
un-decoded form (alongside its decoded parts) for a virus scanner
3351
to see (enabled if $keep_decoded_original_re matched string 'MAIL');
3352
suggested by Tomasz Papszun; (partly backported to amavisd-new-20030616-p8);
3353
3354
- rewrite code that generates new file names (Amavis::Unpackers::Part->new),
3355
and rewrite code dealing with banned names. Keep information about each
3356
part organized as a tree, matching the descendence of each part, new
3357
package/object Amavis::Unpackers::Part to collect such information;
3358
3359
- make Amavis::Unpackers an optional module: the interface code to external
3360
decoder/unpackers/dearchivers does not get compiled and does not consume
3361
virtual memory if $bypass_decode_parts is true; (previously it just
3362
didn't get called, but was sitting in memory nevertheless);
3363
3364
- decode RPM archives by converting them to cpio, if rpm2cpio and cpio
3365
are available;
3366
3367
- do_tnef: extract $tnef->message if it exists, not just $tnef->attachments;
3368
3369
- support extracting MS cabinet files (CAB) by calling cabextract, if enabled
3370
and found. Beware: Lars Hecking warns that cabextract 0.6 is quite buggy
3371
and the author has been notified.
3372
3373
3374
VIRUS SCANNING
3375
3376
- support for ClamAV virus scanner via Perl module Mail::ClamAV,
3377
based on code by Roberto Pereira da Rosa;
3378
3379
- don't call virus scanners if there are no files in the directory to be
3380
scanned (e.g. mail with an empty body); some virus scanners don't like
3381
to be given an empty directory (e.g. Symantec newer savsecls);
3382
reported by Marco Bicca;
3383
3384
- rewritten/unified/generalized subroutines ask_daemon and sophos_savi
3385
based on the new subroutine ask_av;
3386
3387
- scan parts directory for file names exactly once regardless of the number
3388
of virus scanners and their arguments (containing '{}/*' or not);
3389
3390
- supply full original mail to virus scanners in case of MIME parse errors
3391
(in addition to any possibly decoded parts);
3392
3393
- when collecting file names to be virus scanned, prepare a hash which maps
3394
base file names to Amavis::Unpackers::Part objects, and make it available
3395
to virus scanner interface routines, which may benefit from the additional
3396
information about the file to be scanned.
3397
3398
In particular, the new interface to Mail::ClamAV now turns on the option
3399
CL_MAIL, and the interface to SAVI turns on the option MIME, when entire
3400
mail is passed to AV scanner for checking. This enables ClamAV and SAVI
3401
to attempt MIME decoding the file by itself.
3402
3403
TODO: The same option (--mime) would need to be specified when calling
3404
'clamscan' and supplying a non-decoded mail for checking; pointed out
3405
by Riccardo Ghiglianovich and Michael Boelen;
3406
3407
3408
OTHER EXTERNALLY VISIBLE CHANGES
3409
3410
- The 'amavisd reload' command is now implemented differently:
3411
old: signals SIGHUP to a running amavisd process and exits immediately;
3412
the running amavisd process (under control of Net::Server) when it
3413
receives a SIGHUP starts its own copy with same arguments and exits;
3414
new: signals SIGTERM to a running amavisd process, waits for it to finish,
3415
then continues (same as 'amavisd start') to become a new daemon;
3416
The new method works even when chrooted, and is more reliable when the
3417
existing process is slow to terminate, as it actively waits for the previous
3418
daemon to finish before proceeding to promote itself to become a new daemon.
3419
3420
- a simple demo program 'amavisd-agent' is provided, allowing
3421
for continuous inspection of SNMP counters; a path to the
3422
/var/amavis/db is hardwired in the program, modify it if necessary.
3423
3424
- server-side support for optional Postfix SMTP/LMTP command XFORWARD:
3425
information about the original SMTP client IP, its DNS name, HELO name
3426
and protocol used is now made available to the amavisd program for
3427
logging and other purposes. The same information can also be obtained
3428
from the 'Amavis policy delegation protocol (AM.PDP)' if the helper
3429
program supports it (useful for sendmail milter setup);
3430
3431
- client-side support for optional Postfix SMTP command XFORWARD:
3432
if MTA announces in its SMTP EHLO response that it supports XFORWARD,
3433
amavisd will provide additional information about the original SMTP client
3434
if the information is available (either from XFORWARD on the receiving
3435
side, or by the 'Amavis policy delegation protocol';
3436
3437
- server side support for the new amavis helper protocol AM.PDP which allows
3438
for header modifications, removal of recipient addresses (e.g. non- spam
3439
lovers) or rewrite of recipient addresses (e.g. adding address extensions),
3440
and specification of full SMTP response;
3441
3442
- modified search logic for matching mail parts against $banned_filename_re;
3443
3444
The old search order for names did not result in what one might
3445
expect when pattern list elements with a value of false were used
3446
in $banned_filename_re. Namely, all three components were searched
3447
independently (Content-Type, declared name, and file(1) type)
3448
and a logical or was used. Because searches for each mail part were
3449
independent, it was not possible to specify for example that anything
3450
within a zip would be allowed. If any of these searches returned true,
3451
mail was blocked.
3452
3453
To make this useful, a complete rewrite of mail unpacking was needed
3454
and all information be made available in one place after the unpacking
3455
is over, so that checking for banned names can be done all at once.
3456
3457
The search order is now much the same as used in rsync and its server,
3458
see man rsync, section 'EXCLUDE PATTERNS'. The new comments in
3459
amavisd.conf-sample explain the new situation.
3460
3461
- replaced $relayhost_is_client by a more flexible specification:
3462
3463
To make it possible for several hosts to share one content checking daemon,
3464
the IP address and/or the port number in $forward_method and $notify_method
3465
may be specified as an asterisk. An asterisk in the colon-separated
3466
second field (host) will be replaced by the SMTP client peer address
3467
(i.e. the MTA host). An asterisk in the third field (tcp port) will be
3468
replaced by the incoming SMTP/LMTP session port number plus one. This
3469
obsoletes the previously used less flexible configuration parameter
3470
$relayhost_is_client. An example:
3471
3472
$forward_method = 'smtp:*:*'; $notify_method = 'smtp:[127.0.0.1]:10025';
3473
3474
The same functionality can also by achieved by using a bigger hammer,
3475
the policy banks. These may completely replace the global settings
3476
for $forward_method and $notify_method, based on incoming port number;
3477
3478
- turn address extension variables (the so called "plus addressing")
3479
into recipient-based lookup tables, including the SQL lookups. For example:
3480
@addr_extension_virus_maps = ('virus'); # defaults to empty
3481
@addr_extension_spam_maps = ('spam'); # defaults to empty
3482
@addr_extension_banned_maps = ('banned'); # defaults to empty
3483
@addr_extension_bad_header_maps = ('badh'); # defaults to empty
3484
or perhaps:
3485
@addr_extension_virus_maps = (
3486
{'sub.example.com'=>'infected', '.example.com'=>'malware'}, 'virus' );
3487
suggested by Gentoo modification, Jacques Supcik, and others;
3488
3489
- log and report hits and tag/tag2/kill levels rounded to three decimal
3490
places (trailing zeroes trimmed), no longer rounded to one decimal place;
3491
3492
- added @spam_dsn_cutoff_level_maps, making it possible to specify
3493
different DSN cutoff levels for different recipient domains or users.
3494
In multi-recipient messages where recipients can specify different values,
3495
the maximum value is used for deciding whether DSN should be suppressed;
3496
suggested by Ales Casar;
3497
3498
- configuration variable $gets_addr_in_quoted_form is no longer used;
3499
knowledge about address form (quoted or not) is now implicit in the
3500
receiving protocol;
3501
3502
- if tag level turns out to be undef, it will not be shown in X-Spam-*
3503
header fields, and will be interpreted as having a value lower than any
3504
spam score when deciding whether to insert X-Spam-* header fields or not;
3505
3506
- added macros %a and %g:
3507
* %a original SMTP session client IP address (empty if unknown)
3508
* %g original SMTP session client DNS name (empty if unknown)
3509
(like macros %I and %M that were once proposed by Dibo in his 2002-07 patch)
3510
This information may be available from Postfix when XFORWARD protocol
3511
extension to SMTP is enabled, and can be made available by helper program
3512
(e.g. from sendmail milter) when the new AM.PDP protocol is used;
3513
3514
- added macro %p, expanding to a current policy bank name (or empty
3515
if a built-in policy bank is still in place);
3516
3517
- added macro %r, expanding to the contents of the first Resent-Message-ID
3518
header field, or empty if no such field exists. Include reporting the
3519
Resent-Message-ID in the log and in the sender notification;
3520
suggested by Oliver Gorwits;
3521
3522
- new configuration variable $addr_extension_bad_header for completeness;
3523
3524
- added $bad_header_quarantine_to, @bad_header_quarantine_to_maps,
3525
$bad_header_quarantine_method, and $warnbadhrecip for completeness;
3526
suggested by Robin Lynn Frank;
3527
3528
- MIME::Parser errors now contribute to bad-header checks, so that the
3529
header checking is now conceptually extended to MIME sub headers
3530
(Postfix similarly considers MIME subheaders part of mail headers);
3531
MIME::Parser 6.1xx or later is recommended.
3532
3533
- allow $*_quarantine_method to be undef as a quick way of disabling
3534
some quarantine (it also can be disabled as previously, by using
3535
method 'local:' and following its rules);
3536
3537
- persistent cache of recent virus and spam checks, global to all child
3538
processes, can improve the cache hit rate. Uses BerkeleyDB database (hash
3539
and queue) and its interlocking mechanisms (Berkeley DB Concurrent Data
3540
Store) for the purpose. The V3.1 or better is required, V4 is recommended.
3541
3542
- include version information in the 'Usage' text;
3543
3544
- rewritten lookup_ip_acl() and added ip_to_vec() to allow for IPv6
3545
address syntax as specified in rfc3513 to be used in IP lookups;
3546
3547
- @inet_acl now defaults to ('127.0.0.1', '::1'), i.e. it adds the
3548
IPv6 loopback address to the list;
3549
3550
- new configuration variable $sa_spam_level_char (defaults to '*')
3551
allows specifying another character for X-Spam-Level bar. Empty or
3552
undefined value disables inserting the X-Spam-Level header field;
3553
3554
- added configuration variable $sa_spam_report_header to enable/disable
3555
inserting the X-Spam-Report header; patch by Craig Sanders;
3556
3557
- added $banned_quarantine_to configuration setting to make possible the
3558
quarantining of banned mail to a different place from viruses;
3559
3560
- don't insert virus-, banned- and bad headers- related headers for passed
3561
mail to recipients with corresponding bypass_*_checks, making them believe
3562
the mail was not spam-checked (as they are not expecting such headers
3563
anyway); This was already done in version amavisd-new-20030616-p6
3564
but only for spam-related headers;
3565
3566
- for choosing address extensions use the same criteria as for adding
3567
header fields, e.g. pretend to not know the result of a certain test
3568
(virus, spam, ...) when recipient chooses to bypass such test,
3569
even if the result of the test is known;
3570
3571
- added variable $sa_spam_subject_tag1 (undef by default).
3572
If $sa_spam_subject_tag will not be inserted (at tag2 level), and
3573
$sa_spam_subject_tag1 is nonempty, this string (e.g. '***possible SPAM*** ')
3574
will be inserted into the Subject header field for spam levels above
3575
tag level; suggested by Immo Goltz;
3576
3577
- added separate configuration variables $banned_files_quarantine_method
3578
and $bad_header_quarantine_method. Quarantining of banned files and
3579
bad headers were previously controlled by $virus_quarantine_method;
3580
3581
- rewritten read_hash, it is now possible to specify key value (right-hand
3582
side) for each key. If value is not specified, a '1' is assumed as before;
3583
3584
- SMTP server support for rfc2554 authentications (PLAIN and LOGIN only)
3585
allows client authentication to be relayed to the MTA when message is
3586
forwarded. Might be useful if amavisd-new is used in a Postfix SMTP proxy
3587
setup, but is not needed for other setups. Disabled by default,
3588
see variables $auth_required_inp and @auth_mech_avail.
3589
3590
- SMTP client support for rfc2554 authentications (any authentication method
3591
as supported by Net::SMTP and Authen::SASL Perl modules). Authentication
3592
of forwarded mail (PLAIN and LOGIN only) is carried over from the incoming
3593
mail, authentication to be used when submitting notifications is controlled
3594
by configuration variables $amavis_auth_user and $amavis_auth_pass.
3595
Disabled by default, see variable $auth_supported_out (later renamed
3596
to $auth_required_out).
3597
3598
- provisional/experimental support for DSPAM spam checker (pre 3.0):
3599
if configuration variable $dspam is nonempty and represents a path to
3600
a 'dspam' program, a message is passed to dspam and its inserted headers
3601
of the form X-DSPAM-* are axtracted and then made available for
3602
SpamAssassin rules to check and score if desired.
3603
3604
All messages are currently presented to dspam as the same user, affecting
3605
how its database is being built. False negatives and false positives
3606
(based on SA assessment) are fed back into DSPAM as a simple form
3607
of auto-learning. Works reasonably well, but do not expect miracles.
3608
See subroutine spam_scan.
3609
3610
Here is how DSPAM can be installed to be able to be used by amavisd-new:
3611
3612
dspam 2.x:
3613
./configure --enable-alternative-bayesian \
3614
--with-userdir=/var/amavis/dspam --enable-signature-headers \
3615
--without-local-delivery-agent --without-quarantine-agent
3616
3617
dspam 3.0.0:
3618
./configure \
3619
--with-dspam-home=/var/amavis/dspam --enable-signature-headers \
3620
--without-delivery-agent --without-quarantine-agent
3621
3622
make install
3623
chmod u-s,a+rx /usr/local/bin/dspam
3624
chown vscan:vscan /var/amavis/dspam
3625
3626
User 'vscan' may need to be added to DSPAM trusted.users file.
3627
3628
The following can be inserted into the SA config file (local.cf)
3629
to make it recognize and incorporate DSPAM's assessment:
3630
3631
header DSPAM_SPAM X-DSPAM-Result =~ /^Spam$/
3632
describe DSPAM_SPAM DSPAM claims it is spam
3633
score DSPAM_SPAM 0.5
3634
3635
header DSPAM_HAM X-DSPAM-Result =~ /^Innocent$/
3636
describe DSPAM_HAM DSPAM claims it is ham
3637
score DSPAM_HAM -0.1
3638
3639
Eventually DSPAM support should be removed from amavisd-new, as soon as
3640
SA will be able to call it on its own.
3641
3642
3643
INTERNAL CHANGES
3644
3645
- reformatted the whole program, reducing indentation from 4 to 2
3646
and replacing TABs with spaces (with some dubious help from perltidy,
3647
plus lots of manual adjustments);
3648
3649
- completely rewritten code to handle both the old and the new amavis helper
3650
protocol, as well as Postfix 'TCP client/server table lookup protocol'
3651
as specified in the Postfix documentation: tcp_table(5);
3652
(process_policy_request, prepare_policy_query, check_amcl_policy)
3653
3654
- type_short may now be a list of short types, not necessarily just a single
3655
value. Typical use is to classify a MS executable as both an 'exe' and
3656
as 'exe-ms', which makes more specific banned rules possible without
3657
unnecessary complication in regexps;
3658
3659
- parts now carry attributes, which can be inspected for banned checks;
3660
current attributes are U for undecodable, and C for (en)crypted;
3661
3662
- opened another can of Perl worms (taint bugs): turn on Perl pragma
3663
"use re 'taint'" in all modules, and selectively turn it off where needed.
3664
It replaces cumbersome manual preservation of taintedness when regexp
3665
saved ranges are used without intention to untaint. Because of Perl bugs,
3666
strategically placed local($1,$2,...) are needed, otherwise previous
3667
taint flag in $1, $2, ... can be brought on to new variables, which can
3668
all of a sudden become tainted out of nowhere;
3669
3670
- catch and report throws (die) in pre_loop_hook() to properly report
3671
problems during initialization;
3672
3673
- introduced subroutine exit_status_str and unify reporting of
3674
subprocess status;
3675
3676
- enhanced sub best_try_originator to ignore IP addresses from private,
3677
local and dedicated IANA networks (rfc3330) and look for the first
3678
public address in the 'Received' path;
3679
3680
- examine first four (chronologically) Received header fields (instead of
3681
first two) when looking for an originator IP address, and ignore those
3682
with private IP addresses;
3683
3684
- moved code dealing with body digest cache to a new package
3685
Amavis::Cache to facilitate transition to shared or persistent cache;
3686
3687
- new explicit cache expiration time controls (time to live in seconds):
3688
$virus_check_negative_ttl $virus_check_positive_ttl
3689
$spam_check_negative_ttl $spam_check_positive_ttl
3690
$cache_entry_ttl
3691
3692
- discard cache db ($db_home) and recreate it during restart;
3693
3694
- more informative changes of child process $0, which may show in the ps(1)
3695
output;
3696
3697
- store tempdir of a current message to the Amavis::In::Message object;
3698
3699
- gather some statistics about idle time;
3700
3701
- reorder and adjust mapping from file(1) results to file type classes;
3702
3703
- optimization: instead of invoking file(1) utility program for each
3704
part to be analyzed, now call it once for each round of currently
3705
available parts, giving it the list of all available parts as arguments;
3706
3707
- shorten the names of parts from part-..... to p..., to be able to stash
3708
more files names into a command line, e.g. when calling file(1) or
3709
external virus checker which can not deal with directories;
3710
3711
- use regexp lookup table mechanism (table $map_full_type_to_short_type)
3712
to match long types (output of file(1) utility) to short types (.exe,
3713
.jpg, .doc, ...). The default table can be replaced by a customized
3714
table in amavisd.conf;
3715
3716
- replace $(?!\n) with \z in regular expressions throughout;
3717
replace most of the remaining $ with \z in regular expressions.
3718
The regular expression primitive \z is available since perl 5.005.
3719
3720
- TODO: disregard $MAXFILES during initial MIME unpacking;
3721
reported by Stephane Lentz and Robert LeBlanc (done in 2.3.2);
3722
3723
3724
3725
3726
Some un-edited notes on the new banned rules mechanism:
3727
(wrapped log lines, and replaced \\ by \ for clarity:
3728
3729
| Feb 24 19:07:29 hauptpostamt amavis[29847]: (29847-04-5) p.path BANNED:
3730
|
3731
| "P=p002,M=application/octet-stream,T=zip,N=document.zip |
3732
| P=p003,T=exe,T=exe-ms,N=document.htm .scr",
3733
3734
part p003 is of type (file(1)) MS executable, with suggested
3735
name "document.htm .scr" (lots of spaces in the name)
3736
3737
its parent resides on temp file p002 (i.e. p003 was extracted from it),
3738
which is of type (T) zip archive, with suggested (MIME) name
3739
(N) "document.zip", and has a MIME type (M) "application/octet-stream".
3740
3741
such a component p003 lying within such p002 is considered banned
3742
by the following regexp rule (one rule within the $banned_namepath_re list):
3743
3744
| matching_key="(?mix-s:^ (.*\t)? N= [^\t\n]* \. [^./\t\n]* \.
3745
| (exe|vbs|pif|scr|bat|cmd|com|dll) (\t.*)? $)"
3746
3747
which says that any component at any level must not have a name (N)
3748
matching a pattern:
3749
any number of characters,
3750
a dot,
3751
any number of non-dot and non-slash characters
3752
a dot,
3753
and ending with: exe or vbs ...
3754
(basically: double extension ending with listed extensions)
3755
3756
The complications such as using [^\t\n]* instead of .* are there
3757
to keep regexp contained within fields and ancestors/descendents.
3758
3759
3760
There is one detail to remember when comparing logged p.path log entries
3761
and the actual matching rules:
3762
3763
- for the sake of readability the logged entry has \n (newlines) converted
3764
into ' | '. The \n is a separator between components in the tree
3765
from the root (the mail itself, hidden) to the leaf component
3766
which can not be further expanded (i.e. not an archive)
3767
3768
- for the sake of readability the logged entry has \t (a tab) converted
3769
into comma, separating information fields such as P=... M=.. T=.. M=..
3770
3771
So the above logged string:
3772
P=p002,M=appl...,T=zip,N=document.zip | P=p003,T=exe,T=exe-ms,N=document.htm.scr
3773
is actually a single string:
3774
P=p002\tM=appl...\tT=zip\tN=document.zip\nP=p003\tT=exe\tT=exe-ms\tN=document.htm.scr
3775
3776
and a Perl regexp is applied directly to it.
3777
3778
The raw string is rather unsightly, but the \n and \t were chosen
3779
to minimize clash with valid characters within file names.
3780
3781
If a \n or \t is present in a name of the components, such character
3782
is converted into a space to avoid clashing with separators.
3783
3784
3785
| Feb 24 19:11:58 hauptpostamt amavis[31505]: (31505-01-5) p.path BANNED:
3786
| "P=p002,M=application/octet-stream,T=zip,N=paypal.zip |
3787
| P=p003,T=exe,T=exe-ms,N=paypal.scr",
3788
3789
a MS executable named "paypal.scr" within a zip archive "paypal.zip"
3790
3791
| matching_key="(?mix-s:^ (.*\t)? N= [^\t\n]* \.
3792
| (exe|vbs|pif|scr|bat|com) (\t.*)? $)"
3793
3794
block component at any level with a name (N) terminating
3795
by dot followed by any of the listed extensions.
3796
3797
3798
3799
| Feb 24 19:18:25 hauptpostamt amavis[32159]: (32159-01-2) p.path BANNED:
3800
| "P=p002,M=application/octet-stream,T=zip,N=text.zip |
3801
| P=p003,T=exe,T=exe-ms,N=text.txt .exe",
3802
3803
a MS executable named "text.txt .exe" (with lots of spaces in the name)
3804
within a zip archive named "text.zip"
3805
3806
| matching_key="(?mix-s:^ (.*\t)? N= [^\t\n]* \. [^.\t\n]* \.\n
3807
| (exe|vbs|pif|scr|bat|cmd|com|dll) (\t.*)? $)"
3808
3809
blocked by the double-extension rule.
3810
3811
3812
| Feb 24 19:30:15 hauptpostamt amavis[1690]: (01690-02-8) p.path BANNED:
3813
| "P=p002,M=application/octet-stream,T=zip,N=jokes.zip |
3814
| P=p003,T=exe,T=exe-ms,N=jokes.doc .exe",
3815
| matching_key="(?mix-s:^ (.*\t)? N= [^\t\n]* \. [^.\t\n]* \.\n
3816
| (exe|vbs|pif|scr|bat|cmd|com|dll) (\t.*)? $)"
3817
3818
same thing
3819
---------------------------
3820
3821
3822
---------------------------------------------------------------------------
1
3823
June 29, 2004
2
3824
Patch: amavisd-new-20030616-p10
3
3825
4
- insert a security check for a missing Net::Server patch, and abort
3826
- insert a security check to test for a missing Net::Server patch, and abort
5
3827
if vulnerable;
6
3828
7
3829
- provide and use our own subroutine q_encode to do the Q-encoding when
24
3846
* relax permissions on created directory and temporary file to allow
25
3847
group read access (needed if virus scanner runs under a different
26
3848
user id but within the same group); by Adam C. Migus;
27
* add queue id (when available) ) to most log messages;
3849
* add queue id (when available) to most log messages;
28
3850
29
- do not preserve evidence just because a message gets an X-AMaViS-HOLD
3851
- do not preserve evidence just because a message gets an X-Amavis-Hold
30
3852
header field;
31
3853
32
- recognize SQL server error 2013/"Lost connection to" and treat it
33
the same as 2006/"MySQL server has gone away"; by Max Kalika;
34
35
- added amavisSpamQuarantineTo to the LDAP schema;
36
new version of LDAP schema; by Jacques Supcik;
37
38
3854
- when logging directly to a file and started as root, change UID of a
39
3855
log file to $daemon_user to avoid restart problems; based on a patch
40
by Carsten Hoeger;
3856
by Carsten Hoeger and Gregor Weiss;
41
3857
42
3858
- added configuration variable $first_infected_stops_scan
43
3859
to stop anti-virus scanning when the first scanner detects a virus;
44
the default is false, all scanners are called (as usual);
3860
the default is false, all scanners in a group are called (as usual);
45
3861
46
3862
- in the AV entry for clamscan added the option '--tempdir=$TEMPBASE';
47
3863
55
3871
introduced in -p8; removing it gets in a way when more than one content
56
3872
filter is being chained;
57
3873
58
- missing a SpamAssassin v3.0 module when running in chroot;
3874
- load a missing SpamAssassin v3.0 module, needed when running in chroot;
59
3875
60
3876
- contributed (for now in the form of an optional patch): support for
61
3877
the incoming qmqpqq protocol over a TCP socket - to be used with qmail.
76
3892
fixed, thanks to Francis Stevens;
77
3893
78
3894
- updates to the @av_scanners list in amavisd.conf:
79
* DrWebD now works with new DrWeb daemon 4.31,
80
thanks to information provided by Krzysztof Cegielski;
3895
* DrWebD now works with a new DrWeb daemon 4.31,
3896
thanks to information provided by Krzysztof Cegielski, DrWeb Polska;
81
3897
* updated BitDefender bdc, thanks to Alfredo Milani Comparetti;
82
3898
* clamscan: use documented option --no-summary instead of the
83
3899
undocumented/old(?) option --disable-summary; by Georgy Salnikov;
88
3904
statuses 2 and 5 considered infected; all suggested by Georgy Salnikov;
89
3905
90
3906
- when original undecoded mail is to be kept for virus scanners (requested
91
by patterns /^MAIL$/ or /^MAIL-MAIL-UNDECIPHERABLE$/ in $banned_filename_re),
3907
by patterns /^MAIL$/ or /^MAIL-UNDECIPHERABLE$/ in $banned_filename_re),
92
3908
the preserved file is now named parts/part-00000 instead of parts/email.txt
93
to preserve the size or the name, upon which some virus scanners depend
3909
to preserve the size of the name, upon which some virus scanners depend
94
3910
(e.g. DrWebD);
95
3911
96
3912
A note to Courier users: due to the way a file name in Courier is created
97
3913
and passed to amavisd-new, it is currently not possible to use the triggers
98
3914
/^MAIL$/ and /^MAIL-UNDECIPHERABLE$/ in $keep_decoded_original_re, or a
99
failure to create a hard link occurs; I hope to have this limitation lifted
100
soon. Thanks to Bowie Bailey for helping to troubleshoot the problem;
3915
failure to create a hard link occurs; Thanks to Bowie Bailey for helping to
3916
troubleshoot the problem; (this limitation is lifted in amavisd-new-2.2.0)
101
3917
102
3918
- remove option -c when calling gzip, bzip2, compress, lzop and unfreeze
103
3919
(as has long been done in the development version); the option -c is not
111
3927
running in chroot jail;
112
3928
113
3929
- look for program 'gcpio' ( $cpio = ['gcpio','cpio'] ), as on OpenBSD the
114
plain cpio does not support the required options (--no-absolute-filenames
3930
plain cpio does not support required options (the --no-absolute-filenames
115
3931
is essential), but GNU cpio does; thanks to Manfred Gloiber;
116
3932
117
3933
- relax parsing of file(1) output to allow tab as well as space to follow
185
4001
http://marc.theaimsgroup.com/?l=amavis-tech&m=102569946910555
186
4002
187
4003
* make a clear distinction between message data and connection data,
188
which required code reshuffling and revealed previous unlean solutions;
4004
which required code reshuffling and revealed previous unclean solutions;
189
4005
190
4006
* add error checking and reporting to mkdir/rmdir/open/unlink/write
191
4007
system calls; previously an error could pass by unnoticed or just
224
4040
225
4041
* ability to present the full non-decoded original message to virus
226
4042
scanners was partly back-ported from the development version.
227
Enabled by adding qr'^MAIL$' or qr'^MAIL-MAIL-UNDECIPHERABLE$' to the
4043
Enabled by adding qr'^MAIL$' or qr'^MAIL-UNDECIPHERABLE$' to the
228
4044
list in $keep_decoded_original_re, as illustrated in amavisd.conf;
229
4045
similar to a patch by Ted Cabeen. The following keys are used for a
230
4046
lookup into $keep_decoded_original_re:
231
4047
232
4048
'MAIL'
233
always provide the full original message to virus scanners
4049
always provide a full original message to virus scanners
234
4050
(besides its successfully decoded components);
235
4051
236
4052
'MAIL-UNDECIPHERABLE'
243
4059
* a key 'UNDECIPHERABLE' is matched against $banned_filename_re when mail
244
4060
contains any undecipherable components, and if lookup returns true,
245
4061
mail will be banned. For example:
246
4062
247
4063
$banned_filename_re = new_RE(
248
4064
qr'^UNDECIPHERABLE$', # contains any undecipherable components
249
4065
... );
255
4071
$undecipherable_subject_tag = '***UNCHECKED*** '; # undef disables it
256
4072
257
4073
NOTE: this solution is a quick-fix response to popular demand.
258
Although the same or similar functionality will probably remain in future
259
versions, it is likely that the syntax and exact semantics will be refined.
4074
Although the same or similar functionality will probably remain
4075
in future versions, the syntax and exact semantics may be refined.
260
4076
261
4077
- bring do_unarj in line with the rest of de-archivers: provide the
262
4078
same degree of mail bomb protection that was available in do_unrar;
274
4090
to suppress sending a DSN (delivery status notification) when spam level
275
4091
is above this value, effectively turning D_BOUNCE into D_DISCARD for
276
4092
this message; undef disables the feature and is a default. A good
277
first approximation value is 10, or with some risk go down to 8 or 7.
4093
first approximation value is 10, or with some risk go down to 8.
278
4094
The parameter has no effect if DSNs are already disabled (e.g. when
279
4095
$final_spam_destiny is D_DISCARD or D_REJECT);
280
4096
284
4100
sender address, so as to not rise false alarms when sender address is known
285
4101
to be faked; patch by Joerg Friedrich;
286
4102
287
- consider the SQL user id a string (no longer required to be numeric);
288
thanks to Max Kalika / Gentoo support;
289
290
4103
- reduce the number of retries to connect to a daemonized virus scanner
291
4104
from 3 to 2, so that a fallback to backup scanners occurs sooner;
292
4105
298
4111
status only reflects the result of the last file scanned, we must use
299
4112
regular expression to detect viruses; fix provided by Andreas Triller;
300
4113
301
* Trend Micro FileScanner - vscan: added option -za to the entry, otherwise
4114
* Trend Micro FileScanner - vscan: added option -za, otherwise
302
4115
some broken archive may sneak-in a virus; suggested by Stephane Lentz;
303
4116
304
4117
* Sophos sweep: added options -cab -tnef --no-reset-atime;
324
4137
to a mailbox;
325
4138
326
4139
- helper-progs/amavis-milter.c (sendmail) enhancement based on work
327
by Stephane Lentz:
4140
by Stephane Lentz:
328
4141
329
4142
* the name of a temporary directory is derived from the sendmail queue ID,
330
4143
making it easier to match sendmail and amavisd-new log entries;
478
4291
problem reported by Aspa and by Tomasz Papszun;
479
4292
480
4293
- fix a potential security problem: don't let rmdir_recursively and
481
rmdir_flat follow symbolic links; this might be exploited by trying
4294
rmdir_flat follow symbolic links; this might be exploited by attempting
482
4295
to delete some foreign file using privileges of the amavisd process
483
4296
(which should not be root);
484
4297
496
4309
on the most recent version 1.1.2.3.2.40 from the AMaViS CVS (maintained
497
4310
by Lars Hecking), but hacked a bit to make its options mostly compatible
498
4311
with the previous version. Start it with option -h to get current usage text.
499
See helper-progs/README is using sendmail milter setup;
4312
See helper-progs/README if using sendmail milter setup;
500
4313
501
4314
Please revert to the old one if the .40 gives you trouble;
502
4315
both version are included in the helper-progs subdirectory.
552
4365
- added option '-packed' to 'FRISK F-Prot Daemon' entry;
553
4366
thanks to Manfred Gloiber;
554
4367
555
- NAI uvscan entry: commented pre/post actions in the entry to show
4368
- NAI uvscan entry: commented pre/post actions in the entry to show
556
4369
how to remove environment variable LD_PRELOAD after finishing,
557
4370
suggested by David Tilley; added option --mime, suggested by Max and
558
4371
Kevin W. Gagel; added note on how to treat password-protected files
559
4372
as viruses, by Seth Parker;
560
4373
561
4374
- added 'Dr.Web Daemon' entry which talks directly to Dr.Web daemon over
562
its Unix socket, speeding up a single-file check by more than 200 times;
4375
its Unix socket, speeding up a single-file check more than 200 times;
563
4376
provided by Andrew I Baznikin;
564
4377
565
4378
- another entry for Symantec AntiVirus Scan Engine provided by Guido
589
4402
August 25, 2003
590
4403
Patch: amavisd-new-20030616-p5
591
4404
592
- fix 'Modification of non-creatable array value attempted' bug when
4405
- fix 'Modification of non-creatable array value attempted' bug when
593
4406
no 'Received' header field was present in an infected mail;
594
4407
reported by Paul Miner;
595
4408
648
4461
versions of pavcl; updated entry kindly provided by Panda Software;
649
4462
650
4463
- stop the timer after SMTP transaction is over to better behave in
651
persistent SMTP/LMTP sessions; start the timer at the beginning of
4464
persistent SMTP/LMTP sessions; start the timer at the beginning of a
652
4465
SMTP transaction, in addition to restarting it when DATA mode is entered;
653
4466
654
4467
- sub mail_via_smtp_single: properly report SMTP response code when all
789
4602
790
4603
- commented-out adding of 'X-Spam-Report' header field to retain behaviour
791
4604
from amavisd-new-20030314 - I find this header too long and intrusive
792
for regular use. Uncomment it, if adding X-Spam-Report is desired;
4605
for regular use. Uncomment it if adding X-Spam-Report is desired;
793
4606
794
4607
- support lzop (.lzo) and freeze (.F) compressors if available;
795
4608
858
4671
being converted to 2xx as D_BOUNCE would suggests; reported by Peter Bates;
859
4672
860
4673
- security: avoid inadvertently untainting values (e.g. mail addresses)
861
at several places throughout the program;
4674
at several places throughout the program;
862
4675
863
4676
- work around a Perl 5.8.0 taint bug where global variable $1 could become
864
4677
tainted, making further untainting attempts unsuccessful;
883
4696
- new subroutine flatten_and_tidy_dir() recursively descends into a directory
884
4697
containing potentially unsafe files with unpredictable names, soft links,
885
4698
etc., rename each regular nonempty file giving it a generated name, and
886
discard all the rest.
4699
discard all the rest.
887
4700
888
4701
- do_unrar and do_unarj now let the archiver itself recursively unpack
889
4702
the archive and then use flatten_and_tidy_dir() to tidy up the result.
948
4761
- change log level of several less important av scanner log messages
949
4762
from 2 to 3;
950
4763
951
- small documentation updates in amavisd,conf;
4764
- small documentation updates in amavisd.conf;
952
4765
953
4766
- updated README.lookups: the new substring capture and reference mechanism
954
4767
in regexp lookups explained. The SQL example modified to work under
960
4773
961
4774
NEW FEATURES
962
4775
963
- provide a command line option 'debug-sa', which is equivalent (but shorter)
964
then setting $sa_debug to true and starting as '# amavisd foreground';
4776
- provide a command line option 'debug-sa', which is equivalent
4777
(but more convenient) then setting $sa_debug to true and starting
4778
as '# amavisd foreground';
965
4779
966
4780
- list version numbers of the more important Perl modules at startup;
967
4781
980
4794
- read_hash() now returns the hashref (first argument) also as a return
981
4795
value, making it easier to create new hashes (instead of only adding values
982
4796
to existing ones). Also the first argument (hashref) is now optional,
983
and if missing a new empty hash is created. The following three cases
984
are now equivalent:
4797
and if missing a new hash is created. The following three cases are now
4798
equivalent:
985
4799
986
4800
a) $sa_tag2_level_deflt = {};
987
4801
read_hash($sa_tag2_level_deflt, '/var/amavis/tag2_levels.dat');
1101
4915
*_destiny=D_PASS; reported by Steve Khoo. The "deal_with_spam" section
1102
4916
needed to be overhauled for this reason. For consistency with the
1103
4917
documentation and with the change in adding address extensions,
1104
quarantining is now done is mail is considered spam, regardless of
4918
quarantining is now done if mail is considered spam, regardless of
1105
4919
it being delivered (e.g. to spam lovers), or not;
1106
4920
1107
4921
- if only spam checking is performed, but no virus scanning nor banned
1139
4953
which could result in multiple deliveries if MTA on the input side
1140
4954
times out;
1141
4955
1142
- debugging aids: subroutine call backtrace is logged if
4956
- debugging aids: call stack backtrace is logged if
1143
4957
Mail::SpamAssassin::NoMailAudit::check exceeds allowed time;
1144
4958
1145
4959
- LDAP enhancement: allow "hostname" to be a string or a reference to an
1298
5112
1299
5113
- SQL SELECT clause settable from the config file;
1300
5114
1301
- can store quarantined or forwarded messages as BSMTP files;
5115
- can store quarantined or forwarded messages as BSMTP files (RFC 2442);
1302
5116
1303
5117
1304
5118
1404
5218
and no longer every time the child process is started. This is because
1405
5219
starting with SAVI 3.0 (March 2003) the initialization is slower.
1406
5220
1407
NOTE: it is now necessary to restart (HUP) amavisd master process
5221
NOTE: it is now necessary to restart amavisd master process
1408
5222
after SAVI database is updated, to make it notice new virus descriptions.
1409
5223
1410
5224
- in 'MkS_Vir for Linux' entry replaced options -e -c with -s,
1475
5289
native or UTF-8 Perl strings, which is handled (mostly) transparently
1476
5290
by Perl 5.8.x;
1477
5291
5292
- added option $hdr_encoding_qb to be able to explicitly specify
5293
MIME encoding ("Q" or "B") for $hdr_encoding MIME charset;
5294
1478
5295
- rfc2822_timestamp: produce correct date format regardless of
1479
5296
LC_TIME locale setting;
1480
5297
1574
5391
and kill_level. A quick reference:
1575
5392
tag_level controls adding the X-Spam-Status and X-Spam-Level headers,
1576
5393
tag2_level controls adding 'X-Spam-Flag: YES', and editing Subject,
1577
kill_level controls 'evasive actions' (reject, quarantine, extensions);
5394
kill_level controls 'evasive actions' (reject, quarantine);
1578
5395
The default is tag2_level=kill_level, which retains compatibility.
1579
5396
1580
5397
- work around a bug in Perl 5.8.0 in UTF-8 environment, which could
1798
5615
amavisd for every change in the Sophos IDE database - at the expense
1799
5616
of additional SAVI-Perl startup time every $max_requests messages;
1800
5617
5618
---------------------------------------------------------------------------
5619
December 30, 2002
1801
5620
1802
5621
amavisd-new-20021227-p1 (2002-12-30):
1803
5622
fix the problem when using multiple virus scanners, where the
2450
6269
'Found trojan or variant Exploit-CodeBase !!!', a patch by Anton Berezin;
2451
6270
2452
6271
- Clam Antivirus supported;
2453
6272
2454
6273
- fixed a problem in 'avp' scanner interface ($TEMPBASE not imported),
2455
6274
thanks to Joshua E Warchol;
2456
6275
2693
6512
of spam exceeds 64k characters, and this is probably well below the
2694
6513
false-negative SA rate, so it is not worth wasting time to check large mail.
2695
6514
2696
6515
2697
6516
remaining changes:
2698
6517
2699
6518
- new file README.exim (thanks to Jochen Erwied, Patrice Fournier
2738
6557
2739
6558
- avoid changing sender address <> to <""> under certain circumstances
2740
6559
(not strictly wrong, but still a bad idea);
2741
6560
2742
6561
- replaced header 'X-Razor-Warning:' with SA-compatible 'X-Spam-Status:' ;
2743
6562
the 'X-Razor-id:' is still provided to facilitate user spam reporting;
2744
6563
Loggerhead is a web-based interface for Breezy
Version: 2.0.1