1
.TH CALAMARIS 1 "$Date: 2003/05/14 21:22:37 $" Handmade "Calamaris Manual"
4
calamaris \- generate statistic out of log files from Proxy-Cache-Servers
1
.TH "CALAMARIS" "1" "$Date: 2006-03-19 17:52:48 $" "Handmade" "Calamaris Manual"
5
- generate text and graphical statistics out of log files from
13
is used to produce statistical output from
16
.B Inktomi Traffic Server,
17
.B Oops! proxy server,
19
.B Cisco Content Engines
20
or related Proxy log files. The resulting output can be
24
It is possible to cache calculated data in a file to use them in later runs.
26
This manual page describes the options of
28
and gives a few examples.
11
[ --config-file .../calamaris.conf ] [ switches ]
15
is used to produce statistical output from \fBSquid\fR, \fBNetCache\fR,
16
\fBInktomi Traffic Server\fR, \fBOops! proxy server\fR,
17
\fBCompaq Tasksmart\fR, \fBCisco Content Engines\fR or related Proxy log
18
files. The resulting output can be ascii or html with or without graphic
19
and with or without frames. It is possible to cache calculated data in a
20
file to use them in later runs.
22
This manual page describes the options of \fBCalamaris\fR and gives a few
26
.SS Configuration File
31
Not all reports and modification can be made through command-line-switches.
32
To use all options of Calamaris you\'ll have to use the configuration file.
33
You'll find the configuration-directives below, always inside of braces.
34
Examples are in the calamaris.conf which should come with this package.
36
all: extracts all reports available,
39
.I --all-useful-reports|-a
40
extracts all useful reports available,
41
.I --all-useful-reports
39
.IR "-D 10 -d 20 -P 60 -r 20 -s -t 20" .
43
domain report: switches the top level and the second level report on. The
44
data is derived from the URL. The output is limited by n. (-1 is unlimited)
48
peak report: Measures the peaks of the Proxy usage in requests per second,
49
per minute and per hour. It reports peaks for TCP, UDP and ALL requests. If
43
.I --size-distribution-report 10
45
.I --performance-report 60
46
.I --requester-report 20
49
.I --response-time-report
50
.I --errorcode-distribution-report
54
.I --domain-report|-d n ($domain_report)
55
switches the top level and the second level report on. The data is derived
56
from the URL. The output is limited by n. (-1 is unlimited)
59
.I --domain-report-limit n ($domain_report_limit)
60
limit the domain-report to lines which have n or more requests.
63
.I --domain-report-n-level|-N n ($domain_report_n_level)
64
All URL-Host reports will be switched from 2nd-level to n-level-reports. (-1
65
shows a full urlhost-report)
68
This option is only useful with activated domain-report.
71
.I --errorcode-distribution-report ($errorcode_distribution_report)
72
shows the Response code distribution over all objects
75
.I ($object_freshness_report)
76
shows the freshness of objects in your cache.
78
looks for freshness tags like 'TCP_HIT', 'TCP_REFRESH_MISS', ... and make
79
statistics on it. With this information you can optimize the caching behaviour
80
of your cache depending on the objects content type. E.g. squid admins could
81
use this information to configure the refresh_pattern. This option needs more
82
configuration in the configuration-file.
85
.I --peak-report|-p type ($peak_report)
86
Measures the peaks of the Proxy usage in requests per second, per minute and
87
per hour. It reports peaks for TCP, UDP and ALL requests. If set to
52
89
these values were calculated with the old slow method, if set to
54
91
the new faster (but still slow) method is used.
58
performance report: Shows the throughput of TCP requests for every n minutes.
62
requester report: Switches the UDP and TCP requester reports on. The output
63
is limited by n. (-1 is unlimited)
67
Requested report: adds to each line of the requester report the requested
68
URLs. The output is limited by n. (-1 is unlimited, and can result in very
94
.I --performance-report|-P n ($performance_report)
95
Shows the throughput of TCP requests for every n minutes.
98
.I --performance-report-adjust|-T n ($performance_report_adjust)
99
Time: Adjust the Performancereport in minutes for non GMT-Timezoners.
102
.I --requester-report|-r n ($requester_report)
103
Switches the UDP and TCP requester reports on. The output is limited by n.
107
.I --requester-report-no-dns-lookup|-n ($requester_report_no_dns_lookup)
108
Switches the IP lookup for the requesters off.
111
.I --requester-report-use-user-info|-u ($requester_report_use_user_info)
112
Switches the usage of eventually available ident information for requester
116
This breaks the privacy of your users! (see PRIVACY-Section below)
119
.I --requester-report-with-targets|-R n ($requester_report_with_targets)
120
adds to each line of the requester report the requested URLs. The output is
121
limited by n. (-1 is unlimited, and can result in very very long reports.)
72
124
Using this option breaks the privacy of your users! (see PRIVACY-Section
77
status report: alters the default behaviour of
128
.I --response-time-report ($response_time_report)
129
sums up the time distribution over all objects
132
.I (@response_time_report_interval)
133
This array defines the time steps, which should be reported in the
134
response-time-report.
137
.I --size-distribution-report|-D n ($size_distribution_report)
138
shows size-based distribution of requested objects, smaller numbers result in
139
more verbose reports. (choose 2, 10 or 100 for useful output.)
142
.I --status-report|-s ($status_report)
143
alters the default behaviour of
79
145
and makes the status reports more verbose.
83
type report: switches the content type and the file extension report on. The
84
output is limited by n. (-1 is unlimited)
88
Distribution Histogram: shows size-based distribution of requested objects,
89
smaller numbers result in more verbose reports. (choose 2, 10 or 100 for
92
Allocation report: shows how different sized objects perform on your Proxy.
97
format: sets the type of input logfiles. If set to
148
.I --type-report|-t n ($type_report)
149
switches the content type and the file extension report on. The output is
150
limited by n. (-1 is unlimited)
153
.I --type-report-ignore-case|-c ($type_report_ignore_case)
154
Switch to case-insensitive. This is useful for the 'Requested extensions'
159
.I --input-format|-f type ($input_format)
160
sets the type of input logfiles. If set to
101
164
tries to guess the input file format. This is the Default.
104
If the first line of your input file is corrupted, Calamaris will stop with
167
If the first line of your input file is corrupted,
169
will stop with an error.
109
173
expects native logfile derived from
110
.B Squid V1.1.beta26-V2.x
114
178
.I 'squid-extended'
116
180
expects native logfile derived from
117
.B Squid V1.1.alpha1-V2.x
182
V1.1.alpha1-V2.x with
123
.B Smartfilter-Patch.
124
188
or squid-style logfiles out of
125
189
.B Cisco Content Engines.
126
190
(This only enables parsing of these kind of logfile, the additional
151
216
.B Compaq Tasksmart,
152
217
.B Novell Internet Caching System
159
224
expects Logfiles in Netscape Extended-1 or Netscape Extended-2
160
225
Logfile Format (from
161
.B Netscape/ipoint Proxy-Server
226
.B Netscape/iPlanet/SunOne Proxy-Server
230
.I --ipfilter-exclude IP/range ($ipfilter_exclude)
231
all IPs are analyzed, except IP/range.
232
Format: 1.1.1.1/32:1.1.2.0/24
233
1.1.1.1/255.255.255.255:1.1.2.0/255.255.255.0
235
IP list separated by ':'
236
This switch needs the perl Module NetAddr::IP.
239
This breaks the privacy of your users! (see PRIVACY-Section below)
242
.I --ipfilter-include IP/range ($ipfilter_include)
243
no IPs are analyzed, except IP/range. Format: see --ipfilter-exclude
246
This breaks the privacy of your users! (see PRIVACY-Section below)
249
.I --no-input|-z ($no_input)
250
Switches reading from standard input off. You can use this to merge
251
many cache files to one (see
252
.I --cache-input-file
254
.I --cache-output-file)
255
or to generate a report out of cache files.
258
.I --time-interval|-I t-t ($time_interval)
259
defines which time-interval should be parsed. t has to be the
260
format yyyymmddhhmmss (localtime)
263
omitting the beginning or ending date is allowed.
165
266
Standard output format is plain ascii with 80 chars width.
168
.I -F type[,type[,type[,...]]]
271
defines the colors for the columns in graphics. (only useful with
277
Through the config-file you are able to modify the width of the report and
278
alter the culomns that are displayed in the reports. n is the number of the
279
report, as displayed by --help in the --show-reports-option.
282
.I --hostname|-H name ($hostname)
283
The name for the title or subject of the output. If set to
286
looks up the host name of the system its been run on.
289
.I --image-type ($image_type)
290
Sets the image type to gif, png, jpeg, gd or gd2. Only useful when
292
--output-format graph
293
is set. The available images types are dependend
294
on your GD::Graph installation. Default is 'png'.
297
.I --logo|-l string ($logo)
298
add a custom string to a HTML-Report. It'll be added to a table on the
300
.I -l '<A HREF="http://cord.de/"><IMG BORDER=0
301
SRC="http://cord.de/Images/cord/cordlog2n.gif" ALT="Cord"></A>'
302
will add my logo with a link to the Report.
306
works only in combination with
307
.I --output-format html or html-frame
310
.I --meta|-M string ($meta)
311
Meta: adds a custom string or the content of a file into the <HEAD> of a
312
HTML-Report. Useful if you want to add Stylesheets or something to the Report.
316
works only in combination with
317
.I --output-format html or html-frame
320
.I --output-format|-F type[,type[,type[,...]]] ($output_format)
169
321
Format: sets the format of the output-report. If set to
177
329
to send html mails.
332
all output is given in html frames with tables.
180
335
all output is given in html with tables without HTML-Headers. Useful for
181
336
Server-Side-Includes.
339
enables graphics for html, html-embed or html-frame.
184
342
gives out the raw numbers seperated by spaces. Useful for re-using the output
185
in other scripts. If you use this along with
343
in other scripts. If you use this along with
187
345
the byte values are calculated in the given Unit, and displayed without
188
346
indication along with the numbers. the indication moves up to the header of
193
Meta: adds a custom string into the <HEAD> of a HTML-Report. Useful if you
194
want to add Stylesheets or something to the Report.
198
works only in combination with
204
logo: add a custom string to a HTML-Report. It'll be added to a table on the
206
.I -l '<A HREF="http://cord.de/"><IMG BORDER=0 SRC="http://cord.de/Images/cord/cordlog2n.gif" ALT="Cord"></A>'
207
will add my logo with a link to the Report.
211
works only in combination with
217
Unit: You can define this to K(ilo), M(ega), G(iga) or T(era) for
221
.I -S n[,n[,n[,...]]]
350
.I --output-path ($output_path)
351
output calamaris statistics to /path. In case of graph output, the graphics
352
destination is /path and the filename is index.html, else it is calamaris.txt.
355
is not given, all graphics are written to the working directory.
358
.I --output-file ($output_file)
359
alter the filename of --output-path.
362
.I --output-file-prefix ($output_file_prefix)
367
is replaced by the timerange of the report,
374
.I --show-reports|-S n[,n[,n[,...]]] ($show_reports)
222
375
Show: Shows only the defined reports in the specified order. Default is to
223
376
display the reports as they are defined through the report-switches above. The
224
377
following numbers are defined:
269
422
14 Incoming TCP-requests by host
272
15 Distribution Histogram
275
16 Performance in minute steps
425
15 Size Distribution Diagram
428
16 Performance in n minute steps
431
17 UDP-Request duration distribution in msec
434
18 TCP-Request duration distribution in msec
437
19 UDP Response code distribution
440
20 TCP Response code distribution
278
444
Using this doesn't make
280
any faster, the internal calculations will
281
be done as the report-switches were set (see above).
446
any faster, the internal calculations will be done as the report-switches
447
were set (see above).
450
.I --sort-order|-O ($sort_order)
451
Changes the sort order in the reports to request size, default is
452
sorting by number of requests.
456
defines the colors for text/axis/legend/labels in graphics. (only useful with
457
.I --output-format graph
461
.I --unit|-U string ($unit)
462
You can define this to K(ilo), M(ega), G(iga) or T(era) for the Byte-units.
466
defines the width of the graphics. height is calculated from this with a
467
3:2-ratio. (only useful with
468
.I --output-format graph
473
defines how many datasets should be drawn on the graph. 30 is a good value,
474
but you can play with this. if $x_scale gets to big, you're on your own ;-)
477
.I --generate-index ($generate_index)
478
generates an index for all reports that match
479
.I --output-file-prefix.
287
input file: You can reuse a cache file generated with
484
.I --cache-input-file|-i file ($cache_input_file)
485
You can reuse a cache file generated with
486
.I --cache-output-file file
289
487
to add old data to a new report. Several files can be seperated with a ':'.
319
Host name: The name for the title or subject of the output. If set to
322
looks up the host name of the system its been run on.
326
no lookup: Switches the IP lookup for the requesters off.
330
Order: Changes the sort order in the reports to request size, default is
331
sorting by number of requests.
335
Interval: defines which time-interval should be parsed. t has to be the
336
format yyyymmddhhmmss
339
omitting the beginning or ending date is allowed.
343
N-level: All URL-Host reports will be switched from 2nd-level to
344
N-level-reports. (-1 shows a full urlhost-report)
347
This option is only useful with activated domain-report.
351
Time: Adjust the Performancereport in minutes for non GMT-Timezoners.
355
case: Switch to case-insensitive. This is useful for
356
the 'Requested extensions' report.
360
user: Switches the usage of eventually available ident information for
361
requester reports on.
364
This breaks the privacy of your users! (see PRIVACY-Section below)
368
zero: Switches reading from standard input off. You can use this to merge
369
many cache files to one (see
373
or to generate a report out of cache files.
517
.I --benchmark|-b n ($benchmark)
377
518
benchmark: A switch for the impatient as it prints a '#' for every n parsed
382
verbose: print more information about what is Calamaris is doing
387
Loop: prints the internal loop to STDERR. (for Debugging)
391
Copyright: Prints the copyright information of
396
help: Prints a brief description of the command line options.
400
Version: Prints out the Version-Number.
523
Prints the copyright information of
529
Prints a brief description of the command line options.
533
Prints out the Version-Number.
539
prints the internal loop to STDERR. (for Debugging)
543
activates some small tests for the programmer.
546
.I --verbose|-v ($verbose)
547
print more information about what is Calamaris is doing and believing.
405
551
This example mails the daily statistics to root:
406
552
cat /var/log/squid/access.log | nice -39
408
-aH 'daily worf' -F mail | mail root
554
.I --all-useful-reports
411
561
This one only caches a summary for later use:
412
562
cat /var/log/squid/access.log |
414
-a -o daily.`date +"%w"` > /dev/null
564
--all-useful-reports --cache-output-file daily.`date +"%w"` > /dev/null
417
567
You can then use the caches to have weekly statistics:
419
569
if [ $DAYOFWEEK = "0" ]; then
421
-a -i daily.1:daily.2:daily.3:daily.4:daily.5:daily.6:daily.0 -z -F mail -H "weekly worf" | mail root ;
571
--all-useful-reports --cache-input-file daily.1:daily.2:daily.3:daily.4:daily.5:daily.6:daily.0 --no-input --output-format mail --hostname "weekly worf" | mail root ;
425
575
If you have a problem with
427
please make sure that you use the recent version of
577
, please make sure that you use the recent version of
429
579
(see VERSION below). Also check that your proxy works correctly and doesn't
430
580
produce invalid Logfiles. (see the README for buglist and pointers.)
432
If you're sure that you've encountered a bug in Calamaris, please report it to
433
Calamaris-bug@cord.de. This also applies if Calamaris itself says 'please
582
If you're sure that you've encountered a bug in
584
please report it to Calamaris-bug@cord.de. This also applies if
586
itself says 'please report this'.
439
590
can be (mis-)used to track what users are requesting.
459
If you still want to use Calamaris that way, let your vict^Wusers
460
know, that they'll be monitored. (in Germany you have to let them
610
If you still want to use
612
that way, let your vict^Wusers know, that they'll be monitored. (in Germany you
613
have to let them know!)
467
Cord Beermann <Cord@Wunder-Nett.org>. There are also a lot of people who
468
contributed code, gave ideas or requested features. Look them up in the
619
Cord Beermann <Cord@Wunder-Nett.org>,
620
Michael Pophal <michael.pophal@nefkom.net>.
621
There are also a lot of people who contributed code, gave ideas or requested
622
features. Look them up in the executable.
471
624
This man page was written by Philipp Frauenfelder <pfrauenf@debian.org>,
472
625
maintainer of the Debian package. Maintenance is now taken over by
477
Version of this manpage: $Id: calamaris.1,v 1.28 2003/05/14 21:22:37 cord Exp $
479
It describes the usage of Calamaris V2.58 and later.
629
Version of this manpage: $Id: calamaris.1,v 3.1 2006-03-19 17:52:48 cord Exp $
631
It describes the usage of Calamaris V3.0 and later.
481
633
Information about new releases, mailing lists, and other related issues
482
634
can be found from the
484
home page at http://Calamaris.Cord.de/
637
.URL http://Calamaris.Cord.de/
488
641
comes with "absolutely no warranty".
491
Copyright (C) 1997, 1998, 1999, 2000, 2001, 2002, 2003 Cord Beermann
644
Copyright (C) 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004 Cord Beermann
493
646
This program is free software; you can redistribute it and/or modify it under
494
647
the terms of the GNU General Public License as published by the Free Software