~ubuntu-branches/ubuntu/intrepid/calamaris/intrepid

.TH "CALAMARIS" "1" "$Date: 2006-03-19 17:52:48 $" "Handmade" "Calamaris Manual"

.SH "NAME"
.B calamaris
- generate text and graphical statistics out of log files from
Proxy-Cache-Servers

.SH "SYNOPSIS"
cat log |
.B calamaris
[ --config-file .../calamaris.conf ] [ switches ]

.SH "DESCRIPTION"
.B Calamaris
is used to produce statistical output from \fBSquid\fR, \fBNetCache\fR,
\fBInktomi Traffic Server\fR, \fBOops! proxy server\fR,
\fBCompaq Tasksmart\fR, \fBCisco Content Engines\fR or related Proxy log
files.  The resulting output can be ascii or html with or without graphic
and with or without frames. It is possible to cache calculated data in a
file to use them in later runs.

This manual page describes the options of \fBCalamaris\fR and gives a few
examples.

.SH "OPTIONS"
.SS Configuration File

.TP
.I --config-file file

Not all reports and modification can be made through command-line-switches.
To use all options of Calamaris you\'ll have to use the configuration file.
You'll find the configuration-directives below, always inside of braces.
Examples are in the calamaris.conf which should come with this package.

.SS Reports

.TP
.I --all-useful-reports|-a
extracts all useful reports available,
.I --all-useful-reports
equals
.I --size-distribution-report 10
.I --domain-report 20
.I --performance-report 60
.I --requester-report 20
.I --status-report
.I --type-report 20
.I --response-time-report
.I --errorcode-distribution-report
.

.TP
.I --domain-report|-d n ($domain_report)
switches the top level and the second level report on.  The data is derived
from the URL.  The output is limited by n.  (-1 is unlimited)

.TP
.I --domain-report-limit n ($domain_report_limit)
limit the domain-report to lines which have n or more requests.

.TP
.I --domain-report-n-level|-N n ($domain_report_n_level)
All URL-Host reports will be switched from 2nd-level to n-level-reports. (-1
shows a full urlhost-report)

.B Note:
This option is only useful with activated domain-report.

.TP
.I --errorcode-distribution-report ($errorcode_distribution_report)
shows the Response code distribution over all objects

.TP
.I ($object_freshness_report)
shows the freshness of objects in your cache.
.B Calamaris
looks for freshness tags like 'TCP_HIT', 'TCP_REFRESH_MISS', ...  and make
statistics on it. With this information you can optimize the caching behaviour
of your cache depending on the objects content type.  E.g. squid admins could
use this information to configure the refresh_pattern. This option needs more
configuration in the configuration-file.

.TP
.I --peak-report|-p type ($peak_report)
Measures the peaks of the Proxy usage in requests per second, per minute and
per hour.  It reports peaks for TCP, UDP and ALL requests. If set to
.I 'old'
these values were calculated with the old slow method, if set to
.I 'new'
the new faster (but still slow) method is used.

.TP
.I --performance-report|-P n ($performance_report)
Shows the throughput of TCP requests for every n minutes.

.TP
.I --performance-report-adjust|-T n ($performance_report_adjust)
Time: Adjust the Performancereport in minutes for non GMT-Timezoners.

.TP
.I --requester-report|-r n ($requester_report)
Switches the UDP and TCP requester reports on.  The output is limited by n.
(-1 is unlimited)

.TP
.I --requester-report-no-dns-lookup|-n ($requester_report_no_dns_lookup)
Switches the IP lookup for the requesters off.

.TP
.I --requester-report-use-user-info|-u ($requester_report_use_user_info)
Switches the usage of eventually available ident information for requester
reports on.

.B Warning:
This breaks the privacy of your users! (see PRIVACY-Section below)

.TP
.I --requester-report-with-targets|-R n ($requester_report_with_targets)
adds to each line of the requester report the requested URLs. The output is
limited by n.  (-1 is unlimited, and can result in very very long reports.)

.B Warning:
Using this option breaks the privacy of your users! (see PRIVACY-Section
below)

.TP
.I --response-time-report ($response_time_report)
sums up the time distribution over all objects

.TP
.I (@response_time_report_interval)
This array defines the time steps, which should be reported in the
response-time-report.

.TP
.I --size-distribution-report|-D n ($size_distribution_report)
shows size-based distribution of requested objects, smaller numbers result in
more verbose reports. (choose 2, 10 or 100 for useful output.)

.TP
.I --status-report|-s ($status_report)
alters the default behaviour of
.B Calamaris
and makes the status reports more verbose.

.TP
.I --type-report|-t n ($type_report)
switches the content type and the file extension report on.  The output is
limited by n.  (-1 is unlimited)

.TP
.I --type-report-ignore-case|-c ($type_report_ignore_case)
Switch to case-insensitive. This is useful for the 'Requested extensions'
report.

.SS Input
.TP
.I --input-format|-f type ($input_format)
sets the type of input logfiles. If set to

.I 'auto'
.B Calamaris
tries to guess the input file format.  This is the Default.

.B Note:
If the first line of your input file is corrupted,
.B Calamaris
will stop with an error.

.I 'squid'
.B Calamaris
expects native logfile derived from
.B Squid
V1.1.beta26-V2.x or
.B OOPS.

.I 'squid-extended'
.B Calamaris
expects native logfile derived from
.B Squid
V1.1.alpha1-V2.x with
.I log_mime_hdrs
enabled or
.B Squid
with
.B Smartfilter-Patch
or squid-style logfiles out of
.B Cisco Content Engines.
(This only enables parsing of these kind of logfile, the additional
data will be ignored.)

(Logging of MIME-Headers breaks the privacy of your users! (see
PRIVACY-Section below)

.I 'squid-old'
.B Calamaris
expects native logfile derived from
.B Squid
V1.1.alpha1-V1.1.beta25.

.I 'nc'
.B Calamaris
expects Logfiles from
.B NetCache
up to V4.x. (Please see the README on this.)

.I 'its'
.B Calamaris
expects Logfiles from
.B Inktomi Traffic Server.

.I 'elff'
.B Calamaris
expects Logfiles in Extended Logfile Format (i.e. from
.B Compaq Tasksmart,
.B Novell Internet Caching System
or
.B NetCache
V5.x)

.I 'nse'
.B Calamaris
expects Logfiles in Netscape Extended-1 or Netscape Extended-2
Logfile Format (from
.B Netscape/iPlanet/SunOne Proxy-Server
)

.TP
.I --ipfilter-exclude IP/range ($ipfilter_exclude)
all IPs are analyzed, except IP/range.
Format: 1.1.1.1/32:1.1.2.0/24
1.1.1.1/255.255.255.255:1.1.2.0/255.255.255.0

IP list separated by ':'
This switch needs the perl Module NetAddr::IP.

.B Warning:
This breaks the privacy of your users! (see PRIVACY-Section below)

.TP
.I --ipfilter-include IP/range ($ipfilter_include)
no IPs are analyzed, except IP/range. Format: see --ipfilter-exclude

.B Warning:
This breaks the privacy of your users! (see PRIVACY-Section below)

.TP
.I --no-input|-z ($no_input)
Switches reading from standard input off.  You can use this to merge
many cache files to one (see
.I --cache-input-file
and
.I --cache-output-file)
or to generate a report out of cache files.

.TP
.I --time-interval|-I t-t ($time_interval)
defines which time-interval should be parsed.  t has to be the
format yyyymmddhhmmss (localtime)

.B Note:
omitting the beginning or ending date is allowed.

.SS Output
Standard output format is plain ascii with 80 chars width.

.TP
.I ($column1_color)
.I ($column2_color)
defines the colors for the columns in graphics. (only useful with
.I --output-format
graph)

.TP
.I ($formats[n])
Through the config-file you are able to modify the width of the report and
alter the culomns that are displayed in the reports. n is the number of the
report, as displayed by --help in the --show-reports-option.

.TP
.I --hostname|-H name ($hostname)
The name for the title or subject of the output.  If set to
.I 'lookup'
.B Calamaris
looks up the host name of the system its been run on.

.TP
.I --image-type ($image_type)
Sets the image type to gif, png, jpeg, gd or gd2. Only useful when
.I
--output-format graph
is set. The available images types are dependend
on your GD::Graph installation. Default is 'png'.

.TP
.I --logo|-l string ($logo)
add a custom string to a HTML-Report. It'll be added to a table on the
top of the output.
.I -l '<A HREF="http://cord.de/"><IMG BORDER=0
SRC="http://cord.de/Images/cord/cordlog2n.gif" ALT="Cord"></A>'
will add my logo with a link to the Report.

.B Note:
.I --logo
works only in combination with
.I --output-format html or html-frame

.TP
.I --meta|-M string ($meta)
Meta: adds a custom string or the content of a file into the <HEAD> of a
HTML-Report. Useful if you want to add Stylesheets or something to the Report.

.B Note:
.I --meta
works only in combination with
.I --output-format html or html-frame

.TP
.I --output-format|-F type[,type[,type[,...]]] ($output_format)
Format: sets the format of the output-report. If set to

.I 'mail'
adds a subject header to the beginning of the report.

.I 'html'
all output is given in html with tables.  Can be combined with
.I 'mail'
to send html mails.

.I 'html-frame'
all output is given in html frames with tables.

.I 'html-embed'
all output is given in html with tables without HTML-Headers. Useful for
Server-Side-Includes.

.I 'graph'
enables graphics for html, html-embed or html-frame.

.I 'unformatted'
gives out the raw numbers seperated by spaces. Useful for re-using the output
in other scripts. If you use this along with
.I -U,
the byte values are calculated in the given Unit, and displayed without
indication along with the numbers. the indication moves up to the header of
the report.

.TP
.I --output-path ($output_path)
output calamaris statistics to /path. In case of graph output, the graphics
destination is /path and the filename is index.html, else it is calamaris.txt.
If
.I --output-path
is not given, all graphics are written to the working directory.

.TP
.I --output-file ($output_file)
alter the filename of --output-path.

.TP
.I --output-file-prefix ($output_file_prefix)
adds a prefix to 
.I --output-file
.
.I %t
is replaced by the timerange of the report, 
.I %h
by the hostname (see
.I --hostname
)

.TP
.I --show-reports|-S n[,n[,n[,...]]] ($show_reports)
Show: Shows only the defined reports in the specified order. Default is to
display the reports as they are defined through the report-switches above. The
following numbers are defined:

.RS
0  Summary
.RE
.RS
1  Incoming request peak per protocol
.RE
.RS
2  Incoming transfer volume peak per protocol
.RE
.RS
3  Incoming requests by method
.RE
.RS
4  Incoming UDP-requests by status
.RE
.RS
5  Incoming TCP-requests by status
.RE
.RS
6  Outgoing requests by status
.RE
.RS
7  Outgoing requests by destination
.RE
.RS
8  Request-destinations by 2ndlevel-domain
.RE
.RS
9  Request-destinations by toplevel-domain
.RE
.RS
10 TCP-Request-protocol
.RE
.RS
11 Requested content-type
.RE
.RS
12 Requested extensions
.RE
.RS
13 Incoming UDP-requests by host
.RE
.RS
14 Incoming TCP-requests by host
.RE
.RS
15 Size Distribution Diagram
.RE
.RS
16 Performance in n minute steps
.RE
.RS
17 UDP-Request duration distribution in msec
.RE
.RS
18 TCP-Request duration distribution in msec
.RE
.RS
19 UDP Response code distribution
.RE
.RS
20 TCP Response code distribution
.RE

.B Note:
Using this doesn't make
.B Calamaris
any faster, the internal calculations will be done as the report-switches
were set (see above).

.TP
.I --sort-order|-O ($sort_order)
Changes the sort order in the reports to request size, default is
sorting by number of requests.

.TP
.I ($text_color)
defines the colors for text/axis/legend/labels in graphics. (only useful with
.I --output-format graph
)

.TP
.I --unit|-U string ($unit)
You can define this to K(ilo), M(ega), G(iga) or T(era) for the Byte-units.

.TP
.I ($width)
defines the width of the graphics. height is calculated from this with a
3:2-ratio. (only useful with
.I --output-format graph
)

.TP
.I ($x_scale)
defines how many datasets should be drawn on the graph. 30 is a good value,
but you can play with this. if $x_scale gets to big, you're on your own ;-)

.TP
.I --generate-index ($generate_index)
generates an index for all reports that match 
.I --output-file-prefix.

.SS Caching

.TP
.I --cache-input-file|-i file ($cache_input_file)
You can reuse a cache file generated with
.I --cache-output-file file
to add old data to a new report.  Several files can be seperated with a ':'.

.B Note:
if you use more than one cache file, make sure they are chronologicaly
ordered (oldest first).

.B Note:
if you reuse cache-files, which were not created with
.I -d -1 -r -1 -t -1 -R -1
the number of 'others' would be wrong everywhere. In this 
case the number of 'others' are omitted.

.TP
.I --cache-output-file|-o file ($cache_output_file)
.B Calamaris
stores a summary of the computed information in
.I file
and you can reuse it at a later time with
.I --cache-input-file
.

.B Note:
The output file can be the same as the input file: it is simply overwritten
after reading the data.  It is not recommended to change the options between
different runs if you include older data as this can result in strange
measurements.

.SS Misc

.TP
.I --benchmark|-b n ($benchmark)
benchmark: A switch for the impatient as it prints a '#' for every n parsed
lines.

.TP
.I --copyright|-C
Prints the copyright information of
.B Calamaris
.

.TP
.I --help|-h
Prints a brief description of the command line options.

.TP
.I --version|-V
Prints out the Version-Number.

.SS Debug

.TP
.I --dump-loop|-L
prints the internal loop to STDERR. (for Debugging)

.TP
.I ($test)
activates some small tests for the programmer.

.TP
.I --verbose|-v ($verbose)
print more information about what is Calamaris is doing and believing.

.SH "EXAMPLES"
.TP
This example mails the daily statistics to root:
cat /var/log/squid/access.log | nice -39
.B calamaris
.I --all-useful-reports
.I --hostname
'daily worf'
.I --output-format
mail | mail root

.TP
This one only caches a summary for later use:
cat /var/log/squid/access.log |
.B calamaris
--all-useful-reports --cache-output-file daily.`date +"%w"` > /dev/null

.TP
You can then use the caches to have weekly statistics:

if [ $DAYOFWEEK = "0" ]; then
.B calamaris
--all-useful-reports --cache-input-file daily.1:daily.2:daily.3:daily.4:daily.5:daily.6:daily.0 --no-input --output-format mail --hostname "weekly worf" | mail root ;
fi

.SH "BUGS"
If you have a problem with
.B Calamaris
, please make sure that you use the recent version of
.B Calamaris
(see VERSION below). Also check that your proxy works correctly and doesn't
produce invalid Logfiles. (see the README for buglist and pointers.)

If you're sure that you've encountered a bug in
.B Calamaris
please report it to Calamaris-bug@cord.de. This also applies if
.B Calamaris
itself says 'please report this'.

.SH "PRIVACY"
.B Calamaris
can be (mis-)used to track what users are requesting.

So please read the following and think about it, before using
.B Calamaris
to be the
.I Big Brother.

.TP
-
If you don't trust your users than there is something more wrong
than the loss of productivity.

.TP
-
Squid has some nice acl-mechanisms. If you think that your users
don't use the net properly, don't let them use it. (You can also
open the net at specific times or to specific sites, if you want.)

.TP
-
If you still want to use
.B Calamaris
that way, let your vict^Wusers know, that they'll be monitored. (in Germany you
have to let them know!)

.SH "SEE ALSO"
.B squid(8)

.SH "AUTHOR"
Cord Beermann <Cord@Wunder-Nett.org>,
Michael Pophal <michael.pophal@nefkom.net>.
There are also a lot of people who contributed code, gave ideas or requested
features. Look them up in the executable.

This man page was written by Philipp Frauenfelder <pfrauenf@debian.org>,
maintainer of the Debian package.  Maintenance is now taken over by
Cord Beermann.

.SH "VERSION"
Version of this manpage: $Id: calamaris.1,v 3.1 2006-03-19 17:52:48 cord Exp $

It describes the usage of Calamaris V3.0 and later.

Information about new releases, mailing lists, and other related issues
can be found from the
.B Calamaris
home page at 
.URL http://Calamaris.Cord.de/

.SH "WARRANTY"
.B Calamaris
comes with "absolutely no warranty".

.SH "COPYRIGHT"
Copyright (C) 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004 Cord Beermann

This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation; either version 2 of the License, or (at your option) any later
version.

(If you modify and want to publish it under the name
.B Calamaris
, please ask me.  I don't want to confuse the 'audience' with many different
versions of the same name and/or Version number.  (This is not part of the
license, it is only a favour i asked of you.))

This program is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more
details.

You should have received a copy of the GNU General Public License along with
this program; if not, write to the Free Software Foundation, Inc., 59 Temple
Place - Suite 330, Boston, MA 02111-1307, USA.