~ubuntu-branches/ubuntu/wily/autopkgtest/wily

.TH adt\-run 1 2014 autopkgtest
.SH NAME
adt\-run \- test an installed binary package using the source package's tests
.SH SYNOPSYS
.B adt\-run
.IR options ...
.B \-\-\-
.I virt\-server
.RI [ virt\-server\-arg ...]
.br
.SH DESCRIPTION
.B adt\-run
is the program for invoking the autopkgtest package testing machinery.

autopkgtest is a facility for testing binary Debian or Click packages, as
installed on a system (such as a testbed system).  The tests are those supplied
in the source package.

adt\-run runs each test supplied by a particular package and reports
the results.  It drives the specified virtualisation regime as
appropriate, and parses the test description metadata, and arranges
for data to be copied to and from the testbed as required.

See /usr/share/doc/autopkgtest/README.running\-tests.rst.gz for an
introduction about how to use adt\-run.

.SH SPECIFYING TESTS

Actions specify the source and binary packages to test, or change
what happens with package arguments:

.TP
.BR --source " " \fIdsc\fR
Run tests from Debian source package \fIdsc\fR. By default the package will
also be built and the resulting binaries will be used to satisfy test
dependencies; to disable that, specify the
.BR -B / --no-built-binaries
option before.

The ordering is significant: each \fB--source\fR option should precede
options whose dependencies are to be satisfied by the binaries it
produces.

.TP
.BR --unbuilt-tree " " \fIdirectory\fR
Specifies that tests from the unbuilt Debian source tree
.IR directory
should be run.  This is very similar to specifying \fB\-\-source\fR
except that a directory tree (which should be pristine) is supplied,
instead of a source package.

.TP
.BR --built-tree " " \fIdirectory\fR
Specifies that tests from the built Debian source tree
.IR directory
should be run. Note that all test dependencies are then satisfied by
archive packages, unless you explicitly specify locally built .debs with
.BR --binary .

.TP
.BR --apt-source " " \fIsrcpkgname\fR
Downloads \fIsrcpkgname\fR with \fBapt\-get source\fR in the testbed and
run its tests. This is similar to specifying
.B \-\-source
but avoids copying the source from the host to the testbed. Possibly built
binaries will
.B not
be used to satisfy dependencies, as usually in this mode you want to test
binaries from a real archive.

.TP
.BR --binary " " \fIdeb\fR
Specifies that \fIdeb\fR should be used for tests of all following
source packages.  By default it will be used to satisfy dependencies,
both during building and testing.

The ordering is significant, as for \fB--source\fR. In particular, if a
subsequent source package will build a binary of the same name, that will be
used from then on, and \fIdeb\fR will be ignored.

.TP
.BR --changes " " \fIchanges\fR
Specifies that the debs in the given .changes should be used for tests of the
source package in that .changes. Acts as if you had specified the .debs and .dsc
from a .changes file as explicit arguments.

.TP
.BR --click-source " " \fIclicksrc
Path to click source tree for subsequent
.B --click
package.

.TP
.BR --click " " \fIclickpkg
If
.I clickpkg
is a file (*.click), install given click package into testbed. If it is a click
name (like "com.example.myapp"), assume it is already installed in the testbed
and read the manifest from it.

Run click package tests from the preceeding
.BR --click-source .
If a click source directory is not specified explicitly, it will be downloaded
according to the manifest's
.B x-source
entry. Currently the only supported schema is
.BR vcs-bzr .



.TP
.I filename
Bare filename arguments are processed as if
.BR --built-tree ", " --source ", " --unbuilt-tree ", " --apt-source ", "
.BR --binary ", " --changes ", " --click-source ", or " --click
was specified; the nature of the argument is guessed from the form of
the filename.  In the case of \fB--built-tree\fR, either the
option must be specified, or the filename must end in a slash; two
slashes at the end are taken to mean \fB--unbuilt-tree\fR. If a given directory
has a "click" subdirectory, it is interpreted as
.BR --click-source .

.SH TEST OPTIONS
Unless stated otherwise, these affect all subsequent test arguments.

.TP
.BR -B " | " --no-built-binaries
All built binaries from subsequent
.B --source
or
.B --unbuilt-tree
tests will not be built or ignored,
and dependencies are satisfied with packages from the archive. Note
that packages still get built if a test requires
\fBbuild-needed\fR.

.TP
.B --built-binaries
Subsequent
.B --source
or
.B --unbuilt-tree
tests will get built and their dependencies be satisfied with the built
binaries. This is the default behaviour, so you only need this to revert a
previously specified
.B --no-built-binaries
option.

.TP
.BI --override-control= PATH
Read the test metadata from
.I PATH
instead of
.B debian/tests/control
(for Debian sources)
or the Click manifest for the following test.

.TP
.BI --testname= TEST
Run only the given test name (from test control file) in the following test.

.SH LOGGING OPTIONS

.TP
.BI -o " dir" " | --output-dir=" dir
Specifies that test artifacts (stderr and stdout from the tests, the log file,
built binary packages etc.) should be placed in the given directory.
\fIdir\fR must not exist yet or be empty, otherwise
.B adt-run
will refuse to use it.

.TP
.BI -l " logfile" " | --log-file=" logfile
Specifies that the trace log should be written to \fIlogfile\fR
instead of to \fIoutput-dir\fR.

.TP
.BI --summary= summary
Specifies that a summary of the outcome should be written to
\fIsummary\fR.  The events in the summary are written to the log
in any case.

.TP
.BR -q " | " --quiet
Do not send a copy of \fBadt-run\fR's trace logstream to stderr.  This
option does not affect the copy sent to \fIlogfile\fR or
\fIoutput-dir\fR.  Note that without the trace
logstream it can be very hard to diagnose problems.

.SH TEST BED SETUP OPTIONS

.TP
.BI \-\-setup\-commands= commands
Run
.I commands
after opening the testbed. This can be used e. g. to enable additional apt
sources, run
.B apt-get update
or similar.
If
.I commands
is an existing file name, the commands are read from that; otherwise it
is a string with the actual commands that gets run as-is. File names
without directory will be searched in both the current directory and in
.B /usr/share/autopkgtest/setup-commands/
so you do not need to give the full path for setup scripts shipped with
autopkgtest.

This option can be specified multiple times.

If
.B \-\-user
is given or the test bed provides a
.B suggested-normal-user
capability, the
.B $ADT_NORMAL_USER
environment variable will be set to that user.

If the setup commands affect anything in boot directories (like /boot or
/lib/systemd/system) and the testbed supports rebooting, the testbed will be
rebooted after the setup commands. This can be suppressed by creating a file
.BR /run/autopkgtest_no_reboot.stamp .

.TP
.BR --apt-upgrade " | " -U
Run
.B apt\-get update
and
.B apt\-get dist-upgrade -y
in the testbed before running the tests.

.TP
.BI \-\-apt\-pocket= pocket
Add apt sources for \fIrelease\fR-\fIpocket\fR. This finds the first
.B deb
line in
.B /etc/apt/sources.list
which does not already specify a pocket and adds a deb and deb-src line with
that pocket to
.B /etc/apt/sources.list.d/\fIpocket\fB.list\fR.
Note that this does not imply calling
.B apt-get update\fR.

.TP
.BI \-\-copy= HOSTPATH:TESTBEDPATH
Copy file or directory from host into testbed after opening. This happens
before
.B \-\-setup-commands
thus you can use these files in the setup commands.


.SH USER/PRIVILEGE HANDLING OPTIONS

.TP
.BI -u " user" " | --user=" user
Run builds and tests as \fIuser\fR on the testbed.  This needs root on
the testbed; if root on the testbed is not available then builds and
tests run as whatever user is provided.

.TP
.BI --gain-root= gain-root
Prefixes
.B debian/rules binary
with
.RB gain-root .
The default is not to use anything, except that if
\fB--user\fR is supplied or root on the testbed is not available the
default is \fBfakeroot\fR.

.SH DEBUGGING OPTIONS

.TP
.BR --debug | -d
Include additional debugging information in the trace log.  Each
additional \fB-d\fR increases the debugging level; the current maximum
is \fB-ddd\fR.  If you like to see what's going on, \fR-d\fB or
\fR-dd\fB is recommended.

.TP
.BR --shell-fail | -s
Run an interactive shell in the testbed after a failed build, test, or
dependency installation.

.TP
.BR --shell
Run an interactive shell in the testbed after every test.

.SH TIMEOUT OPTIONS

.TP
.BR --timeout- \fIwhich\fR = \fIseconds\fR
Use a different timeout for operations on or with the testbed.  There
are five timeouts affected by five values of \fIwhich\fR:
.BR short :
supposedly
short operations like setting up the testbed's apt and checking the
state (default: 100s);
.BR install :
installation of packages including dependencies
(default: 3,000s);
.BR test :
test runs (default: 10,000s);
.BR copy :
copy files/directories between host and testbed
(default: 300s); and
.BR build :
builds (default:
100,000s).  The value must be specified as an integer number of seconds.

.TP
.BR --timeout-factor =\fIdouble\fR
Multiply all of the default timeouts by the specified factor (see
\fB--timeout-\fR\fIwhich\fR above).  Only the defaults are affected;
explicit timeout settings are used exactly as specified.

.SH LOCALE OPTIONS

.TP
.BI --set-lang= langval
When running commands on the testbed, sets the \fBLANG\fR environment
variable to \fIlangval\fR.  The default in \fBadt-run\fR is to set it
to \fBC.UTF-8\fR.
.TP

.BI --leave-lang
Suppresses the setting by \fBadt-run\fR of \fBLANG\fR on the testbed.
This results in tests and builds using the testbed's own normal
\fBLANG\fR value setting.

.SH OTHER OPTIONS

.TP
.BI --no-auto-control
Disable automatic test generation with autodep8, even if it is installed. In
that case, packages without tests will exit with code 8 ("No tests in this
package") just like without autodep8.

.TP
.BR \-h | \-\-help
Show command line help and exit.


.SH VIRTUALIZATION SERVER

.TP
\fB---\fR \fIvirt-server virt-server-arg\fR...
Specifies the virtualisation regime server, as a command and arguments
to invoke.
.I virt-server
must be an existing autopkgtest virtualization server such as
.B adt-virt-schroot
or
.BR adt-virt-qemu .
You can leave out the
.B adt-virt-
prefix and just specify the last part, e. g.
.BR schroot .

All the remaining arguments and options after
.B ---
are passed to the virtualisation server program. See the manpages of the
individual servers for how to use them.

.SH OUTPUT FORMAT
During a normal test run, one line is printed for each test.  This
consists of a short string identifying the test, some horizontal
whitespace, and either
.B PASS
or
.BR FAIL " reason"
or
.BR SKIP " reason"
where the pass/fail indication is separated by any reason by some
horizontal whitespace.

The string to identify the test consists of a short alphanumeric
string invented by \fBadt-run\fR to distinguish different command-line
arguments, the \fIargid\fR, followed by a hyphen and the test name.

Sometimes a
.B SKIP
will be reported when the name of the test is not known or not
applicable: for example, when there are no tests in the package, or a
there is a test stanza which contains features not understood by this
version of
.BR adt-run .
In this case
.B *
will appear where the name of the test should be.

If \fBadt-run\fR detects that erroneous package(s) are involved, it
will print the two lines
.BR "blame: " \fIblamed-thing\fR ...
and
.BR "badpkg: " \fImessage\fR.
Here each whitespace-separated \fIblamed-thing\fR is one of
.BI arg: argument
(representing a pathname found in a command line argument),
.BI dsc: package
(a source package name),
.BI deb: package
(a binary package name)
or possibly other strings to be determined.  This indicates which
arguments and/or packages might have contributed to the problem; the
ones which were processed most recently and which are therefore most
likely to be the cause of a problem are listed last.

.SH CONFIGURATION FILES

If you use lots of options or nontrivial virt server arguments, you can put any
part of the command line into a text file, with one line per option. E. g. you
can create a file
.I sid.cfg
with contents like

.RS
.EX
-s
--output-dir=/tmp/testout
--apt-upgrade
---
schroot
sid
.EE
.RE

and then run

.RS
.EX
adt-run foo_1_amd64.changes @sid.cfg
.EE
.RE

The contents of the configuration file will be expanded in-place as if you
would have given its contents on the command line. Please ensure that you
.B don't place spaces
between short options and their values, they would become a part of the
argument value.


.SH EXIT STATUS
0	all tests passed
.br
2	at least one test skipped
.br
4	at least one test failed
.br
6	at least one test failed and at least one test skipped
.br
8	no tests in this package
.br
12	erroneous package
.br
16	testbed failure
.br
20	other unexpected failures including bad usage

.SH SEE ALSO
\fB/usr/share/doc/autopkgtest/README.running-tests.rst.gz\fR
.br
\fB/usr/share/doc/autopkgtest/README.package-tests.rst.gz\fR

.SH AUTHORS AND COPYRIGHT
This manpage is part of autopkgtest, a tool for testing Debian binary
packages.  autopkgtest is Copyright (C) 2006-2014 Canonical Ltd.

See \fB/usr/share/doc/autopkgtest/CREDITS\fR for the list of
contributors and full copying conditions.