~snappy-ecosystem-tests-dev/snappy-ecosystem-tests/trunk

Snappy Ecosystem Tests

######################

Automated system tests for snappy ecosystem: snapcraft, store and snapd.

To run the tests checkout the branch and follow the below steps

===============================================================

$ bzr branch lp:snappy-ecosystem-tests

$ cd snappy-ecosystem-tests/

$ ./run_system_tests snappy_ecosystem_tests

It will create a virtual env with all the dependencies defined in requirements.txt and run all the tests

A pytest runner is used: http://docs.pytest.org/en/latest/

To run all tests in a python package:

$ ./run_system_tests snappy_ecosystem_tests/tests/example_package

To run all test in a python module:

$ ./run_system_tests snappy_ecosystem_tests/tests/example_package/example_module.py

To run all tests in a python class:

$ ./run_system_tests snappy_ecosystem_tests/tests/example_package/example_module.py::ExampleClass

To run a particular test in a python class:

$ ./run_system_tests snappy_ecosystem_tests/tests/example_package/example_module.py::ExampleClass::example_test

To run a tests that match with a expression:

./run_system_tests -k stringexpr

# only run tests with names that match the

# "string expression", e.g. "MyClass and not method"

# will select TestMyClass.test_something

# but not TestMyClass.test_method_simple

You can also specify the arguments as python packages separated by dots:

$ ./run_system_tests --pyargs tests.example_package.example_module

Notice that in this case you skip the root dir snappy_ecosystem_tests and for python modules skip the .py

To see all options refer to: http://doc.pytest.org/en/latest/usage.html#specifying-tests-selecting-tests

Tests Results

=============

By default the tests results will be stored in 'snappy-ecosystem-results.xml'

Parameters

==========

If you are behind a proxy you can specify --proxy option

All pytest parameters are allowed

Config files

============

You can find configurations files in 'config' folder:

ecosystem_tests.cfg stores the snappy ecosystem related config like: APIs urls, Web urls, store to use, etc.

You can change this config file location by setting the env variable: XDG_CONFIG_HOME

pytest.cfg stores the config related to pytest

User Credentials

================

For storing user credentials you have 2 options:

option 1 - You can create a config file in your local machine with path:

~/.config/ecosystem_tests.cfg

You can change the dir location of that config file by setting the env variable:

XDG_USER_CONFIG_HOME=^DIR_PATH^

The config file should look like:

[user]

user_email=^USER_NAME^

user_password=^USER_PASSWORD^

snapd_hostname_remote=^SNAPD_SSH_HOSTNAME^

snapd_username_remote=^SNAPD_SSH_USERNAME^

snapd_port_remote=^SNAPD_SSH_PORT^

snapcraft_hostname_remote=^SNAPCRAFT_SSH_HOSTNAME^

snapcraft_username_remote=^SNAPCRAFT_SSH_USERNAME^

snapcraft_port_remote=^SNAPCRAFT_SSH_PORT^

option 2 - Set the following environment variables:

user_email=^USER_NAME^

user_password=^USER_PASSWORD^

snapd_hostname_remote=^SNAPD_SSH_HOSTNAME^

snapd_username_remote=^SNAPD_SSH_USERNAME^

snapd_port_remote=^SNAPD_SSH_PORT^

snapcraft_hostname_remote=^SNAPCRAFT_SSH_HOSTNAME^

snapcraft_username_remote=^SNAPCRAFT_SSH_USERNAME^

snapcraft_port_remote=^SNAPCRAFT_SSH_PORT^

Changing store:

===============

You can change the store to use in config file ecosystem_tests.cfg

or by setting the env variable TEST_STORE with values: 'staging' or 'production'.

It will first try to read the config file, then the env variable and finally

if none of them are provided it will take 'staging' as default.

Running static code analysis and/or unittest

============================================

To run static code analysis you can execute from root directory:

$ ./run_checks --static

It will store the results in pylint-results.txt

The config file with pylint rules is pylint.cfg

If you want to run unittests:

$ ./run_checks --unit

If you want to run both:

$ ./run_checks --all

If you are behind a proxy you can specify --proxy option

How to setup snapd staging environment

======================================

Requirements

1. Ubuntu 16.04.1+ with all updates applied

2. LXD >= 2.4.1

To enable staging for snapd, it needs to be re-built with special flags, the script in

tests/utils/build_snapd_staging.py have all the bits in place to achieve that. It can be

run as a standalone script or could be incorporated into snapd tests. Just run the script

to setup the environment.

$ bzr branch lp:snappy-ecosystem-tests

$ cd snappy-ecosystem-tests/

$ python3 -m snappy_ecosystem_tests.snapd.staging_builder my_new_container_name 16.10