~canonical-platform-qa/snappy-ecosystem-tests/fixing_ci

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
Snappy Ecosystem Tests
######################

Automated system tests for snappy ecosystem: snapcraft, store and snapd.


To run the tests checkout the branch and follow the below steps
===============================================================
$ bzr branch lp:snappy-ecosystem-tests
$ cd snappy-ecosystem-tests/
$ ./run_system_tests snappy_ecosystem_tests

It will create a virtual env with all the dependencies defined in requirements.txt and run all the tests
A pytest runner is used: http://docs.pytest.org/en/latest/

To run all tests in a python package:
$ ./run_system_tests snappy_ecosystem_tests/tests/example_package

To run all test in a python module:
$ ./run_system_tests snappy_ecosystem_tests/tests/example_package/example_module.py

To run all tests in a python class:
$ ./run_system_tests snappy_ecosystem_tests/tests/example_package/example_module.py::ExampleClass

To run a particular test in a python class:
$ ./run_system_tests snappy_ecosystem_tests/tests/example_package/example_module.py::ExampleClass::example_test

To run a tests that match with a expression:
./run_system_tests -k stringexpr
# only run tests with names that match the
# "string expression", e.g. "MyClass and not method"
# will select TestMyClass.test_something
# but not TestMyClass.test_method_simple

You can also specify the arguments as python packages separated by dots:
$ ./run_system_tests --pyargs tests.example_package.example_module
Notice that in this case you skip the root dir snappy_ecosystem_tests and for python modules skip the .py

To see all options refer to: http://doc.pytest.org/en/latest/usage.html#specifying-tests-selecting-tests

Tests Results
=============
By default the tests results will be stored in 'snappy-ecosystem-results.xml'

Parameters
==========
If you are behind a proxy you can specify --proxy option
All pytest parameters are allowed

Config files
============
You can find configurations files in 'config' folder:

ecosystem_tests.cfg stores the snappy ecosystem related config like: APIs urls, Web urls, store to use, etc.
You can change this config file location by setting the env variable: XDG_CONFIG_HOME

pytest.cfg stores the config related to pytest

Changing store:
===============
You can change the store to use in config file ecosystem_tests.cfg
or by setting the env variable TEST_STORE with values: 'staging' or 'production'.
It will first try to read the config file, then the env variable and finally
if none of them are provided it will take 'staging' as default.


Running static code analysis and/or unittest
============================================
To run static code analysis you can execute from root directory:
$ ./run_checks --static
It will store the results in pylint-results.txt
The config file with pylint rules is pylint.cfg

If you want to run unittests:
$ ./run_checks --unit

If you want to run both:
$ ./run_checks --all

If you are behind a proxy you can specify --proxy option