1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
|
Snappy Ecosystem Tests
######################
Automated system tests for snappy ecosystem: snapcraft, store and snapd.
To run the tests checkout the branch and follow the below steps
===============================================================
$ bzr branch lp:snappy-ecosystem-tests
$ cd snappy-ecosystem-tests/
$ ./run_system_tests snappy_ecosystem_tests
It will create a virtual env with all the dependencies defined in requirements.txt and run all the tests
A pytest runner is used: http://docs.pytest.org/en/latest/
To run all tests in a python package:
$ ./run_system_tests snappy_ecosystem_tests/tests/example_package
To run all test in a python module:
$ ./run_system_tests snappy_ecosystem_tests/tests/example_package/example_module.py
To run all tests in a python class:
$ ./run_system_tests snappy_ecosystem_tests/tests/example_package/example_module.py::ExampleClass
To run a particular test in a python class:
$ ./run_system_tests snappy_ecosystem_tests/tests/example_package/example_module.py::ExampleClass::example_test
To run a tests that match with a expression:
./run_system_tests -k stringexpr
# only run tests with names that match the
# "string expression", e.g. "MyClass and not method"
# will select TestMyClass.test_something
# but not TestMyClass.test_method_simple
You can also specify the arguments as python packages separated by dots:
$ ./run_system_tests --pyargs tests.example_package.example_module
Notice that in this case you skip the root dir snappy_ecosystem_tests and for python modules skip the .py
To see all options refer to: http://doc.pytest.org/en/latest/usage.html#specifying-tests-selecting-tests
Tests Results
=============
By default the tests results will be stored in 'snappy-ecosystem-results.xml'
Parameters
==========
If you are behind a proxy you can specify --proxy option
All pytest parameters are allowed
Config files
============
You can find configurations files in 'config' folder:
ecosystem_tests.cfg stores the snappy ecosystem related config like: APIs urls, Web urls, store to use, etc.
You can change this config file location by setting the env variable: XDG_CONFIG_HOME
pytest.cfg stores the config related to pytest
User Credentials
================
For storing user credentials you have 2 options:
option 1 - You can create a config file in your local machine with path:
~/.config/ecosystem_tests.cfg
You can change the dir location of that config file by setting the env variable:
XDG_USER_CONFIG_HOME=^DIR_PATH^
The config file should look like:
[user]
user_email=^USER_NAME^
user_password=^USER_PASSWORD^
snapd_hostname_remote=^SNAPD_SSH_HOSTNAME^
snapd_username_remote=^SNAPD_SSH_USERNAME^
snapd_port_remote=^SNAPD_SSH_PORT^
snapcraft_hostname_remote=^SNAPCRAFT_SSH_HOSTNAME^
snapcraft_username_remote=^SNAPCRAFT_SSH_USERNAME^
snapcraft_port_remote=^SNAPCRAFT_SSH_PORT^
option 2 - Set the following environment variables:
user_email=^USER_NAME^
user_password=^USER_PASSWORD^
snapd_hostname_remote=^SNAPD_SSH_HOSTNAME^
snapd_username_remote=^SNAPD_SSH_USERNAME^
snapd_port_remote=^SNAPD_SSH_PORT^
snapcraft_hostname_remote=^SNAPCRAFT_SSH_HOSTNAME^
snapcraft_username_remote=^SNAPCRAFT_SSH_USERNAME^
snapcraft_port_remote=^SNAPCRAFT_SSH_PORT^
Changing store:
===============
You can change the store to use in config file ecosystem_tests.cfg
or by setting the env variable TEST_STORE with values: 'staging' or 'production'.
It will first try to read the config file, then the env variable and finally
if none of them are provided it will take 'staging' as default.
Running static code analysis and/or unittest
============================================
To run static code analysis you can execute from root directory:
$ ./run_checks --static
It will store the results in pylint-results.txt
The config file with pylint rules is pylint.cfg
If you want to run unittests:
$ ./run_checks --unit
If you want to run both:
$ ./run_checks --all
If you are behind a proxy you can specify --proxy option
How to setup snapd staging environment
======================================
Requirements
1. Ubuntu 16.04.1+ with all updates applied
2. LXD >= 2.4.1
To enable staging for snapd, it needs to be re-built with special flags, the script in
tests/utils/build_snapd_staging.py have all the bits in place to achieve that. It can be
run as a standalone script or could be incorporated into snapd tests. Just run the script
to setup the environment.
$ bzr branch lp:snappy-ecosystem-tests
$ cd snappy-ecosystem-tests/
$ python3 -m snappy_ecosystem_tests.snapd.staging_builder my_new_container_name 16.10
|