~fginther/auto-package-testing/merge-releases-update : revision 88

1

Introduction

2

============

3

4

The purpose of this document is to describe the workflow auto package test

5

running in the QA Lab. General description of the project, usage and syntax

6

of the tools is described in the documents README.md, USAGE.md and tools CLI

7

help.

8

9

10

Auto package testing workflow

11

=============================

12

13

Testbed Setup

14

-------------

15

16

Testbed is prepared with the script *bin/prepare-testbed*. It builds a pristine

17

VM from a cloud-image using cloud-init.

18

19

By default it uses the latest server daily build from cloud-images.ubuntu.com.

20

21

This script is triggered by Jenkins when new images are available from

22

cloud-images.ubuntu.com i.e when the URL

23

http://cloud-images.ubuntu.com/quantal/current/MD5SUMS changes.

24

25

It takes approximately 20 minutes to creates fresh amd64 and i386 testing

26

environments.

27

28

Once the environment is setup, pristine VM are created in

29

30

${DISKDIR}/pristine-${RELEASE}-${ARCH}.img

31

32

This file is a symbolic link to the latest environment provisioned and the

33

previous is not destroyed to keep a possibility of manually reverting to the

34

last known good image.

35

36

37

Build trigger

38

-------------

39

40

The master script to trigger builds is *jenkins/trigger-adt-test*.

41

42

This script is triggered when the archive indexes are refreshed, i.e when one of

43

the following file changes:

44

45

* http://archive.ubuntu.com/dist/<RELEASE>/Release

46

* http://archive.ubuntu.com/dist/<RELEASE>-update/Release

47

* http://archive.ubuntu.com/dist/<RELEASE>-proposed/Release (Only used if

48

-proposed testing is enabled with -P)

49

50

This jobs execute the following tasks:

51

52

* Refresh the archive indices

53

* Check Sources index for packages with an XS-Testsuite header and a value of

54

autopkgtest

55

* If a new package is found, adds it to Jenkins. The job is created dynamically

56

from the XML template file *jenkins/jenkins_config.xml.tmpl* . This template

57

file uses Jinja2 syntax.

58

* If the script runs in update mode (with -U) existing jobs are also updated

59

from the template file.

60

* Loads the state of the last run (state files are located in

61

*$APTROOT/var/cache/adt/<RELEASE>/<PACKAGE>.state*) and triggers a build if:

62

* New package with tests.

63

* New version of the package is available.

64

* New version of a dependency of a binary built from the source package is

65

available. This is limited to first level dependencies.

66

* If testing from -proposed is enabled:

67

* A new version of the package is available in -proposed

68

* The package is in -release or -update and a new version of a binary

69

dependency is available from -proposed.

70

* The package is in -proposed and a new version of a binary dependency

71

is available from -release or -updates.

72

* Remotely trigger a jenkins jobs. Jenkins credentials and secret token are

73

defined in *jenkins/crendentials*.

74

* Update the state file of the package with the version of the package under

75

test, the list of dependencies and their version, the cause of the build

76

list of source packages and versions)

77

78

79

autopkgtest job

80

---------------

81

82

The script called to run autopkgtest is *bin/run-adt-test*. It is called

83

remotely by *jenkins/trigger-adt-test*.

84

85

This script executes the following tasks:

86

87

* Prepares a snapshot of the pristine VM to run the test and start it.

88

* Upload the runner to the testbed. The runner is the script

89

*bin/testbed/run-adt*.

90

* Call the runner inside the VM and wait until end of execution.

91

* Collect the results and copy them locally.

92

* Call *jenkins/update-adt-result* to update the state file of the package

93

with the result of the test.

94

* exit with *adt-run* exit code.

95

* On exit, destroy the testing environment and lock files.

96

97

The runner inside the VM executes the following tasks:

98

99

* Prepares the execution environment

100

* Add proposed and ppa apt sources if required

101

* update package cache and apply updates

102

* Downloads the source package in download mode only

103

* Run adt-run against the .dsc

104

* For reference this script can also run adt against:

105

* A package installed from a PPA

106

* A pacakge installed and a packaging branch in bzr

107

* A build of a package from a bzr branch. To build the package just pass the

108

branch name as argument without the source package name.

109

* Exit with the exit code of adt-run

110

111

112

Results

113

-------

114

115

The state file is updated at the end of the run by the script

116

*jenkins/update-adt-result*.

117

118

This script updates the result for a given architecture. The possible values for

119

a result are PASS or FAIL.

120

121

When all the architectures are tested, it updates the global result of the tests

122

in the state file. The status can be:

123

124

* PASS: If tests succeeded on all architectures

125

* FAIL: If tests failed on at least 1 architecture

126

127

It then creates a summary file

128

*$APTROOT/var/cache/<RELEASE>/<PACKAGE>_<TIMESTAMP>* which can be consumed by a

129

machine to automatically proceed with the publication of the package. The format

130

of the summary file is:

131

132

133

134

<causes> is a list of [<scrpkg:vesion> ...]

135

136

137

The results are then collected by a central server for publication. This central

138

server periodically:

139

140

* Copies the summary files from the test server with rsync in move mode.

141

* Aggregates the summary files in a single file following timestamp order. This

142

file can contain more than one record for a given package.

143

* Archives the summary file in

144

*/var/local/adt/<RELEASE>archive/<YYYY>/<mm>/<dd>/*.

145

* Creates a symbolic link to the latest summary file for every package in

146

*/var/local/adt/latest/*.