~ubuntu-archive/britney/britney2 : revision 787

1

A short introduction to migrations

2

==================================

3

4

This is a technical introduction to how britney

5

handles migrations. Being an introduction, it deliberately

6

oversimplifies certain things at the expense of accuracy.

7

It also covers common migration issues and how to fix

8

them in :doc:`solutions-to-common-policy-issues`.

9

10

The document is primarily aimed at contributors for

11

distributions that want to understand the basics of

12

britney and its migration rules.

13

14

The documentation also aspires to be a general purpose document

15

for britney that is applicable for multiple distributions.

16

However, it does reference distribution-specific practises in

17

some examples to prevent the documentation from becoming too

18

abstract. Furthermore, the document assumes familiarity with

19

Debian-based distribution practises and terminology (such as

20

"suites" and "source package").

21

22

A high level overview of britney and migrations

23

-----------------------------------------------

24

25

The purpose of britney is to (semi-)automatically select

26

a number of migration items from a series of source suites

27

(e.g. Debian unstable) that are ready to migrate to

28

the target suite (e.g. Debian testing).

29

30

The definition of "ready" can be summarized as satisfying all

31

of the following points:

32

33

1. The migration items pass a number of policies for the target

34

suite. Most of these policies are basically that the

35

migration items do not regress on selected QA checks.

36

37

* An item satisfying this part is called a `valid candiate`.

38

39

2. Installability will not regress as a result of

40

migrating the migration items.

41

42

* An item that (also) satisfies this part will be selected

43

for migration.

44

45

The keyword in both points being *regress*. If a package has an

46

existing issue in the target suite, the item including a new version

47

of that package is generally allowed to migrate if it has the same

48

issue (as it is not a regression).

49

50

This only leaves the definition of a migration items. They come

51

in several variants defined in the next section.

52

53

Migration items

54

---------------

55

56

Internally, britney groups packages into migration items based on a

57

few rules. There are several kinds of migration items and this

58

document will only describe the source migration item.

59

60

A source migration item is one upload of a source package, with

61

associated binary packages once built.

62

63

Once a new version of a source package appears in the source suite,

64

britney will create track it with a source migration item. As the

65

binary packages are built and uploaded, they will be included into the

66

migration item and various QA checks/policies will be applied to the

67

item.

68

69

Once britney deems the item ready, it will attempt to

70

migrate the item (i.e. source with its binaries) to the

71

target suite.

72

73

74

As implied earlier, there are several other migration types. But they

75

are not covered in this document. They deal with cases like removals,

76

rebuilds of existing binaries, etc.

77

78

Migration phase 1: Policies / Excuses

79

-------------------------------------

80

81

To begin with, britney will apply a number of policies to

82

all migration items. Each policy will rate each migration

83

item and the combined results will be added into one of

84

britney's output documents known as the "excuses" (exists in

85

an HTML and a YAML variant). A migration item that passes all

86

applicable policies will be labelled as a `valid candidate` in

87

the excuses and continue to the next phase.

88

89

90

The policies gives exactly one verdict to each item, some of

91

these verdicts are:

92

93

* The item passes the policy.

94

* The policy is waiting for test suites before providing a

95

pass/fail result (temporary failure).

96

* The item fails the policy and the failure is believed to

97

be "permanent" (given no external changes).

98

* The item does not pass the policy, but britney has

99

insufficient information to determine if the failure is

100

persistent or not.

101

102

It is important to note that all verdicts are based on the current

103

data that britney has access to. This mean that without any change

104

to the items themselves:

105

106

1. Items that passed originally may fail in a later britney run.

107

108

2. Likewise, items may go from a "permanent failure" to a pass.

109

110

This can be seen in the following example case:

111

112

1. A new version of package is uploaded.

113

114

* Britney processes the package and concludes that there no blocking bugs,

115

so the package passes the bug policy.

116

117

2. Then before it migrates, someone files a blocking bug against

118

the new version.

119

120

* Britney reprocesses the package and now concludes it has a regression in

121

the bug policy (i.e. the policy verdict goes from "pass" to "permanent fail").

122

123

3. The bug is examined and it is determined that the bug also affects the

124

version in the target suite. The bug tracker is updated to reflect this.

125

126

* Britney reprocesses the package again and now concludes there is a blocking

127

bug, but it is not a regression (since it also affects the target suite).

128

This means the policy verdict now go from "fail" to "pass".

129

130

This is also applicable to e.g. the piuparts policy, where if the test is

131

rescheduled on the piuparts side and the result changes as a result of that.

132

133

Finally, the people running the britney instance can overrule any

134

policy by applying a [britney hint](hints.html), if they deem it

135

necessary. One caveat here is that not all policies can be overridden

136

directly and some will require the "ignore all policies"-hint (known

137

as the `force`-hint).

138

139

Since most policies are defined based on regressions,

140

a hinted migration generally implies that the problem will not

141

prevent future migrations for newer versions of the same source

142

package (assuming that the problem is deterministic).

143

144

Migration phase 2: Installability regression testing

145

----------------------------------------------------

146

147

For the migration items that pass the previous phase, britney

148

will do a test migration to see if anything becomes uninstallable.

149

This is a more expensive test to ensure the migration does not cause

150

installability regressions.

151

152

The status of this phase is *not* included in the excuses. To debug

153

problems here, the britney log file has to be examined. This requires

154

a bit more technical insight as it has not been polished as much as

155

the excuses.

156

157

Confirming a migration

158

^^^^^^^^^^^^^^^^^^^^^^

159

160

To start with; if a migration is accepted and "committed" (i.e. it will not

161

be rolled back), britney will include in a line starting with `final:` like

162

in this example::

163

164

Apparently successful

165

final: -cwltool,-libtest-redisserver-perl,-pinfo,-webdis,hol88

166

start: 41+0: a-4:i-27:a-1:a-1:a-1:m-0:m-3:m-1:p-1:s-2

167

orig: 41+0: a-4:i-27:a-1:a-1:a-1:m-0:m-3:m-1:p-1:s-2

168

end: 41+0: a-4:i-27:a-1:a-1:a-1:m-0:m-3:m-1:p-1:s-2

169

SUCCESS (182/177)

170

171

The above example is a regular migration run where 4 source removal migration

172

items and one source migration item where accepted (those listed on the

173

`final:` line). The rest of the information are various statistical counters

174

which are useful for other purposes beyond the scope of this document.

175

176

When debugging a migration for an item that passed the previous phase, if the

177

item appears on a `final:` line like that, then it is migrated. That is, the

178

problem is most likely that the britney run crashes later or the britney's

179

output is not committed to the archive (for reasons outside britney's control).

180

181

On the flip side, if the migration item of interest does *not* appear in a

182

final line, then the migration was rejected (or rolled back).

183

184

Reminder: Migration items generally use the name of the source package. There

185

are exceptions to that "rule" (but they are not common cases covered by this

186

document).

187

188

Debugging failed migration attempts

189

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

190

191

Start by confirming that the migration item was not accepted (as described

192

in the above section). If the migration item does not appear on a `final:` line,

193

then we need to debug the actual migration attempts. Migration attempts look

194

something like this::

195

196

trying: -webdis

197

accepted: -webdis

198

ori: 41+0: a-4:i-27:a-1:a-1:a-1:m-0:m-3:m-1:p-1:s-2

199

pre: 41+0: a-4:i-27:a-1:a-1:a-1:m-0:m-3:m-1:p-1:s-2

200

now: 41+0: a-4:i-27:a-1:a-1:a-1:m-0:m-3:m-1:p-1:s-2

201

all: -pinfo -webdis

202

[...]

203

trying: libaws

204

skipped: libaws (0, 165, 11)

205

got: 45+0: a-4:i-27:a-5:a-1:a-1:m-0:m-3:m-1:p-1:s-2

206

* arm64: libaws-bin, libaws17.2.2017, libaws3.3.2.2-dev, liblog4ada3-dev

207

[...]

208

Trying easy from autohinter: asis/2017-1 dh-ada-library/6.12 [...]

209

start: 41+0: a-4:i-27:a-1:a-1:a-1:m-0:m-3:m-1:p-1:s-2

210

orig: 41+0: a-4:i-27:a-1:a-1:a-1:m-0:m-3:m-1:p-1:s-2

211

easy: 261+0: a-26:i-49:a-23:a-23:a-23:m-22:m-25:m-23:p-23:s-24

212

* amd64: asis-programs, libasis2017, libasis2017-dev, libaws-bin, [...]

213

* i386: asis-programs, libasis2017, libasis2017-dev, libaws-bin, [...]

214

* arm64: asis-programs, libasis2017, libasis2017-dev, libaws-bin, [...]

215

* armel: asis-programs, libasis2017, libasis2017-dev, libaws-bin, [...]

216

[...]

217

FAILED

218

219

This example has one succeeding migration (`-webdis`) and one failing

220

(`libaws`) plus finally a failed `easy`-hint with several packages.

221

Both of the two first are "single item" migrations (i.e. the attempt only

222

includes a single item in isolation). However, Britney can do multi-item

223

migrations (even outside hints).

224

225

Please keep in mind that items can attempted multiple times and accepted in a

226

later attempt. It is not always immediately obvious, which attempt is better

227

for debugging. When in doubt, it is *usually* easiest to look at the attempt

228

with the least amount of new uninstallable packages.

229

230

In the libaws example, a total of 4 binary packages become

231

uninstallable on the architecture `arm64`. Here is the output again

232

with this information high lighted::

233

234

migration item(s) being attemped

235

vvvvvv

236

trying: libaws

237

skipped: libaws (0, 165, 11)

238

got: 45+0: a-4:i-27:a-5:a-1:a-1:m-0:m-3:m-1:p-1:s-2

239

* arm64: libaws-bin, libaws17.2.2017, libaws3.3.2.2-dev, liblog4ada3-dev

240

^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

241

||||| The binary packages becoming uninstallable (here 4)

242

Affected architecture (here "arm64")

243

244

Please note that britney is lazy and will often reject an item after proving

245

that there is a regression on a single architecture. So in the above example,

246

we are not actually sure whether this problem is architecture specific. For

247

`easy`-hints, the information is presented slightly different::

248

249

Trying easy from autohinter: asis/2017-1 dh-ada-library/6.12 [...]

250

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

251

migration item(s) being attemped

252

253

[... several lines of statistics from start, before and after ...]

254

* amd64: asis-programs, libasis2017, libasis2017-dev, libaws-bin, [...]

255

^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

256

||||| The binary packages becoming uninstallable on amd64

257

Affected architecture (here "amd64")

258

259

* i386: asis-programs, libasis2017, libasis2017-dev, libaws-bin, [...]

260

^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

261

||||| The binary packages becoming uninstallable on i386

262

Affected architecture (here "i386")

263

[... more architectures with binary packages becoming uninstallable ...]

264

265

266

While this tells us what britney tried to migrate and what would break (become

267

uninstallable) as a result, it is not very helpful at explaining *why*

268

things break. If there are few broken packages, it is often a question of

269

looking for `Breaks`-relations or `Depends`-relations with upper bounds on

270

versions / on old packages being removed. Alternatively, there are also tools

271

like `dose-debcheck`, which attempts to analyse and explain problems like this.