1
Checkbox Enhancement Proposal 1: Template Jobs
6
I'd like to propose that we design and implement template jobs and replace all
7
occurrences of local jobs with equivalent template jobs.
12
Local jobs are very problematic for applications and the core alike. They
13
prevent us from having access to a complete set of jobs all times. That, in
14
turn, prevent us from having simple application execution semantics and
15
severely limits our possibility of catching bugs in job definitions.
17
The mechanism of template jobs is currently used to generate new jobs at
18
runtime, typically to do one test for each of some device or other entity). The
19
code that generates such values is very arcane and goes back deep into old
20
checkbox support scripts. Such essential task should be handled by the library
21
as first-class object, with proper definition, semantics and implementation
24
As a side feature, local jobs are abused to generate "categories". This
25
proposal does not touch that aspect, except in "future work" section below
27
Overview of template jobs
28
=========================
30
Template jobs are just like local jobs in expressiveness, with the limitation
31
that the set of generated jobs (defined as a collection of their attributes,
32
including their name) are constrained and known statically without executing
33
any code provided by the job definition.
35
Instead, of parsing output of arbitrary shell commands, the job definition is
36
given explicitly. As a new feature, each attribute of a template job can be
37
parametrized. The list of parameters is known statically (cannot be generated).
38
This allows us to enumerate possible patterns and instantiate jobs when
41
Each parameter is bound to an attribute of a resource. This creates a
42
dependency on a specific resource job from any local job. In addition, to
43
implement a common theme used by local jobs, the set of resource records can be
44
filtered by a restricted python expression. This is similar in spirit to
45
existing requirements system.
47
For this proposal, each template job must use exactly one resource. In future
48
this requirement may be lifted, if required.
53
The following example converts an existing local job, optical/read, to a template job. This is the
54
(abbreviated) text of the original definition.
59
device.category == 'CDROM'
60
_description: Optical read test.
62
cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=CDROM"'
63
plugin: user-interact-verify
64
name: optical/read_`ls /sys$path/block`
65
requires: device.path == "$path"
67
command: optical_read_test /dev/`ls /sys$path/block`
68
description: (removed)
71
This is the (equivalent template job). The actual "local" job fragment of the
72
previous text is now gone. Any attribute may now use python formatting
73
language. This language is safe and can be parsed and analyzed if required.
75
Two new attributes are present. The template-resource attribute contains a list
76
of jobs (which must be resource jobs) that instantiate concrete job definitions
77
from this template. The template-filter attribute applies filtering, to select
78
a subset of records from each resource. As mentioned earlier, exactly one
79
resource is allowed at this time. In the future we _may_ offer two or more but
80
with additional join semantics that seems unnecessary at this stage.
82
template-resource: device
83
template-filter: device.category == "CDROM"
84
plugin: user-interact-verify
85
name: optical/read_{device.full_path}
87
command: optical_read_test {device.dev_path}
88
description: (removed)
90
The previous definition used some additional shell-outs to compute the
91
pathnames of block devices. This feature is now removed, leading to
92
easier-to-understand code. The device local job would need to be amended to
93
generate all required attributes.
98
PlainBox would need to understand templates. That includes additional job for
99
job validator, dependency resolver / test planner, etc. The new API would need
100
to include an explicit transformation from a set of known definitions into a
101
set of actual instantiated jobs. Applications would need to use this API
102
similarly to how they currently handle "early jobs" to run all of the local
103
jobs, but with less required error handling.
105
A set of additional commands would be added to help developers how their
106
template jobs transform into actual instances. This is an optional but, in my
107
opinion, useful addition. It would allow developers to understand why something
108
fails (how their template definition is expanded into an actual job
111
Local jobs would be unchanged and would remain supported as they are now.
116
As we convert local jobs to template jobs, we could add an additional
117
requirement that local jobs may only generate existing jobs, to implement the
118
illy-defined "category" system, before that is deprecated and replaced.
120
Eventually we could bump job definition version to 2, making local jobs
121
unsupported. This will allow applications to drop complexity related such jobs.
122
This can happen at any time in the future though.
added
removed
cep/CEP-1.txt
