40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
1 |
================== |
2 |
Configuring Bazaar |
|
3 |
================== |
|
4 |
||
5 |
Goal |
|
6 |
==== |
|
7 |
||
8 |
Not all needs can be addressed by the default values used inside bzr and |
|
9 |
bzrlib, no matter how well they are chosen (and they are ;). |
|
10 |
||
62
by Vincent Ladeuil
Update to better match actual progress. |
11 |
Options that are rarely used don't deserve a corresponding command line |
12 |
switch in one or several commands. |
|
13 |
||
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
14 |
Many parts of ``bzrlib`` depends on some constants though and the user |
15 |
should be able to customize the behavior to suit his needs so these |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
16 |
constants need to become configuration options or more generally, be easier |
17 |
to set. |
|
18 |
||
19 |
These options can be set from the command-line, acquired from an environment |
|
20 |
variable or recorded in a configuration file. |
|
21 |
||
22 |
To simplify writing (and reading) this document refers to the old and new |
|
23 |
config designs: |
|
24 |
* the old design is using Config as a base class for all config files, |
|
25 |
* the new design use stacks of config files. |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
26 |
|
27 |
Current issues |
|
28 |
============== |
|
29 |
||
30 |
* Many parts of ``bzrlib`` declare constants and there is no way for the |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
31 |
user to look at or modify them (see http://pad.lv/832061). |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
32 |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
33 |
* The ld design requires a configuration object to create, modify or delete |
34 |
a configuration option in a given configuration file. ``bzr config`` |
|
35 |
makes it almost transparent for the user. Internally though, not all cases |
|
36 |
are handled: only BranchConfig implements chained configs, nothing is |
|
37 |
provided at the repository level and too many plugins define their own |
|
38 |
section or even their own config file. (config.Stack now provides a way to |
|
39 |
chain config files). |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
40 |
|
41 |
* ``locations.conf`` defines the options that needs to override any setting |
|
59
by Vincent Ladeuil
Some clarifications. |
42 |
in ``branch.conf`` for both local and remotes branches (but some remote |
43 |
branch options just ignore ``locations.conf``). Many users want a way to |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
44 |
define default values for options that are not defined in ``branch.conf`` |
45 |
(and even more users think that ``locations.conf`` provide default values, |
|
46 |
see also bug #843211 and bug #832046). This could be approximated today by |
|
47 |
*not* defining these options in ``branch.conf`` but in ``locations.conf`` |
|
48 |
instead. This workaround doesn't allow a user to define defaults in |
|
49 |
``locations.conf`` and override them in ``branch.conf``. (Allowing |
|
50 |
sections in 'bazaar.conf' (or introducing a new defaults.conf' allowing |
|
51 |
sections) can now address that.) |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
52 |
|
53 |
* Defining a new option requires adding a new method in the ``Config`` |
|
54 |
object to get access to features like: |
|
55 |
||
62
by Vincent Ladeuil
Update to better match actual progress. |
56 |
* should the option be inherited by more specific sections, (this was more |
57 |
or less the default in the old design, it is addressed by section |
|
58 |
matchers in the new one). |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
59 |
|
60 |
* should the inherited value append the relative path between the |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
61 |
section one and the location it applies to (see bug #832013), |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
62 |
|
63 |
* the default value (including calling any python code that may be |
|
61
by Vincent Ladeuil
Mention some existing bugs. |
64 |
required to calculate this value)(see bug #832064), |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
65 |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
66 |
* priority between sections in various config files (this is defined by |
67 |
the section matcher associated with a given config store for stacks, bug |
|
68 |
#832046 is about adding a section matcher with clearer semantics than |
|
69 |
the one used for locations.conf). |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
70 |
|
71 |
A related problem is that, in the actual implementation, some |
|
56
by Vincent Ladeuil
More thoughts about name spaces based on jelmer's review. |
72 |
configuration options have defined methods, others don't and this is |
62
by Vincent Ladeuil
Update to better match actual progress. |
73 |
inconsistent. (Using only Stacks address that). |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
74 |
|
75 |
* Access to the 'active' configuration option value from the command line |
|
48
by Vincent Ladeuil
Some updates to reflect the parts already landed. |
76 |
doesn't give access to the specific section. (This is only a concern if |
77 |
the user has no other way to address a specific configuration option |
|
61
by Vincent Ladeuil
Mention some existing bugs. |
78 |
including Store and Section when using ``bzr config``) (see bug #725234). |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
79 |
|
80 |
* Rules for configuration options are not clearly defined for remote |
|
42
by Vincent Ladeuil
More design ideas. |
81 |
branches (they may differ between dumb and smart servers the former will |
82 |
use the local ``bazaar.conf`` and ``locations.conf`` files while the later |
|
59
by Vincent Ladeuil
Some clarifications. |
83 |
will use (or ignore ?) the remote ones). |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
84 |
|
85 |
* The features offered by the Bazaar configuration files should be easily |
|
86 |
accessible to plugin authors either by supporting plugin configuration |
|
42
by Vincent Ladeuil
More design ideas. |
87 |
options in the configuration files or allowing the plugins to define their |
59
by Vincent Ladeuil
Some clarifications. |
88 |
own configuration files. (Separating Section, Store and Stack starts |
62
by Vincent Ladeuil
Update to better match actual progress. |
89 |
addressing that, a stack registry should complete the needed means). |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
90 |
|
91 |
* While the actual configuration files support sections, they are used in |
|
92 |
mutually exclusive ways that make it impossible to offer the same set of |
|
93 |
features to all configuration files: |
|
94 |
||
95 |
* ``bazaar.conf`` use arbitrary names for sections. ``DEFAULT`` is used |
|
96 |
for global options, ``ALIASES`` are used to define command aliases, |
|
97 |
plugins can define their own sections, some plugins do that |
|
98 |
(``bzr-bookmarks`` use ``BOOKMARKS`` for example), some other define |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
99 |
their own sections (this is addressed with the new design by using only |
100 |
the ``DEFAULT`` section and ignore the others. When needed, one can |
|
101 |
create a specific stack to get access to a specific section). |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
102 |
|
103 |
* ``locations.conf`` use globs as section names. This provides an easy |
|
104 |
way to associate a set of options to a matching working tree or |
|
105 |
branch, including remote ones. |
|
106 |
||
107 |
* ``branch.conf`` doesn't use any section. |
|
108 |
||
109 |
* There is no easy way to get configuration options for a given repository |
|
110 |
or an arbitrary path. Working trees and branches are generally organized |
|
111 |
in hierarchies and being able to share the option definitions is an often |
|
112 |
required feature. This can also address some needs exhibited by various |
|
113 |
branch schemes like looms, pipeline, colocated branches and nested |
|
114 |
trees. Being able to specify options *in* a working tree could also help |
|
115 |
support conflict resolution options for a given file, directory or |
|
61
by Vincent Ladeuil
Mention some existing bugs. |
116 |
subtree (see bug #359320). |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
117 |
|
118 |
* Since sections allow different definitions for the same option, a total |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
119 |
order should be defined between sections to select the right definition |
120 |
for a given context (paths or globs for ``locations.conf`` but other |
|
121 |
schemes can be used, window names for qbzr for example). Allowing globs |
|
122 |
for section names is harmful in this respect since the order is currently |
|
123 |
defined as being the lexicographical one. The caveat here is that if the |
|
124 |
order is always defined for a given set of sections it can change when one |
|
125 |
or several globs are modified and the user may get surprising and unwanted |
|
126 |
results in these cases. The lexicographical order is otherwise fine to |
|
127 |
define what section is more specific than another. (This may not be a |
|
128 |
problem in real life since longer globs are generally more specific than |
|
129 |
shorter ones and explicit paths should also be longer than matching |
|
130 |
globs. That may leave a glob and a path of equal length in a gray area but |
|
131 |
in practice using ``bzr config`` should give enough feedback to address |
|
132 |
them. See also bug #832046 asking for a less magical section matcher). |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
133 |
|
134 |
* Internally, configuration files (and their fallbacks, ``bazaar.conf`` and |
|
135 |
``locations.conf`` for ``branch.conf``) are read every time *one* option is |
|
136 |
queried. Likewise, setting or deleting a configuration option implies |
|
137 |
writing the configuration file *immediately* after re-reading the file to |
|
61
by Vincent Ladeuil
Mention some existing bugs. |
138 |
avoid racing updates (see bug #832042). |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
139 |
|
140 |
* The current implementation use a mix of transport-based and direct file |
|
48
by Vincent Ladeuil
Some updates to reflect the parts already landed. |
141 |
systems operations (Addressed by Store implementation relying on |
59
by Vincent Ladeuil
Some clarifications. |
142 |
transports only). |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
143 |
|
144 |
* While the underlying ``ConfigObj`` implementation provides an |
|
145 |
interpolation feature, the ``bzrlib`` implementation doesn't provide an |
|
146 |
easy handling of templates where other configuration options can be |
|
147 |
interpolated. Instead, ``locations.conf`` (and only it) allows for |
|
48
by Vincent Ladeuil
Some updates to reflect the parts already landed. |
148 |
``appendpath`` and ``norecurse``. (Partially implemented, cross-section |
62
by Vincent Ladeuil
Update to better match actual progress. |
149 |
and cross-file interpolation still to be implemented, see bug #832013 for |
150 |
the remaining parts). |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
151 |
|
152 |
* Inherited list values can't be modified, a more specific configuration can |
|
153 |
only redefine the whole list. |
|
154 |
||
155 |
* There is no easy way to define dicts (the most obvious one being to use a |
|
156 |
dedicated section which is already overloaded). Using embedded sections |
|
157 |
for this would not be practical either if we keep using a no-name section |
|
158 |
for default values. In a few known cases, a bencoded dict is stored in a |
|
159 |
config value, so while this isn't user-friendly, not providing a better |
|
48
by Vincent Ladeuil
Some updates to reflect the parts already landed. |
160 |
alternative shouldn't be a concern. A possible, limited, implementation |
56
by Vincent Ladeuil
More thoughts about name spaces based on jelmer's review. |
161 |
can be envisioned: limiting the dict to a single level only, with simple |
162 |
names as keys and unicode strings as values. The keys can then be mapped |
|
163 |
to options prefixed with the dict name. |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
164 |
|
165 |
||
166 |
Proposed implementation |
|
167 |
======================= |
|
168 |
||
169 |
||
170 |
Configuration files definition |
|
171 |
------------------------------ |
|
172 |
||
173 |
While of course configurations files can be versioned they are not intended |
|
174 |
to be accessed in sync with the files they refer to (one can imagine |
|
175 |
handling versioned properties this way but this is *not* what the bazaar |
|
176 |
configuration files are targeted at). ``bzr`` will always refer to |
|
177 |
configuration files as they exist on disk when an option is queried or set. |
|
178 |
||
179 |
The configuration files are generally local to the file system but some of |
|
180 |
them can be accessed remotely (``branch.conf``, ``repo.conf``). |
|
181 |
||
182 |
||
183 |
Naming |
|
184 |
------ |
|
185 |
||
56
by Vincent Ladeuil
More thoughts about name spaces based on jelmer's review. |
186 |
Option names are organized into a name space for a given configuration file |
187 |
(or a set of related configuration files). One such set includes |
|
188 |
``bazaar.conf``, ``locations.conf``, ``branch.conf``, etc. Plugins can |
|
189 |
define their own sets for their own needs. |
|
190 |
||
191 |
Using a name space is meant to help: |
|
192 |
||
193 |
* avoid collisions between bzr and plugins and between plugins, |
|
194 |
||
62
by Vincent Ladeuil
Update to better match actual progress. |
195 |
* discover the available options and making them easier to remember, |
56
by Vincent Ladeuil
More thoughts about name spaces based on jelmer's review. |
196 |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
197 |
* organise the documentation for the option set. |
56
by Vincent Ladeuil
More thoughts about name spaces based on jelmer's review. |
198 |
|
199 |
Using valid python identifiers is recommended but not enforced (but we may |
|
200 |
do so in the future). |
|
201 |
||
62
by Vincent Ladeuil
Update to better match actual progress. |
202 |
The option name space is organized by topic: |
203 |
||
204 |
* bzrlib options are grouped by topic (``branch``, ``tree``, ``repo``) |
|
205 |
||
206 |
* plugins are encouraged (but not required) to prefix their specific options |
|
207 |
with their name (``qbzr.`` for qbzr) |
|
59
by Vincent Ladeuil
Some clarifications. |
208 |
|
209 |
* collisions are detected at registration time so users are protected from |
|
210 |
incompatibilities between plugins, |
|
211 |
||
212 |
* options that need to be used by several plugins (or shared between ``bzr`` |
|
213 |
core and plugins) should be discussed but these discussions are already |
|
214 |
happening so the risk of misuse is low enough. |
|
215 |
||
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
216 |
Value |
217 |
----- |
|
218 |
||
219 |
All option values are text. They are provided as Unicode strings to API |
|
220 |
users with some refinements: |
|
221 |
||
222 |
* boolean values can be obtained for a set of acceptable strings (yes/no, |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
223 |
y/n, on/off, etc), (implemented with the ``from_unicode`` parameter) |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
224 |
|
225 |
* a list of strings from a value containing a comma separated list of |
|
226 |
strings. |
|
227 |
||
228 |
Since the configuration files can be edited by the user, ``bzr`` doesn't |
|
48
by Vincent Ladeuil
Some updates to reflect the parts already landed. |
229 |
expect their content to be valid at all times. Instead, the code using |
230 |
options should be ready to handle *invalid* values by warning the user and |
|
49
by Vincent Ladeuil
Some more bits from various discussions. |
231 |
falling back to a default value. |
232 |
||
233 |
Likely, if an option is not defined in any configuration file, the code |
|
234 |
should fallback to a default value (helpers should be provided by the API to |
|
235 |
handle common cases: warning the user, getting a particular type of value, |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
236 |
returning a default value)(most of that is now handled at Option definition). |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
237 |
|
238 |
This also ensures compatibility with values provided via environment |
|
49
by Vincent Ladeuil
Some more bits from various discussions. |
239 |
variables or from the command line (where no validation can be expected |
62
by Vincent Ladeuil
Update to better match actual progress. |
240 |
either)(done in the new design, some cases missing, see bug #832064). |
49
by Vincent Ladeuil
Some more bits from various discussions. |
241 |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
242 |
|
48
by Vincent Ladeuil
Some updates to reflect the parts already landed. |
243 |
Option expansion |
244 |
---------------- |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
245 |
|
246 |
Some option values can be templates and contain references to other |
|
247 |
options. This is especially useful to define URLs in sections shared for |
|
248 |
multiple branches for example. It can also be used to describe commands |
|
249 |
where some parameters are set by ``bzrlib`` at runtime. |
|
250 |
||
56
by Vincent Ladeuil
More thoughts about name spaces based on jelmer's review. |
251 |
Since option values are text-only, and to avoid clashing with other option |
48
by Vincent Ladeuil
Some updates to reflect the parts already landed. |
252 |
expansion (also known as interpolation) syntaxes, references are enclosed |
253 |
with curly brackets:: |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
254 |
|
255 |
push_location = lp:~{launchpad_username}/bzr/{nick} |
|
256 |
||
257 |
In the example above, ``launchpad_username`` is an already defined |
|
258 |
configuration option while ``nick`` is the branch nickname and is set when a |
|
259 |
configuration applies to a given branch. |
|
260 |
||
261 |
The interpolation implementation should accept an additional dict so that |
|
48
by Vincent Ladeuil
Some updates to reflect the parts already landed. |
262 |
``bzrlib`` or plugins can define references that can be expanded without |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
263 |
being existing configuration options:: |
264 |
||
265 |
diff_command={cmd} {cmd_opts} {file_a} {file_b} |
|
266 |
||
267 |
There are two common errors that should be handled when handling interpolation: |
|
268 |
||
269 |
* loops: when a configuration value refers to itself, directly or indirectly, |
|
270 |
||
271 |
* undefined references: when a configuration value refers to an unknown option. |
|
272 |
||
48
by Vincent Ladeuil
Some updates to reflect the parts already landed. |
273 |
The loop handling can be modified to allow cross-sections and cross-files |
274 |
interpolation: if an option refers to itself (directly or indirectly) during |
|
275 |
an expansion, the fallback sections or files can be queried for its value. |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
276 |
|
48
by Vincent Ladeuil
Some updates to reflect the parts already landed. |
277 |
This allows list values to refer to the definition in the less specific |
42
by Vincent Ladeuil
More design ideas. |
278 |
configurations:: |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
279 |
|
280 |
bazaar.conf: |
|
281 |
debug_flags = hpss |
|
282 |
||
283 |
branch.conf for mybranch: |
|
284 |
debug_flags = {debug_flags}, hpssdetail |
|
285 |
||
286 |
$ bzr -d mybranch config debug_flags |
|
287 |
hpss, hpssdetail |
|
288 |
||
48
by Vincent Ladeuil
Some updates to reflect the parts already landed. |
289 |
Undefined references are detected if they are not defined in any |
290 |
configuration. This will trigger errors while displaying the value. Diagnosing |
|
291 |
typos should be doable in this case. |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
292 |
|
293 |
Configuration file syntax |
|
294 |
------------------------- |
|
295 |
||
48
by Vincent Ladeuil
Some updates to reflect the parts already landed. |
296 |
The configuration file is mostly an ``ini-file``. It contains ``name = value`` |
297 |
lines grouped in sections. A section starts with a string enclosed in squared |
|
298 |
brackets ('[section_name]`), this string uniquely identifies the section in |
|
299 |
the file. Comments are allowed by prefixing them with the '#' character. |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
300 |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
301 |
A section is named by the path (or some other unuique identifier) it should |
302 |
apply to (more examples below). |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
303 |
|
304 |
When sections are used, they provide a finer grain of configuration by |
|
305 |
defining option sets that apply to some working trees, branches, |
|
48
by Vincent Ladeuil
Some updates to reflect the parts already landed. |
306 |
repositories (or any kind of context) or part of them. The relationship |
307 |
between a given context and the sections it applies to is defined by the |
|
308 |
config file. |
|
309 |
||
310 |
So far, Bazaar uses a glob in ``locations.conf`` and select the sections |
|
311 |
that apply to a given url (or a local path). |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
312 |
|
313 |
The subset is defined by the common leading path or a glob. |
|
314 |
||
62
by Vincent Ladeuil
Update to better match actual progress. |
315 |
Different kinds of section names can be defined: |
316 |
||
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
317 |
* a full url: used to described options for remote branches and |
62
by Vincent Ladeuil
Update to better match actual progress. |
318 |
repositories (LocationMatcher supports this). |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
319 |
|
320 |
* local absolute path: used for working trees, branches or repositories |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
321 |
on the local disks (LocationMatcher supports this). |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
322 |
|
323 |
* relative path: the path is relative to the configuration file and can be |
|
324 |
used for colocated branches or threads in a loom, i.e any working tree, |
|
325 |
branch or repository that is located in a place related to the |
|
326 |
configuration file path. Some configuration files may define this path |
|
327 |
relationship in specific ways to make them easier to use (i.e. if a config |
|
328 |
file is somewhere below ``.bzr`` and refers to threads in a loom for |
|
329 |
example, the relative path would be the thread name, it doesn't have to be |
|
330 |
an *exact* relative path, as long as its interpretation is unambiguous and |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
331 |
clear for the user) (No section matchers support this so far, needs to |
332 |
file a bug) |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
333 |
|
49
by Vincent Ladeuil
Some more bits from various discussions. |
334 |
Section matching |
335 |
---------------- |
|
336 |
||
337 |
Section names define another name space (than the option names one) with an |
|
338 |
entirely different purpose: in a given configuration file, for a given |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
339 |
context, only some sections will be relevant and will be queried in a |
340 |
specific order. |
|
49
by Vincent Ladeuil
Some more bits from various discussions. |
341 |
|
342 |
This matching is specific to each config file and implemented by the |
|
343 |
SectionMatcher objects. |
|
344 |
||
345 |
Whatever this matching does, the user can observe the results with the ``bzr |
|
346 |
config`` command which displays the sections in the order they are queried. |
|
347 |
||
348 |
LocationMatcher |
|
349 |
~~~~~~~~~~~~~~~ |
|
350 |
||
351 |
The context here is either: |
|
352 |
||
353 |
* an URL, |
|
354 |
||
355 |
* a local path. |
|
356 |
||
357 |
Note that for both the provided context and the section names, if an URL uses |
|
51
by Vincent Ladeuil
Clarify the know problems about compatibility. |
358 |
a ``file:///`` form, it is converted to a local path. |
49
by Vincent Ladeuil
Some more bits from various discussions. |
359 |
|
360 |
The sections names can use globs for each path component |
|
361 |
(i.e. ``/dir/*/subdir`` is allowed but ``/dir/\*\*/subdir`` will match |
|
362 |
``/dir/a/subdir`` but not ``/dir/a/b/subdir``. |
|
363 |
||
364 |
The reason is that the ordering is defined by sorting the section names |
|
365 |
matching the context on the number of path components followed by the path |
|
366 |
itself in lexicographical order. This results in most specific sections being |
|
367 |
queried before the more generic ones. |
|
368 |
||
369 |
PathMatcher |
|
370 |
~~~~~~~~~~~ |
|
371 |
||
55
by Vincent Ladeuil
PathMatcher and bzr config have different goals, don't try to make them use the same order in config files. |
372 |
``LocationMatcher`` has some obscure (for unaware users) edge cases and |
373 |
limitations that can be surprising. ``PathMatcher`` aims at addressing these |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
374 |
issues by providing simpler rules while still giving full control to the |
375 |
user (bug #832046). |
|
49
by Vincent Ladeuil
Some more bits from various discussions. |
376 |
|
377 |
The context here is a local path, absolute or relative. If the path is |
|
378 |
relative it is interpreted from the file base directory. |
|
379 |
||
380 |
Note that 'base directory' for configuration files in Bazaar directories is |
|
381 |
really: |
|
382 |
||
383 |
* the home directory for files under ``~/.bazaar``, |
|
384 |
||
385 |
* the ``.bzr`` base directory for files under ``.bzr``. |
|
386 |
||
55
by Vincent Ladeuil
PathMatcher and bzr config have different goals, don't try to make them use the same order in config files. |
387 |
The order is the one observed in the file so most generic values are specified |
388 |
first and most specific ones last. As such, the order in the file is the |
|
389 |
opposite of the one displayed by ``bzr config`` which displays most specific |
|
390 |
values first. This seems to be the most natural order in both cases. |
|
49
by Vincent Ladeuil
Some more bits from various discussions. |
391 |
|
392 |
A section matches if the section name is a prefix of the context path |
|
393 |
(relative paths being converted to absolute on the fly). |
|
394 |
||
48
by Vincent Ladeuil
Some updates to reflect the parts already landed. |
395 |
The Option object |
396 |
----------------- |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
397 |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
398 |
(copied from a recent version of bzr.dev for easier reading, refer to the |
399 |
original for an up to date version) |
|
400 |
||
401 |
The Option object is used to define its properties: |
|
402 |
||
403 |
* name: a name: a valid python identifier (even if it's not used as an |
|
404 |
identifier in python itself). This is also used to register the option. |
|
405 |
||
406 |
* default: the default value that Stack.get() should return if no |
|
407 |
value can be found for the option. |
|
408 |
||
409 |
* default_from_env: a list of environment variables. The first variable set |
|
410 |
will provide a default value overriding 'default' which remains the |
|
411 |
default value if *no* environment variable is set. |
|
412 |
||
413 |
* help: a doc string describing the option, the first line should be a |
|
414 |
summary and can be followed by a blank line and a more detailed |
|
415 |
explanation. |
|
416 |
||
417 |
* from_unicode: a callable accepting a unicode string and returning a |
|
418 |
suitable value for the option. If the string cannot be coerced it should |
|
419 |
return None. |
|
420 |
||
421 |
* invalid: the action to be taken when an invalid value is encountered in a |
|
422 |
store (during a Stack.get()). |
|
423 |
||
424 |
The Section object |
|
425 |
------------------ |
|
426 |
||
427 |
Options are grouped into sections which share some properties with the well |
|
428 |
known dict objects: |
|
429 |
||
430 |
* the key is the name, |
|
431 |
* you can get, set and remove an option, |
|
432 |
* the value is a unicode string. |
|
433 |
||
434 |
MutableSection is needed to set or remove an option, ReadOnlySection should |
|
435 |
be used otherwise. |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
436 |
|
48
by Vincent Ladeuil
Some updates to reflect the parts already landed. |
437 |
The Store object |
438 |
---------------- |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
439 |
|
440 |
This is an implementation-level object that should rarely be used directly. |
|
441 |
||
442 |
* it can be local or remote |
|
443 |
||
444 |
* locking |
|
445 |
||
48
by Vincent Ladeuil
Some updates to reflect the parts already landed. |
446 |
All lock operations should be implemented via transport objects. (True for |
447 |
Store). |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
448 |
|
449 |
* option life cycle |
|
450 |
||
451 |
Working trees, branches and repositories should define a config attribute |
|
452 |
following the same life cycle as their lock: the associated config file is |
|
453 |
read once and written once if needed. This should minimize the file system |
|
454 |
accesses or the network requests. There is no known racing scenarios for |
|
455 |
configuration options, changing the existing implementation to this less |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
456 |
constrained one shouldn't introduce any. Yet, in order to detect such |
457 |
racing scenarios, we can add a check that the current content of the |
|
458 |
configuration file is the expected one before writing the new content and |
|
459 |
emit warnings if differences occur. The checks should be performed for the |
|
460 |
modified values only. As of today (and in the foreseeable future), the |
|
461 |
size of the configuration files are small enough to be kept in memory (see |
|
462 |
bug #832042). |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
463 |
|
49
by Vincent Ladeuil
Some more bits from various discussions. |
464 |
The Stack object |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
465 |
----------------- |
466 |
||
467 |
This the object that provides access to the needed features: |
|
468 |
||
469 |
* getting an option value, |
|
470 |
||
471 |
* setting an option value, |
|
472 |
||
473 |
* deleting an option value, |
|
474 |
||
62
by Vincent Ladeuil
Update to better match actual progress. |
475 |
* handling a list of configuration files and for each of them a section |
476 |
matcher providing the sections that should be tried in the given order to |
|
477 |
find an option. |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
478 |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
479 |
* handling a Store and a section where option creation, modification and |
49
by Vincent Ladeuil
Some more bits from various discussions. |
480 |
deletion will occur. |
481 |
||
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
482 |
Depending on the files involved, a working tree, branch or repository object |
62
by Vincent Ladeuil
Update to better match actual progress. |
483 |
(or more generally a context) should be provided to access the corresponding |
484 |
configuration files. Note that providing a working tree object also |
|
485 |
implicitly provides the associated branch and repository object so only one |
|
486 |
of them is required (or none for configuration files specific to the user |
|
487 |
like ``bazaar.conf``). |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
488 |
|
489 |
Getting an option value |
|
490 |
~~~~~~~~~~~~~~~~~~~~~~~ |
|
491 |
||
492 |
Depending on the option, there are various places where it can be defined |
|
493 |
and several ways to override these settings when needed. |
|
494 |
||
495 |
The following lists all possible places where a configuration option can |
|
496 |
be defined, but some options will make sense in only some of them. The |
|
497 |
first to define a value for an option wins (None is therefore used to |
|
498 |
express that an option is not set). |
|
499 |
||
500 |
* command-line (Not Implemented Yet) |
|
501 |
``-Ooption=value`` see bug #491196. |
|
502 |
||
62
by Vincent Ladeuil
Update to better match actual progress. |
503 |
* ``~/.bazaar/locations.conf`` |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
504 |
|
505 |
When an option is set in ``locations.conf`` it overrides any other |
|
506 |
configuration file. This should be used with care as it allows setting a |
|
507 |
different value than what is recommended by the project |
|
508 |
||
62
by Vincent Ladeuil
Update to better match actual progress. |
509 |
* ``tree`` (Not Implemented Yet) |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
510 |
|
511 |
The options related to the working tree. |
|
512 |
||
513 |
This includes all options related to commits, ignored files, junk files, |
|
514 |
etc. |
|
515 |
||
516 |
Note that the sections defined there can use relative paths if some |
|
517 |
options should apply to a subtree or some specific files only. |
|
518 |
||
519 |
See bug #430538 and bug #654998. |
|
520 |
||
49
by Vincent Ladeuil
Some more bits from various discussions. |
521 |
* ``branch`` located in ``.bzr/branch/branch.conf`` |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
522 |
|
523 |
The options related to the branch. |
|
524 |
||
47
by Vincent Ladeuil
Fix spelling mistakes. |
525 |
Sections can be defined for colocated branches or loom threads. |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
526 |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
527 |
* ``repository`` (Not Implemented Yet) |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
528 |
|
529 |
The options related to the repository. |
|
530 |
||
531 |
Using an option to describe whether or not a repository is shared could |
|
532 |
help address bug #342119 but this will probably requires a format bump). |
|
533 |
||
62
by Vincent Ladeuil
Update to better match actual progress. |
534 |
* ``project`` (Not Implemented Yet) |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
535 |
|
536 |
The options common to all branches and working trees for a project. |
|
537 |
||
62
by Vincent Ladeuil
Update to better match actual progress. |
538 |
* ``organization`` (Not Implemented Yet) |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
539 |
|
540 |
The options common to all branches and working trees for an organization. |
|
541 |
||
542 |
See bug #419854. |
|
543 |
||
62
by Vincent Ladeuil
Update to better match actual progress. |
544 |
* ``system`` (Not Implemented Yet but see bug #419854 and |
545 |
https://code.launchpad.net/~thomir/bzr/add-global-config/+merge/69592) |
|
49
by Vincent Ladeuil
Some more bits from various discussions. |
546 |
|
547 |
The options common to all users of a system (may be /etc/bzr/defaults |
|
548 |
or /usr/local/etc/bzr/defaults or |
|
50
by Vincent Ladeuil
Merge changes resolving conflicts |
549 |
/Library/Preferences/com.canonical.defaults or c:\windows\bazaar.conf |
550 |
(someone fix this one please ;) depending on the OS). |
|
49
by Vincent Ladeuil
Some more bits from various discussions. |
551 |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
552 |
* ``bazaar.conf`` |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
553 |
|
554 |
The options the user has selected for the host he is using. |
|
555 |
||
556 |
Sections can be defined for both remote and local branches to define |
|
557 |
default values (i.e. the most common use of ``locations.conf`` today). |
|
558 |
||
62
by Vincent Ladeuil
Update to better match actual progress. |
559 |
* default (implemented by the OptionRegistry) |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
560 |
|
561 |
The options defined in the ``bzr`` source code. |
|
562 |
||
49
by Vincent Ladeuil
Some more bits from various discussions. |
563 |
This will be implemented via the Option objects. |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
564 |
|
565 |
Plugins can define additional configuration files as they see fit and |
|
566 |
insert them in this list, see their documentation for details. |
|
567 |
||
568 |
Compatibility |
|
569 |
============= |
|
570 |
||
51
by Vincent Ladeuil
Clarify the know problems about compatibility. |
571 |
There are ways to keep the same files while ensuring compatibility via various |
572 |
tricks but there are cases where using new files to replace the old ones is |
|
573 |
definitely easier: |
|
574 |
||
575 |
* no need to ensure that the new files are correctly handled by old bzr |
|
576 |
versions, |
|
577 |
||
578 |
* making it clear for users that there is a switch and let them migrate at |
|
579 |
their own pace. |
|
580 |
||
581 |
The known cases so far are described below. |
|
582 |
||
583 |
Obvious at this point: |
|
584 |
||
63
by Vincent Ladeuil
Mention that the two config designs co-exist. |
585 |
* Branch provides ``get_config`` for the old design and ``get_config_stack`` |
586 |
for the new design so that both designs are supported. Once the store |
|
587 |
sharing is implemented, we may want to use an attribute for the stack and |
|
588 |
deprecate both ``get_config`` and ``get_config_stack``. |
|
589 |
||
62
by Vincent Ladeuil
Update to better match actual progress. |
590 |
* Sections names in ``bazaar.conf`` are arbitrary (except ``DEFAULT``) so |
591 |
it's easier to leave the file untouched and let plugin authors and users |
|
592 |
migrate away (or not) from them. For ``bzr`` itself, that means |
|
593 |
``DEFAULT`` is the only section used for most of the options and provides |
|
594 |
user defaults. ``ALIASES`` requires a specific stack but only the ``bzr |
|
595 |
alias`` command cares about that. |
|
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
596 |
|
597 |
* Option policies should be deprecated: |
|
598 |
||
599 |
* The ``norecurse`` policy is useless, all options are recursive by |
|
600 |
default. If specific values are needed for specific paths, they can just |
|
51
by Vincent Ladeuil
Clarify the know problems about compatibility. |
601 |
be defined as such (in the appropriate sections or files). |
40
by Vincent Ladeuil
Add wip about a new configuration implementation. |
602 |
|
603 |
* The ``appendpath`` policy should be implemented via interpolation and a |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
604 |
``relpath`` option provided by the configuration framework (bug |
605 |
#832013). |
|
42
by Vincent Ladeuil
More design ideas. |
606 |
|
51
by Vincent Ladeuil
Clarify the know problems about compatibility. |
607 |
* Section order in ``locations.conf`` has issues which make a migration to a |
608 |
different way to organize the sections (hence the file content) far easier |
|
609 |
with a new file. |
|
610 |
||
611 |
* ``locations.conf`` is really for overrides but many users have been using it |
|
612 |
to provide defaults. There is no way to know if the whole content has been |
|
53
by Vincent Ladeuil
Fix typo |
613 |
used for defaults or overrides or a mix of both. So there is no way to |
51
by Vincent Ladeuil
Clarify the know problems about compatibility. |
614 |
migrate this automatically. |
615 |
||
616 |
Unclear at this point: |
|
617 |
||
618 |
* [BOOKMARKS] section can be replaced by ``bookmarks.xxx`` options (the |
|
619 |
bookmarks plugins already uses ``bookmarks_xxx`` in branch.conf since no |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
620 |
sections were supported there). The easiest here is probably to just merge |
621 |
the plugin into core and use the appropriate option names consistently. A |
|
622 |
``config:`` directory service may even be better as any option can be used |
|
623 |
as a bookmark. This allows things like:: |
|
52
by Vincent Ladeuil
One more example |
624 |
|
625 |
[/whatever/path] |
|
626 |
my_push = lp:<launchpad.login>/xxx/{nick} |
|
627 |
web_site=ftp://example.com/ |
|
628 |
||
629 |
bzr push config:web_site |
|
630 |
||
631 |
Which means we completely replace the plugin and don't need to care about |
|
632 |
migrating the section. |
|
51
by Vincent Ladeuil
Clarify the know problems about compatibility. |
633 |
|
634 |
* [ALIASES] section can be replaced by corresponding bzr.alias.xxx |
|
635 |
options. This could be automated by creating the corresponding options ? |
|
636 |
||
637 |
* I don't know about other sections, feedback welcome. Plugin authors are |
|
638 |
encouraged to migrate to the new name space scheme by prefixing their |
|
639 |
options with their plugin name. |
|
640 |
||
42
by Vincent Ladeuil
More design ideas. |
641 |
Notes |
642 |
===== |
|
643 |
||
62
by Vincent Ladeuil
Update to better match actual progress. |
644 |
These are random notes about concepts, ideas or issues not implemented yet. |
42
by Vincent Ladeuil
More design ideas. |
645 |
|
646 |
Developer facing concepts |
|
647 |
------------------------- |
|
648 |
||
649 |
Option |
|
650 |
~~~~~~ |
|
50
by Vincent Ladeuil
Merge changes resolving conflicts |
651 |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
652 |
* list of allowed Config IDs (this allows a list of possible config files in |
653 |
bazaar.conf only option and use it while bootstrapping the config |
|
654 |
creations). |
|
50
by Vincent Ladeuil
Merge changes resolving conflicts |
655 |
|
42
by Vincent Ladeuil
More design ideas. |
656 |
* blacklist of config IDs (some options *can't* be stored (modified) by the |
657 |
user) |
|
658 |
||
62
by Vincent Ladeuil
Update to better match actual progress. |
659 |
An alternative is to just let the devs decide which stack they use for a |
660 |
given option, ``stacked_on_location`` for example is said to relate to the |
|
661 |
branch only and changing it or setting it in a different config file may not |
|
662 |
be appropriate. This may not be a good example as there is also the |
|
663 |
``default_stack_on`` option which can be set only in ``control.conf`` |
|
664 |
though... |
|
42
by Vincent Ladeuil
More design ideas. |
665 |
|
666 |
Stack |
|
667 |
~~~~~ |
|
46
by Vincent Ladeuil
More rst fixes. |
668 |
|
42
by Vincent Ladeuil
More design ideas. |
669 |
* a lazy cache for the option values (should be reset on modifications as |
62
by Vincent Ladeuil
Update to better match actual progress. |
670 |
interpolations will make it tricky to update incrementally) (see FIXME in |
671 |
config.py Stack.get())) |
|
42
by Vincent Ladeuil
More design ideas. |
672 |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
673 |
* ensures that the Stores involved generate as less IOs as possible (see bug |
674 |
#832042) |
|
42
by Vincent Ladeuil
More design ideas. |
675 |
|
676 |
* ensures that the transaction is the object life time (i.e. modifications |
|
44
by Vincent Ladeuil
Fix more typos ;-/ |
677 |
will be taken into account *iff* they are committed explicitly). |
42
by Vincent Ladeuil
More design ideas. |
678 |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
679 |
StackRegistry |
680 |
~~~~~~~~~~~~~ |
|
42
by Vincent Ladeuil
More design ideas. |
681 |
|
682 |
* ensures that a config ID is a unique identifier |
|
683 |
* register Stacks |
|
684 |
||
685 |
Store |
|
686 |
~~~~~ |
|
687 |
||
688 |
* ensures that the transaction is the object life time (i.e. modifications |
|
44
by Vincent Ladeuil
Fix more typos ;-/ |
689 |
will be taken into account *iff* they are committed explicitly). |
42
by Vincent Ladeuil
More design ideas. |
690 |
|
691 |
Examples |
|
692 |
-------- |
|
693 |
||
694 |
store examples: |
|
695 |
~~~~~~~~~~~~~~~ |
|
696 |
||
697 |
* ConfigObj (bazaar.conf) |
|
698 |
||
699 |
* DB (<scheme>://bazaar.launchpad.net/bazaar.conf) |
|
700 |
||
50
by Vincent Ladeuil
Merge changes resolving conflicts |
701 |
|
702 |
Why and when locking config files matter |
|
703 |
---------------------------------------- |
|
704 |
||
62
by Vincent Ladeuil
Update to better match actual progress. |
705 |
This is relevant for bug #832042. |
706 |
||
707 |
``bzr`` behavior, as well as the objects it acts upon, is configured via a |
|
708 |
set of so-called configuration files. |
|
709 |
||
710 |
These files allow to define working trees, branches and repositories, their |
|
711 |
relationships and how ``bzr`` should handle them. |
|
712 |
||
713 |
The default behavior of ``bzr`` is aimed at making this configuration as |
|
714 |
transparent as possible by keeping track of how these objects are created |
|
715 |
and modified when they are used. In short, they are useless until you want |
|
716 |
to change the default behavior in some specific context. |
|
717 |
||
718 |
We mostly **read** config options. Therefore all we care about is to |
|
719 |
guarantee that: |
|
50
by Vincent Ladeuil
Merge changes resolving conflicts |
720 |
|
721 |
* we get a valid config file at all times when reading, |
|
722 |
||
723 |
* we always leave a valid config file when writing (via the rename dance) |
|
724 |
||
62
by Vincent Ladeuil
Update to better match actual progress. |
725 |
From there, conceptually, all operations can clearly define whether or not |
726 |
they need to modify a config file and do so only when they succeed. All |
|
727 |
modifications occurring during such an operation are delayed until the very |
|
728 |
end of the operation. |
|
50
by Vincent Ladeuil
Merge changes resolving conflicts |
729 |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
730 |
Now, we want to minimize the overlapping times where one bzr operation has |
731 |
changed a value and another concurrent operation is unaware of this |
|
732 |
modification. |
|
50
by Vincent Ladeuil
Merge changes resolving conflicts |
733 |
|
734 |
These overlapping periods are *as of today* rare. |
|
735 |
||
62
by Vincent Ladeuil
Update to better match actual progress. |
736 |
The only known case, bug #525571 has been fixed in bzr-2.1.3. The bug there |
737 |
was triggered when two processes tried to write the same config file at the |
|
738 |
same time leaving an invalid file in the end. |
|
739 |
||
740 |
Such a period can be recognized and detected though: when changing an option |
|
741 |
value, if the preserved original value is different in the config file, |
|
742 |
someone else modified it and the operation can be invalid because it relied |
|
743 |
on the original value. |
|
744 |
||
745 |
For the sake of the example, if an option value represent a global unique ID |
|
746 |
via a simple counter (very bad idea), if two operations try to increment it, |
|
747 |
both will use the same value that won't be unique anymore. Checking the |
|
748 |
value present in the file when trying to save the updated value with |
|
749 |
identify such a collision. |
|
750 |
||
751 |
An assumption is floating around: it should be enough to report when an |
|
752 |
operation is modifying an already modified option and observe that no-one |
|
753 |
reports such occurrences. |
|
754 |
||
755 |
Note that this assumption is made in a context where *no* known scenarios |
|
756 |
exist in the bzr code base not in any plugin (for a best effort value of |
|
757 |
'any', feedback highly welcome, bug reports even ;) |
|
758 |
||
759 |
With this in mind, we can change the definition of config options, stores |
|
760 |
and stacks to ensure that: |
|
50
by Vincent Ladeuil
Merge changes resolving conflicts |
761 |
|
762 |
* a config file is read only once when in read access, |
|
763 |
||
62
by Vincent Ladeuil
Update to better match actual progress. |
764 |
* a config file is read only once and written only once when in write |
765 |
access, adding the check mentioned above will require *one* additional |
|
766 |
read. |
|
50
by Vincent Ladeuil
Merge changes resolving conflicts |
767 |
|
62
by Vincent Ladeuil
Update to better match actual progress. |
768 |
A reader can then safely assume that reading a config file gives it a valid |
769 |
(and coherent) definition of the configuration when the operation |
|
770 |
starts. All the operation has to do is to declare which config files may be |
|
771 |
modified by an operation (whether or not we can be liberal on this 'may be' |
|
772 |
is yet to be defined). |