« back to all changes in this revision
Viewing changes to docs/tutorial-part-2.rst
- browse files at revision 109
- compare with another revision
- download diff
- download tarball
- view history from revision 109
- Committer: Tarmac
- Author(s): Wes Mason
- Date: 2015-06-17 09:41:42 UTC
- mfrom: (101.2.59 tutorial)
- Revision ID: tarmac-20150617094142-pixwow6tgkw104ut
[r=wesmason] Add sphinx/rtd docs with a basic tutorial
- files added:
- docs
- docs/static
- files modified:
- .bzrignore
added
removed
docs/tutorial-part-2.rst
1
Tutorial Part 2: Auto-generating conn-check config for a Django app
2
===================================================================
3
4
Hello World (again)
5
-------------------
6
7
Let's assume that you've actually created the ``Hello World`` service from
8
:doc:`part 1 </tutorial-part-1>` as a
9
`Django app <https://www.djangoproject.com/>`_, and you think to yourself:
10
11
*"Hang on, aren't all these connections I want conn-check to check for me
12
already defined in my Django settings module?"*
13
14
conn-check-configs
15
------------------
16
17
Yes, yes they are, and with the handy-dandy
18
`conn-check-configs <https://pypi.python.org/pypi/conn-check-configs>`_
19
package you can automatically generate conn-check config YAML from a range of
20
standard Django settings values (in theory from other environments
21
too, such as `Juju <https://jujucharms.com/>`_, but for now just Django).
22
23
exempli gratia
24
--------------
25
26
Given the following ``settings.py`` in our *HWaaS* app:
27
28
.. code-block:: python
29
30
INSTALLED_APPS = [
31
'hwaas'
32
]
33
DATABASES = {
34
'default': {
35
'ENGINE': 'django.db.backends.postgresql_psycopg2',
36
'HOST': 'gibson.hwass.internal',
37
'NAME': 'hwaas_production',
38
'PASSWORD': '123456asdf',
39
'PORT': 11211,
40
'USER': 'hwaas',
41
}
42
CACHES = {
43
'default': {
44
'LOCATION': 'freeside.hwaas.internal:11211',
45
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
46
},
47
}
48
PROXY_HOST = 'countzero.hwaas.internal'
49
PROXY_PORT = 8080
50
TRANSLATE_API_KEY = 'BLAH'
51
52
We can create a ``settings-to-conn-check.py`` script with the least possible
53
effort like so:
54
55
.. code-block:: python
56
57
#!/usr/bin/env python
58
from conn_check_configs.django import run
59
60
if __name__ == '__main__':
61
run()
62
63
This will output *postgresql* and *memcached* checks to similar our
64
hand-written config:
65
66
.. code-block:: sh
67
68
$ chmod +x settings-to-conn-check.py
69
$ ./settings-to-conn-check.py -f cc-config.yaml -m hwaas.settings
70
$ cat cc-config.yaml
71
72
.. code-block:: yaml
73
74
- type: postgresql
75
database: hwaas_production
76
host: gibson.hwaas.internal
77
port: 5432
78
username: hwaas
79
password: 123456asdf
80
- type: memcached
81
host: freeside.hwaas.internal
82
port: 11211
83
84
Customising generated checks
85
----------------------------
86
87
In order to generate the checks we need for Squid / Google Translate API, we
88
can add some custom callbacks:
89
90
.. code-block:: python
91
92
#!/usr/bin/env python
93
from conn_check_configs.django import run, EXTRA_CHECK_MAKERS
94
95
96
def make_proxied_translate_check(settings, options):
97
checks = []
98
if settings['PROXY_HOST']:
99
checks.append({
100
'type': 'http',
101
'url': 'https://www.googleapis.com/language/translate/v2?q='
102
'Hello%20World&target=de&source=en&key={}'.format(
103
settings['TRANSLATE_API_KEY']),
104
'proxy_host': settings['PROXY_HOST'],
105
'proxy_port': int(settings.get('PROXY_PORT', 8080)),
106
'expected_code': 200,
107
})
108
return checks
109
110
EXTRA_CHECK_MAKERS.append(make_proxied_translate_check)
111
112
113
if __name__ == '__main__':
114
run()
115
116
117
In the above we define a callable which takes 2 params, ``settings`` which
118
is a wrapper around the Django settings module, and ``options`` which is
119
an object containing the command line arguments that were passed to the script.
120
121
The ``settings`` module is not the direct settings module but a dict-like
122
wrapper so that you can access the settings just a like a dict (using indices,
123
``.get`` method, etc.)
124
125
To ensure ``make_proxied_translate_check`` is collected and called by the main
126
``run`` function we add it to the ``EXTRA_CHECK_MAKERS`` list.
127
128
The above generates our required HTTP check:
129
130
.. code-block:: yaml
131
132
- type: http
133
url: https://www.googleapis.com/language/translate/v2?q=Hello%20World&target=de&source=en&key=BLAH
134
proxy_host: countzero.hwaas.internal
135
proxy_port: 8080
136
expected_code: 200
137
138
A note on statstd checks
139
------------------------
140
141
Getting more operational visibility on how *HWaaS* runs would be great,
142
wouldn't it?
143
144
So let's add some metrics collection using
145
`StatsD <https://github.com/etsy/statsd>`_, and as luck would have it we can
146
get a lot for *nearly free* with the
147
`django-statsd <https://django-statsd.readthedocs.org/>`_, after adding it to
148
our dependencies we update our ``settings.py`` to include:
149
150
.. code-block:: python
151
152
INSTALLED_APPS = [
153
'hwaas'
154
'django_statsd',
155
]
156
MIDDLEWARE_CLASSES = [
157
'django_statsd.middleware.GraphiteMiddleware',
158
]
159
STATSD_CLIENT = 'django_statsd.clients.normal'
160
STATSD_HOST = 'bigend.hwaas.internal'
161
STATSD_PORT = 10021
162
163
**Note**: You don't actually need the django-statsd app to have
164
conn-check-configs generate statsd checks, only the use of ``STATSD_HOST``
165
and ``STATSD_PORT`` in your settings module matters.
166
167
Another run of our ``settings-to-conn-check.py`` script will result in the
168
extra statsd check:
169
170
.. code-block:: yaml
171
172
- type: udp
173
host: bigend.hwaas.internal
174
port: 10021
175
send: conncheck.test:1|c
176
expect:
177
178
As you can see this is just a generic UDP check that attempts to send an
179
incremental counter metric to the statsd host.
180
181
Unfortunately the fire-and-forget nature of this use of statsd/UDP will not
182
error in a number of common situations (the simplest being that statsd is not
183
running on the target host, or even a routing issue along the way).
184
185
It will catch simple problems such as not being able to open up the local UDP
186
port to send from, but that's usually not enough.
187
188
If you use a third-party implementation of statsd, such as
189
`txStatsD <https://launchpad.net/txstatsd>`_ then you might have the ability
190
to define a pair of health check strings, for example by changing the send
191
and expect values in the ``STATSD_CHECK`` dict we can send and expect different
192
strings:
193
194
.. code-block:: python
195
196
#!/usr/bin/env python
197
from conn_check_configs.django import run, STATSD_CHECK
198
199
STATSD_CHECK['send'] = 'Hakuna'
200
STATSD_CHECK['expect'] = 'Matata'
201
202
if __name__ == '__main__':
203
run()
204
205
Which generates this check:
206
207
.. code-block:: yaml
208
209
- type: udp
210
host: bigend.hwaas.internal
211
port: 10021
212
send: Hakuna
213
expect: Matata
214
215
In the above we would configure our txStatD (for example) instance to respond
216
to the string ``Hakuna`` with the string ``Matata``, which would catch pretty
217
much all the possible issues with contacting our metrics service.
Loggerhead is a web-based interface for Breezy
Version: 2.0.1