1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
|
**************************************************
python-oops-amqp: Transmit error reports over amqp
**************************************************
Copyright (c) 2011, Canonical Ltd
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Lesser General Public License as published by
the Free Software Foundation, version 3 only.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
GNU Lesser General Public License version 3 (see the file LICENSE).
The oops_amqp package provides an AMQP OOPS http://pypi.python.org/pypi/oops)
publisher, and a small daemon that listens on amqp for OOPS reports and
republishes them (into a supplied publisher). The OOPS framework permits
falling back to additional publishers if AMQP is down.
Dependencies
============
* Python 2.6+
* bson
* oops (http://pypi.python.org/pypi/oops) 0.0.11 or newer.
* amqplib
Testing Dependencies
====================
* oops-datedir-repo (http://pypi.python.org/pypi/oops_datedir_repo)
* rabbitfixture (http://pypi.python.org/pypi/rabbitfixture)
* subunit (http://pypi.python.org/pypi/python-subunit) (optional)
* testresources (http://pypi.python.org/pypi/testresources)
* testtools (http://pypi.python.org/pypi/testtools)
Usage
=====
Publishing to AMQP
++++++++++++++++++
Where you are creating OOPS reports, configure oops_amqp.Publisher. This takes
a connection factory - a simple callable that creates an amqp
connection - and the exchange name and routing key to submit to.
>>> factory = partial(amqp.Connection, host="localhost:5672",
... userid="guest", password="guest", virtual_host="/", insist=False)
>>> publisher = oops_amqp.Publisher(factory, "oopses", "")
Provide the publisher to your OOPS config::
>>> config = oops.Config()
>>> config.publisher = publisher
Any oops published via that config will now be sent via amqp.
OOPS ids are generating by hashing the oops message (without the id field) -
this ensures unique ids.
The reason a factory is used is because amqplib is not threadsafe - the
publisher maintains a thread locals object to hold the factories and creates
connections when new threads are created(when they first generate an OOPS).
Dealing with downtime
---------------------
From time to time your AMQP server may be unavailable. If that happens then
the Publisher will not assign an oops id - it will return None to signal that
the publication failed. To prevent losing the OOPS its a good idea to have a
fallback publisher - either another AMQP publisher (to a different server) or
one that spools locally (where you can pick up the OOPSes via rsync or some
other mechanism. Using the oops standard helper publish_with_fallback will let
you wrap the fallback publisher so that it only gets invoked if the primary
method failed::
>>> fallback_factory = partial(amqp.Connection, host="otherserver:5672",
... userid="guest", password="guest", virtual_host="/", insist=False)
>>> fallback_publisher = oops_amqp.Publisher(fallback_factory, "oopses", "")
>>> config.publisher = publish_with_fallback(publisher, fallback_publisher)
Receiving from AMQP
+++++++++++++++++++
There is a simple method that will run an infinite loop processing reports from
AMQP. To use it you need to configure a local config to publish the received
reports. A full config is used because that includes support for filtering
(which can be useful if you need to throttle volume, for instance).
Additionally you need an amqp connection factory (to handle the amqp server
being restarted) and a queue name to receive from.
This example uses the DateDirRepo publisher, telling it to accept whatever
id was assigned by the process publishing to AMQP::
>>> publisher = oops_datedir_repo.DateDirRepo('.', inherit_id=True)
>>> config = oops.Config()
>>> config.publisher = publisher.publish
>>> receiver = oops_amqp.Receiver(config, factory, "my queue")
>>> receiver.run_forever()
For more information see pydoc oops_amqp.
Installation
============
Either run setup.py in an environment with all the dependencies available, or
add the working directory to your PYTHONPATH.
Development
===========
Upstream development takes place at https://launchpad.net/python-oops-amqp.
To setup a working area for development, if the dependencies are not
immediately available, you can use ./bootstrap.py to create bin/buildout, then
bin/py to get a python interpreter with the dependencies available.
To run the tests use the runner of your choice, the test suite is
oops_amqp.tests.test_suite.
For instance::
$ bin/py -m testtools.run oops_amqp.tests.test_suite
If you have testrepository you can run the tests with that::
$ testr run
|