1
.. PyZMQ Version compatibility doc, by Min Ragan-Kelley, 2010
5
PyZMQ, Python2.5, and Python3
6
=============================
8
PyZMQ is a fairly light, low-level library, so supporting as many versions
9
as is reasonable is our goal. Currently, we support at least Python 2.5-3.1.
10
Making the changes to the codebase required a few tricks, which are documented here
11
for future reference, either by us or by other developers looking to support several
16
It is far simpler to support 2.6-3.x than to include 2.5. Many of the significant
17
syntax changes have been backported to 2.6, so just writing new-style code would work
18
in many cases. I will try to note these points as they come up.
24
Many functions we use, primarily involved in converting between C-buffers and Python
25
objects, are not available on all supported versions of Python. In order to resolve
26
missing symbols, we added a header :file:`utils/pyversion_compat.h` that defines missing
27
symbols with macros. Some of these macros alias new names to old functions (e.g.
28
``PyBytes_AsString``), so that we can call new-style functions on older versions, and some
29
simply define the function as an empty exception raiser. The important thing is that the
30
symbols are defined to prevent compiler warnings and linking errors. Everywhere we use
31
C-API functions that may not be available in a supported version, at the top of the file
36
cdef extern from "pyversion_compat.h":
39
This ensures that the symbols are defined in the Cython generated C-code. Higher level
40
switching logic exists in the code itself, to prevent actually calling unavailable
41
functions, but the symbols must still be defined.
48
If you are using Python >= 2.6, to prepare your PyZMQ code for Python3 you should use
49
the ``b'message'`` syntax to ensure all your string literal messages will still be
50
:class:`bytes` after you make the upgrade.
52
The most cumbersome part of PyZMQ compatibility from a user's perspective is the fact
53
that, since ØMQ uses C-strings, and would like to do so without copying, we must use the
54
Py3k :class:`bytes` object, which is backported to 2.6. In order to do this in a
55
Python-version independent way, we added a small utility that unambiguously defines the
56
string types: :class:`bytes`, :class:`unicode`, :obj:`basestring`. This is important,
57
because :class:`str` means different things on 2.x and 3.x, and :class:`bytes` is
58
undefined on 2.5, and both :class:`unicode` and :obj:`basestring` are undefined on 3.x.
59
All typechecking in PyZMQ is done against these types:
61
================= ================= ====================
63
================= ================= ====================
64
:obj:`bytes` :obj:`str` :obj:`bytes`
65
:obj:`unicode` :obj:`unicode` :obj:`str`
66
:obj:`basestring` :obj:`basestring` :obj:`(str, bytes)`
67
================= ================= ====================
73
Where we really noticed the issue of :class:`bytes` vs :obj:`strings` coming up for
74
users was in updating the tests to run on every version. Since the ``b'bytes
75
literal'`` syntax was not backported to 2.5, we must call ``"message".encode()`` for
76
*every* string in the test suite.
78
.. seealso:: :ref:`Unicode discussion <unicode>` for more information on strings/bytes.
83
The standard C-API function for turning a C-string into a Python string was a set of
84
functions with the prefix ``PyString_*``. However, with the Unicode changes made in
85
Python3, this was broken into ``PyBytes_*`` for bytes objects and ``PyUnicode_*`` for
86
unicode objects. We changed all our ``PyString_*`` code to ``PyBytes_*``, which was
94
Since Python 2.5 doesn't support the ``PyBytes_*`` functions, we had to alias them to
95
the ``PyString_*`` methods in utils/pyversion_compat.h.
99
#define PyBytes_FromStringAndSize PyString_FromStringAndSize
100
#define PyBytes_FromString PyString_FromString
101
#define PyBytes_AsString PyString_AsString
102
#define PyBytes_Size PyString_Size
107
The layer that is most complicated for developers, but shouldn't trouble users, is the
108
Python C-Buffer APIs. These are the methods for converting between Python objects and C
109
buffers. The reason it is complicated is that it keeps changing.
111
There are two buffer interfaces for converting an object to a C-buffer, known as new-style
112
and old-style. Old-style buffers were introduced long ago, but the new-style is only
113
backported to 2.6. The old-style buffer interface is not available in 3.x. There is also
114
an old- and new-style interface for creating Python objects that view C-memory. The
115
old-style object is called a :class:`buffer`, and the new-style object is
116
:class:`memoryview`. Unlike the new-style buffer interface for objects,
117
:class:`memoryview` has only been backported to *2.7*. This means that the available
118
buffer-related functions are not the same in any two versions of Python 2.5, 2.6, 2.7, or
121
We have a :file:`utils/buffers.pxd` file that defines our :func:`asbuffer` and
122
:func:`frombuffer` functions. :file:`utils/buffers.pxd` was adapted from mpi4py_'s
123
:file:`asbuffer.pxi`. The :func:`frombuffer` functionality was added. These functions
124
internally switch based on Python version to call the appropriate C-API functions.
126
.. seealso:: `Python Buffer API <bufferapi>`_
128
.. _bufferapi: http://docs.python.org/c-api/buffer.html
134
As discussed, :class:`str` is not a platform independent type. The two places where we are
135
required to return native str objects are :func:`error.strerror`, and
136
:func:`Message.__str__`. In both of these cases, the natural return is actually a
137
:class:`bytes` object. In the methods, the native :class:`str` type is checked, and if the
138
native str is actually unicode, then we decode the bytes into unicode:
154
This section is only relevant for supporting Python 2.5 and 3.x, not for 2.6-3.x.
156
The syntax for handling exceptions has `changed <PEP-3110>`_ in Python 3. The old syntax:
162
except zmq.ZMQError, e:
165
is no longer valid in Python 3. Instead, the new syntax for this is:
171
except zmq.ZMQError as e:
174
This new syntax is backported to Python 2.6, but is invalid on 2.5. For 2.6-3.x compatible
175
code, we could just use the new syntax. However, the only method we found to catch an
176
exception for handling on both 2.5 and 3.1 is to get the exception object inside the
184
e = sys.exc_info()[1]
187
This is certainly not as elegant as either the old or new syntax, but it's the only way we
188
have found to work everywhere.
190
.. seealso:: PEP-3110_
192
.. _PEP-3110: http://www.python.org/dev/peps/pep-3110/
195
.. _mpi4py: http://mpi4py.googlecode.com
b'\\ No newline at end of file'
added
removed
docs/source/pyversions.rst
