2
:mod:`socketserver` --- A framework for network servers
3
=======================================================
5
.. module:: socketserver
6
:synopsis: A framework for network servers.
8
The :mod:`socketserver` module simplifies the task of writing network servers.
10
There are four basic server classes: :class:`TCPServer` uses the Internet TCP
11
protocol, which provides for continuous streams of data between the client and
12
server. :class:`UDPServer` uses datagrams, which are discrete packets of
13
information that may arrive out of order or be lost while in transit. The more
14
infrequently used :class:`UnixStreamServer` and :class:`UnixDatagramServer`
15
classes are similar, but use Unix domain sockets; they're not available on
16
non-Unix platforms. For more details on network programming, consult a book
18
W. Richard Steven's UNIX Network Programming or Ralph Davis's Win32 Network
21
These four classes process requests :dfn:`synchronously`; each request must be
22
completed before the next request can be started. This isn't suitable if each
23
request takes a long time to complete, because it requires a lot of computation,
24
or because it returns a lot of data which the client is slow to process. The
25
solution is to create a separate process or thread to handle each request; the
26
:class:`ForkingMixIn` and :class:`ThreadingMixIn` mix-in classes can be used to
27
support asynchronous behaviour.
29
Creating a server requires several steps. First, you must create a request
30
handler class by subclassing the :class:`BaseRequestHandler` class and
31
overriding its :meth:`handle` method; this method will process incoming
32
requests. Second, you must instantiate one of the server classes, passing it
33
the server's address and the request handler class. Finally, call the
34
:meth:`handle_request` or :meth:`serve_forever` method of the server object to
35
process one or many requests.
37
When inheriting from :class:`ThreadingMixIn` for threaded connection behavior,
38
you should explicitly declare how you want your threads to behave on an abrupt
39
shutdown. The :class:`ThreadingMixIn` class defines an attribute
40
*daemon_threads*, which indicates whether or not the server should wait for
41
thread termination. You should set the flag explicitly if you would like threads
42
to behave autonomously; the default is :const:`False`, meaning that Python will
43
not exit until all threads created by :class:`ThreadingMixIn` have exited.
45
Server classes have the same external methods and attributes, no matter what
46
network protocol they use.
52
There are five classes in an inheritance diagram, four of which represent
53
synchronous servers of four types::
60
+-----------+ +------------------+
61
| TCPServer |------->| UnixStreamServer |
62
+-----------+ +------------------+
65
+-----------+ +--------------------+
66
| UDPServer |------->| UnixDatagramServer |
67
+-----------+ +--------------------+
69
Note that :class:`UnixDatagramServer` derives from :class:`UDPServer`, not from
70
:class:`UnixStreamServer` --- the only difference between an IP and a Unix
71
stream server is the address family, which is simply repeated in both Unix
74
Forking and threading versions of each type of server can be created using the
75
:class:`ForkingMixIn` and :class:`ThreadingMixIn` mix-in classes. For instance,
76
a threading UDP server class is created as follows::
78
class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass
80
The mix-in class must come first, since it overrides a method defined in
81
:class:`UDPServer`. Setting the various member variables also changes the
82
behavior of the underlying server mechanism.
84
To implement a service, you must derive a class from :class:`BaseRequestHandler`
85
and redefine its :meth:`handle` method. You can then run various versions of
86
the service by combining one of the server classes with your request handler
87
class. The request handler class must be different for datagram or stream
88
services. This can be hidden by using the handler subclasses
89
:class:`StreamRequestHandler` or :class:`DatagramRequestHandler`.
91
Of course, you still have to use your head! For instance, it makes no sense to
92
use a forking server if the service contains state in memory that can be
93
modified by different requests, since the modifications in the child process
94
would never reach the initial state kept in the parent process and passed to
95
each child. In this case, you can use a threading server, but you will probably
96
have to use locks to protect the integrity of the shared data.
98
On the other hand, if you are building an HTTP server where all data is stored
99
externally (for instance, in the file system), a synchronous class will
100
essentially render the service "deaf" while one request is being handled --
101
which may be for a very long time if a client is slow to receive all the data it
102
has requested. Here a threading or forking server is appropriate.
104
In some cases, it may be appropriate to process part of a request synchronously,
105
but to finish processing in a forked child depending on the request data. This
106
can be implemented by using a synchronous server and doing an explicit fork in
107
the request handler class :meth:`handle` method.
109
Another approach to handling multiple simultaneous requests in an environment
110
that supports neither threads nor :func:`fork` (or where these are too expensive
111
or inappropriate for the service) is to maintain an explicit table of partially
112
finished requests and to use :func:`select` to decide which request to work on
113
next (or whether to handle a new incoming request). This is particularly
114
important for stream services where each client can potentially be connected for
115
a long time (if threads or subprocesses cannot be used). See :mod:`asyncore` for
116
another way to manage this.
118
.. XXX should data and methods be intermingled, or separate?
119
how should the distinction between class and instance variables be drawn?
126
.. function:: fileno()
128
Return an integer file descriptor for the socket on which the server is
129
listening. This function is most commonly passed to :func:`select.select`, to
130
allow monitoring multiple servers in the same process.
133
.. function:: handle_request()
135
Process a single request. This function calls the following methods in
136
order: :meth:`get_request`, :meth:`verify_request`, and
137
:meth:`process_request`. If the user-provided :meth:`handle` method of the
138
handler class raises an exception, the server's :meth:`handle_error` method
139
will be called. If no request is received within :attr:`self.timeout`
140
seconds, :meth:`handle_timeout` will be called and :meth:`handle_request`
144
.. function:: serve_forever(poll_interval=0.5)
146
Handle requests until an explicit :meth:`shutdown` request. Polls for
147
shutdown every *poll_interval* seconds.
150
.. function:: shutdown()
152
Tells the :meth:`serve_forever` loop to stop and waits until it does.
155
.. data:: address_family
157
The family of protocols to which the server's socket belongs.
158
Common examples are :const:`socket.AF_INET` and :const:`socket.AF_UNIX`.
161
.. data:: RequestHandlerClass
163
The user-provided request handler class; an instance of this class is created
167
.. data:: server_address
169
The address on which the server is listening. The format of addresses varies
170
depending on the protocol family; see the documentation for the socket module
171
for details. For Internet protocols, this is a tuple containing a string giving
172
the address, and an integer port number: ``('127.0.0.1', 80)``, for example.
177
The socket object on which the server will listen for incoming requests.
179
The server classes support the following class variables:
181
.. XXX should class variables be covered before instance variables, or vice versa?
184
.. data:: allow_reuse_address
186
Whether the server will allow the reuse of an address. This defaults to
187
:const:`False`, and can be set in subclasses to change the policy.
190
.. data:: request_queue_size
192
The size of the request queue. If it takes a long time to process a single
193
request, any requests that arrive while the server is busy are placed into a
194
queue, up to :attr:`request_queue_size` requests. Once the queue is full,
195
further requests from clients will get a "Connection denied" error. The default
196
value is usually 5, but this can be overridden by subclasses.
199
.. data:: socket_type
201
The type of socket used by the server; :const:`socket.SOCK_STREAM` and
202
:const:`socket.SOCK_DGRAM` are two common values.
206
Timeout duration, measured in seconds, or :const:`None` if no timeout is
207
desired. If :meth:`handle_request` receives no incoming requests within the
208
timeout period, the :meth:`handle_timeout` method is called.
210
There are various server methods that can be overridden by subclasses of base
211
server classes like :class:`TCPServer`; these methods aren't useful to external
212
users of the server object.
214
.. XXX should the default implementations of these be documented, or should
215
it be assumed that the user will look at socketserver.py?
218
.. function:: finish_request()
220
Actually processes the request by instantiating :attr:`RequestHandlerClass` and
221
calling its :meth:`handle` method.
224
.. function:: get_request()
226
Must accept a request from the socket, and return a 2-tuple containing the *new*
227
socket object to be used to communicate with the client, and the client's
231
.. function:: handle_error(request, client_address)
233
This function is called if the :attr:`RequestHandlerClass`'s :meth:`handle`
234
method raises an exception. The default action is to print the traceback to
235
standard output and continue handling further requests.
237
.. function:: handle_timeout()
239
This function is called when the :attr:`timeout` attribute has been set to a
240
value other than :const:`None` and the timeout period has passed with no
241
requests being received. The default action for forking servers is
242
to collect the status of any child processes that have exited, while
243
in threading servers this method does nothing.
245
.. function:: process_request(request, client_address)
247
Calls :meth:`finish_request` to create an instance of the
248
:attr:`RequestHandlerClass`. If desired, this function can create a new process
249
or thread to handle the request; the :class:`ForkingMixIn` and
250
:class:`ThreadingMixIn` classes do this.
252
.. Is there any point in documenting the following two functions?
253
What would the purpose of overriding them be: initializing server
254
instance variables, adding new network families?
257
.. function:: server_activate()
259
Called by the server's constructor to activate the server. The default behavior
260
just :meth:`listen`\ s to the server's socket. May be overridden.
263
.. function:: server_bind()
265
Called by the server's constructor to bind the socket to the desired address.
269
.. function:: verify_request(request, client_address)
271
Must return a Boolean value; if the value is :const:`True`, the request will be
272
processed, and if it's :const:`False`, the request will be denied. This function
273
can be overridden to implement access controls for a server. The default
274
implementation always returns :const:`True`.
277
RequestHandler Objects
278
----------------------
280
The request handler class must define a new :meth:`handle` method, and can
281
override any of the following methods. A new instance is created for each
285
.. function:: finish()
287
Called after the :meth:`handle` method to perform any clean-up actions
288
required. The default implementation does nothing. If :meth:`setup` or
289
:meth:`handle` raise an exception, this function will not be called.
292
.. function:: handle()
294
This function must do all the work required to service a request. The
295
default implementation does nothing. Several instance attributes are
296
available to it; the request is available as :attr:`self.request`; the client
297
address as :attr:`self.client_address`; and the server instance as
298
:attr:`self.server`, in case it needs access to per-server information.
300
The type of :attr:`self.request` is different for datagram or stream
301
services. For stream services, :attr:`self.request` is a socket object; for
302
datagram services, :attr:`self.request` is a pair of string and socket.
303
However, this can be hidden by using the request handler subclasses
304
:class:`StreamRequestHandler` or :class:`DatagramRequestHandler`, which
305
override the :meth:`setup` and :meth:`finish` methods, and provide
306
:attr:`self.rfile` and :attr:`self.wfile` attributes. :attr:`self.rfile` and
307
:attr:`self.wfile` can be read or written, respectively, to get the request
308
data or return data to the client.
311
.. function:: setup()
313
Called before the :meth:`handle` method to perform any initialization actions
314
required. The default implementation does nothing.
320
:class:`socketserver.TCPServer` Example
321
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
323
This is the server side::
327
class MyTCPHandler(socketserver.BaseRequestHandler):
329
The RequestHandler class for our server.
331
It is instantiated once per connection to the server, and must
332
override the handle() method to implement communication to the
337
# self.request is the TCP socket connected to the client
338
self.data = self.request.recv(1024).strip()
339
print("%s wrote:" % self.client_address[0])
341
# just send back the same data, but upper-cased
342
self.request.send(self.data.upper())
344
if __name__ == "__main__":
345
HOST, PORT = "localhost", 9999
347
# Create the server, binding to localhost on port 9999
348
server = socketserver.TCPServer((HOST, PORT), MyTCPHandler)
350
# Activate the server; this will keep running until you
351
# interrupt the program with Ctrl-C
352
server.serve_forever()
354
An alternative request handler class that makes use of streams (file-like
355
objects that simplify communication by providing the standard file interface)::
357
class MyTCPHandler(socketserver.StreamRequestHandler):
360
# self.rfile is a file-like object created by the handler;
361
# we can now use e.g. readline() instead of raw recv() calls
362
self.data = self.rfile.readline().strip()
363
print("%s wrote:" % self.client_address[0])
365
# Likewise, self.wfile is a file-like object used to write back
367
self.wfile.write(self.data.upper())
369
The difference is that the ``readline()`` call in the second handler will call
370
``recv()`` multiple times until it encounters a newline character, while the
371
single ``recv()`` call in the first handler will just return what has been sent
372
from the client in one ``send()`` call.
375
This is the client side::
380
HOST, PORT = "localhost", 9999
381
data = " ".join(sys.argv[1:])
383
# Create a socket (SOCK_STREAM means a TCP socket)
384
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
386
# Connect to server and send data
387
sock.connect((HOST, PORT))
388
sock.send(bytes(data + "\n","utf8"))
390
# Receive data from the server and shut down
391
received = sock.recv(1024)
394
print("Sent: %s" % data)
395
print("Received: %s" % received)
398
The output of the example should look something like this:
402
$ python TCPServer.py
404
b'hello world with TCP'
410
$ python TCPClient.py hello world with TCP
411
Sent: hello world with TCP
412
Received: b'HELLO WORLD WITH TCP'
413
$ python TCPClient.py python is nice
415
Received: b'PYTHON IS NICE'
418
:class:`socketserver.UDPServer` Example
419
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
421
This is the server side::
425
class MyUDPHandler(socketserver.BaseRequestHandler):
427
This class works similar to the TCP handler class, except that
428
self.request consists of a pair of data and client socket, and since
429
there is no connection the client address must be given explicitly
430
when sending data back via sendto().
434
data = self.request[0].strip()
435
socket = self.request[1]
436
print("%s wrote:" % self.client_address[0])
438
socket.sendto(data.upper(), self.client_address)
440
if __name__ == "__main__":
441
HOST, PORT = "localhost", 9999
442
server = socketserver.UDPServer((HOST, PORT), MyUDPHandler)
443
server.serve_forever()
445
This is the client side::
450
HOST, PORT = "localhost", 9999
451
data = " ".join(sys.argv[1:])
453
# SOCK_DGRAM is the socket type to use for UDP sockets
454
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
456
# As you can see, there is no connect() call; UDP has no connections.
457
# Instead, data is directly sent to the recipient via sendto().
458
sock.sendto(bytes(data + "\n","utf8"), (HOST, PORT))
459
received = sock.recv(1024)
461
print("Sent: %s" % data)
462
print("Received: %s" % received)
464
The output of the example should look exactly like for the TCP server example.
470
To build asynchronous handlers, use the :class:`ThreadingMixIn` and
471
:class:`ForkingMixIn` classes.
473
An example for the :class:`ThreadingMixIn` class::
479
class ThreadedTCPRequestHandler(socketserver.BaseRequestHandler):
482
data = self.request.recv(1024)
483
cur_thread = threading.current_thread()
484
response = bytes("%s: %s" % (cur_thread.getName(), data),'ascii')
485
self.request.send(response)
487
class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
490
def client(ip, port, message):
491
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
492
sock.connect((ip, port))
494
response = sock.recv(1024)
495
print("Received: %s" % response)
498
if __name__ == "__main__":
499
# Port 0 means to select an arbitrary unused port
500
HOST, PORT = "localhost", 0
502
server = ThreadedTCPServer((HOST, PORT), ThreadedTCPRequestHandler)
503
ip, port = server.server_address
505
# Start a thread with the server -- that thread will then start one
506
# more thread for each request
507
server_thread = threading.Thread(target=server.serve_forever)
508
# Exit the server thread when the main thread terminates
509
server_thread.setDaemon(True)
510
server_thread.start()
511
print("Server loop running in thread:", server_thread.name)
513
client(ip, port, b"Hello World 1")
514
client(ip, port, b"Hello World 2")
515
client(ip, port, b"Hello World 3")
520
The output of the example should look something like this::
522
$ python ThreadedTCPServer.py
523
Server loop running in thread: Thread-1
524
Received: b"Thread-2: b'Hello World 1'"
525
Received: b"Thread-3: b'Hello World 2'"
526
Received: b"Thread-4: b'Hello World 3'"
529
The :class:`ForkingMixIn` class is used in the same way, except that the server
530
will spawn a new process for each request.
added
removed
Doc/library/socketserver.rst
