9
This document shows how Python fits into the web. It presents some ways on
10
how to integrate Python with the web server and general practices useful for
9
This document shows how Python fits into the web. It presents some ways
10
to integrate Python with a web server, and general practices useful for
11
11
developing web sites.
14
Programming for the Web has become a hot topic since the raise of the "Web 2.0",
14
Programming for the Web has become a hot topic since the rise of "Web 2.0",
15
15
which focuses on user-generated content on web sites. It has always been
16
16
possible to use Python for creating web sites, but it was a rather tedious task.
17
Therefore, many so-called "frameworks" and helper tools were created to help
18
developers creating sites faster and these sites being more robust. This HOWTO
19
describes some of the methods used to combine Python with a web server to create
20
dynamic content. It is not meant as a general introduction as this topic is far
21
too broad to be covered in one single document. However, a short overview of
22
the most popular libraries is provided.
17
Therefore, many frameworks and helper tools have been created to assist
18
developers in creating faster and more robust sites. This HOWTO describes
19
some of the methods used to combine Python with a web server to create
20
dynamic content. It is not meant as a complete introduction, as this topic is
21
far too broad to be covered in one single document. However, a short overview
22
of the most popular libraries is provided.
26
While this HOWTO tries to give an overview over Python in the Web, it cannot
27
always be as up to date as desired. Web development in Python is moving
28
forward rapidly, so the wiki page on `Web Programming
29
<http://wiki.python.org/moin/WebProgramming>`_ might be more in sync with
26
While this HOWTO tries to give an overview of Python in the web, it cannot
27
always be as up to date as desired. Web development in Python is rapidly
28
moving forward, so the wiki page on `Web Programming
29
<http://wiki.python.org/moin/WebProgramming>`_ may be more in sync with
30
30
recent development.
36
.. .. image:: http.png
38
When a user enters a web site, his browser makes a connection to the site's
39
webserver (this is called the *request*). The server looks up the file in the
36
When a user enters a web site, their browser makes a connection to the site's
37
web server (this is called the *request*). The server looks up the file in the
40
38
file system and sends it back to the user's browser, which displays it (this is
41
the *response*). This is roughly how the unterlying protocol, HTTP works.
43
Now, dynamic web sites are not files in the file system, but rather programs
44
which are run by the web server when a request comes in. They can do all sorts
45
of useful things, like display the postings of a bulletin board, show your
46
mails, configurate software or just display the current time. These programs
47
can be written in about any programming language the server supports, so it is
48
easy to use Python for creating dynamic web sites.
50
As most of HTTP servers are written in C or C++, they cannot execute Python code
51
in a simple way -- a bridge is needed between the server and the program. These
52
bridges or rather interfaces define how programs interact with the server. In
53
the past there have been numerous attempts to create the best possible
54
interface, but there are only a few worth mentioning.
56
Not every web server supports every interface. Many web servers do support only
57
old, now-obsolete interfaces. But they can often be extended using some
58
third-party modules to support new interfaces.
39
the *response*). This is roughly how the underlying protocol, HTTP, works.
41
Dynamic web sites are not based on files in the file system, but rather on
42
programs which are run by the web server when a request comes in, and which
43
*generate* the content that is returned to the user. They can do all sorts of
44
useful things, like display the postings of a bulletin board, show your email,
45
configure software, or just display the current time. These programs can be
46
written in any programming language the server supports. Since most servers
47
support Python, it is easy to use Python to create dynamic web sites.
49
Most HTTP servers are written in C or C++, so they cannot execute Python code
50
directly -- a bridge is needed between the server and the program. These
51
bridges, or rather interfaces, define how programs interact with the server.
52
There have been numerous attempts to create the best possible interface, but
53
there are only a few worth mentioning.
55
Not every web server supports every interface. Many web servers only support
56
old, now-obsolete interfaces; however, they can often be extended using
57
third-party modules to support newer ones.
61
60
Common Gateway Interface
62
61
------------------------
64
This interface is the oldest one, supported by nearly every web server out of
65
the box. Programs using CGI to communicate with their web server need to be
66
started by the server for every request. So, every request starts a new Python
67
interpreter -- which takes some time to start up -- thus making the whole
68
interface only usable for low load situations.
70
The upside of CGI is that it is simple -- writing a program which uses CGI is a
71
matter of about three lines of code. But this simplicity comes at a price: it
72
does very few things to help the developer.
74
Writing CGI programs, while still possible, is not recommended anymore. With
75
WSGI (more on that later) it is possible to write programs that emulate CGI, so
76
they can be run as CGI if no better option is available.
63
This interface, most commonly referred to as "CGI", is the oldest, and is
64
supported by nearly every web server out of the box. Programs using CGI to
65
communicate with their web server need to be started by the server for every
66
request. So, every request starts a new Python interpreter -- which takes some
67
time to start up -- thus making the whole interface only usable for low load
70
The upside of CGI is that it is simple -- writing a Python program which uses
71
CGI is a matter of about three lines of code. This simplicity comes at a
72
price: it does very few things to help the developer.
74
Writing CGI programs, while still possible, is no longer recommended. With
75
:ref:`WSGI <WSGI>`, a topic covered later in this document, it is possible to write
76
programs that emulate CGI, so they can be run as CGI if no better option is
125
125
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
127
127
If you don't have your own web server, this does not apply to you. You can
128
check whether if works as-is and if not you need to talk to the administrator of
129
your web server anyway. If it is a big hoster, you can try filing a ticket
130
asking for Python support.
132
If you're your own administrator or want to install it for testing purposes on
133
your own computers, you have to configure it by yourself. There is no one and
134
single way on how to configure CGI, as there are many web servers with different
135
configuration options. The currently most widely used free web server is
136
`Apache HTTPd <http://httpd.apache.org/>`_, Apache for short -- this is the one
137
that most people use, it can be easily installed on nearly every system using
138
the systems' package management. But `lighttpd <http://www.lighttpd.net>`_ has
139
been gaining attention since some time and is said to have a better performance.
140
On many systems this server can also be installed using the package management,
141
so manually compiling the web server is never needed.
143
* On Apache you can take a look into the `Dynamic Content with CGI
128
check whether it works as-is, and if not you will need to talk to the
129
administrator of your web server. If it is a big host, you can try filing a
130
ticket asking for Python support.
132
If you are your own administrator or want to set up CGI for testing purposes on
133
your own computers, you have to configure it by yourself. There is no single
134
way to configure CGI, as there are many web servers with different
135
configuration options. Currently the most widely used free web server is
136
`Apache HTTPd <http://httpd.apache.org/>`_, or Apache for short. Apache can be
137
easily installed on nearly every system using the system's package management
138
tool. `lighttpd <http://www.lighttpd.net>`_ is another alternative and is
139
said to have better performance. On many systems this server can also be
140
installed using the package management tool, so manually compiling the web
141
server may not be needed.
143
* On Apache you can take a look at the `Dynamic Content with CGI
144
144
<http://httpd.apache.org/docs/2.2/howto/cgi.html>`_ tutorial, where everything
145
145
is described. Most of the time it is enough just to set ``+ExecCGI``. The
146
146
tutorial also describes the most common gotchas that might arise.
147
148
* On lighttpd you need to use the `CGI module
148
<http://trac.lighttpd.net/trac/wiki/Docs%3AModCGI>`_ which can be configured
149
<http://redmine.lighttpd.net/wiki/lighttpd/Docs:ModCGI>`_\ , which can be configured
149
150
in a straightforward way. It boils down to setting ``cgi.assign`` properly.
152
153
Common problems with CGI scripts
153
154
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
155
Trying to use CGI sometimes leads to small annoyances that one might experience
156
while trying to get these scripts to run. Sometimes it happens that a seemingly
157
correct script does not work as expected, which is caused by some small hidden
158
reason that's difficult to spot.
160
Some of these reasons are:
162
* The Python script is not marked executable. When CGI scripts are not
163
executable most of the web servers will let the user download it, instead of
156
Using CGI sometimes leads to small annoyances while trying to get these
157
scripts to run. Sometimes a seemingly correct script does not work as
158
expected, the cause being some small hidden problem that's difficult to spot.
160
Some of these potential problems are:
162
* The Python script is not marked as executable. When CGI scripts are not
163
executable most web servers will let the user download it, instead of
164
164
running it and sending the output to the user. For CGI scripts to run
165
properly the ``+x`` bit needs to be set. Using ``chmod a+x your_script.py``
166
might already solve the problem.
167
* The line endings must be of Unix-type. This is important because the web
168
server checks the first line of the script (called shebang) and tries to run
169
the program specified there. It gets easily confused by Windows line endings
170
(Carriage Return & Line Feed, also called CRLF), so you have to convert the
171
file to Unix line endings (only Line Feed, LF). This can be done
172
automatically by uploading the file via FTP in text mode instead of binary
173
mode, but the preferred way is just telling your editor to save the files with
174
Unix line endings. Most proper editors support this.
175
* Your web server must be able to read the file, you need to make sure the
176
permissions are fine. Often the server runs as user and group ``www-data``,
177
so it might be worth a try to change the file ownership or making the file
178
world readable by using ``chmod a+r your_script.py``.
179
* The webserver must be able to know that the file you're trying to access is a
180
CGI script. Check the configuration of your web server, maybe there is some
182
* The path to the interpreter in the shebang (``#!/usr/bin/env python``) must be
183
currect. This line calls ``/usr/bin/env`` to find Python, but it'll fail if
184
there is no ``/usr/bin/env``. If you know where your Python is installed, you
185
can also use that path. The commands ``whereis python`` and ``type -p
186
python`` might also help to find where it is installed. Once this is known,
187
the shebang line can be changed accordingly: ``#!/usr/bin/python``.
165
properly on Unix-like operating systems, the ``+x`` bit needs to be set.
166
Using ``chmod a+x your_script.py`` may solve this problem.
168
* On a Unix-like system, The line endings in the program file must be Unix
169
style line endings. This is important because the web server checks the
170
first line of the script (called shebang) and tries to run the program
171
specified there. It gets easily confused by Windows line endings (Carriage
172
Return & Line Feed, also called CRLF), so you have to convert the file to
173
Unix line endings (only Line Feed, LF). This can be done automatically by
174
uploading the file via FTP in text mode instead of binary mode, but the
175
preferred way is just telling your editor to save the files with Unix line
176
endings. Most editors support this.
178
* Your web server must be able to read the file, and you need to make sure the
179
permissions are correct. On unix-like systems, the server often runs as user
180
and group ``www-data``, so it might be worth a try to change the file
181
ownership, or making the file world readable by using ``chmod a+r
184
* The web server must know that the file you're trying to access is a CGI script.
185
Check the configuration of your web server, as it may be configured
186
to expect a specific file extension for CGI scripts.
188
* On Unix-like systems, the path to the interpreter in the shebang
189
(``#!/usr/bin/env python``) must be correct. This line calls
190
``/usr/bin/env`` to find Python, but it will fail if there is no
191
``/usr/bin/env``, or if Python is not in the web server's path. If you know
192
where your Python is installed, you can also use that full path. The
193
commands ``whereis python`` and ``type -p python`` could help you find
194
where it is installed. Once you know the path, you can change the shebang
195
accordingly: ``#!/usr/bin/python``.
188
197
* The file must not contain a BOM (Byte Order Mark). The BOM is meant for
189
determining the byte order of UTF-16 encodings, but some editors write this
190
also into UTF-8 files. The BOM interferes with the shebang line, so be sure
191
to tell your editor not to write the BOM.
192
* :ref:`mod-python` might be making problems. mod_python is able to handle CGI
193
scripts by itself, but it can also be a source for problems. Be sure you
198
determining the byte order of UTF-16 and UTF-32 encodings, but some editors
199
write this also into UTF-8 files. The BOM interferes with the shebang line,
200
so be sure to tell your editor not to write the BOM.
202
* If the web server is using :ref:`mod-python`, ``mod_python`` may be having
203
problems. ``mod_python`` is able to handle CGI scripts by itself, but it can
204
also be a source of issues.
202
212
People coming from PHP often find it hard to grasp how to use Python in the web.
203
Their first thought is mostly `mod_python <http://www.modpython.org/>`_ because
204
they think that this is the equivalent to ``mod_php``. Actually it is not
205
really. It does embed the interpreter into the Apache process, thus speeding up
206
requests by not having to start a Python interpreter every request. On the
207
other hand, it is by far not "Python intermixed with HTML" as PHP often does.
208
The Python equivalent of that is a template engine. mod_python itself is much
209
more powerful and gives more access to Apache internals. It can emulate CGI, it
210
can work an a "Python Server Pages" mode similar to JSP which is "HTML
211
intermangled with Python" and it has a "Publisher" which destignates one file to
212
accept all requests and decide on what to do then.
214
But mod_python has some problems. Unlike the PHP interpreter the Python
215
interpreter uses caching when executing files, so when changing a file the whole
216
web server needs to be re-started to update. Another problem ist the basic
217
concept -- Apache starts some child processes to handle the requests and
218
unfortunately every child process needs to load the whole Python interpreter
219
even if it does not use it. This makes the whole web server slower. Another
220
problem is that as mod_python is linked against a specific version of
221
``libpython``, it is not possible to switch from an older version to a newer
222
(e.g. 2.4 to 2.5) without recompiling mod_python. mod_python is also bound to
223
the Apache web server, so programs written for mod_python cannot easily run on
226
These are the reasons why mod_python should be avoided when writing new
227
programs. In some circumstances it might be still a good idea to use mod_python
228
for deployment, but WSGI makes it possible to run WSGI programs under mod_python
213
Their first thought is mostly `mod_python <http://www.modpython.org/>`_\ ,
214
because they think that this is the equivalent to ``mod_php``. Actually, there
215
are many differences. What ``mod_python`` does is embed the interpreter into
216
the Apache process, thus speeding up requests by not having to start a Python
217
interpreter for each request. On the other hand, it is not "Python intermixed
218
with HTML" in the way that PHP is often intermixed with HTML. The Python
219
equivalent of that is a template engine. ``mod_python`` itself is much more
220
powerful and provides more access to Apache internals. It can emulate CGI,
221
work in a "Python Server Pages" mode (similar to JSP) which is "HTML
222
intermingled with Python", and it has a "Publisher" which designates one file
223
to accept all requests and decide what to do with them.
225
``mod_python`` does have some problems. Unlike the PHP interpreter, the Python
226
interpreter uses caching when executing files, so changes to a file will
227
require the web server to be restarted. Another problem is the basic concept
228
-- Apache starts child processes to handle the requests, and unfortunately
229
every child process needs to load the whole Python interpreter even if it does
230
not use it. This makes the whole web server slower. Another problem is that,
231
because ``mod_python`` is linked against a specific version of ``libpython``,
232
it is not possible to switch from an older version to a newer (e.g. 2.4 to 2.5)
233
without recompiling ``mod_python``. ``mod_python`` is also bound to the Apache
234
web server, so programs written for ``mod_python`` cannot easily run on other
237
These are the reasons why ``mod_python`` should be avoided when writing new
238
programs. In some circumstances it still might be a good idea to use
239
``mod_python`` for deployment, but WSGI makes it possible to run WSGI programs
240
under ``mod_python`` as well.
303
317
There is some documentation on `setting up Django with FastCGI
304
<http://www.djangoproject.com/documentation/fastcgi/>`_, most of which can be
305
reused for other WSGI-compliant frameworks and libraries. Only the
306
``manage.py`` part has to be changed, the example used here can be used
307
instead. Django does more or less the exact same thing.
318
<http://docs.djangoproject.com/en/dev/howto/deployment/fastcgi/>`_, most of
319
which can be reused for other WSGI-compliant frameworks and libraries.
320
Only the ``manage.py`` part has to be changed, the example used here can be
321
used instead. Django does more or less the exact same thing.
313
`mod_wsgi <http://www.modwsgi.org/>`_ is an attempt to get rid of the low level
314
gateways. As FastCGI, SCGI, mod_python are mostly used to deploy WSGI
315
applications anyway, mod_wsgi was started to directly embed WSGI aplications
316
into the Apache web server. The benefit from this approach is that WSGI
317
applications can be deployed much easier as is is specially designed to host
318
WSGI applications -- unlike the other low level methods which have glue code to
319
host WSGI applications (like flup which was mentioned before). The downside is
320
that mod_wsgi is limited to the Apache web server, other servers would need
327
`mod_wsgi <http://code.google.com/p/modwsgi/>`_ is an attempt to get rid of the
328
low level gateways. Given that FastCGI, SCGI, and mod_python are mostly used to
329
deploy WSGI applications, mod_wsgi was started to directly embed WSGI applications
330
into the Apache web server. mod_wsgi is specifically designed to host WSGI
331
applications. It makes the deployment of WSGI applications much easier than
332
deployment using other low level methods, which need glue code. The downside
333
is that mod_wsgi is limited to the Apache web server; other servers would need
321
334
their own implementations of mod_wsgi.
323
It supports two modes: the embedded mode in which it integrates with the Apache
324
process and the daemon mode which is more FastCGI-like. Contrary to FastCGI,
325
mod_wsgi handles the worker-processes by itself which makes administration
336
mod_wsgi supports two modes: embedded mode, in which it integrates with the
337
Apache process, and daemon mode, which is more FastCGI-like. Unlike FastCGI,
338
mod_wsgi handles the worker-processes by itself, which makes administration
334
WSGI was already mentioned several times so it has to be something important.
335
In fact it really is, so now it's time to explain.
337
The *Web Server Gateway Interface*, :pep:`333` or WSGI for short is currently
338
the best possible way to Python web programming. While it is great for
339
programmers writing frameworks, the normal person does not need to get in direct
340
contact with it. But when choosing a framework for web development it is a good
341
idea to take one which supports WSGI.
343
The big profit from WSGI is the unification. When your program is compatible
344
with WSGI -- that means that your framework has support for WSGI, your program
345
can be deployed on every web server interface for which there are WSGI wrappers.
346
So you do not need to care about whether the user uses mod_python or FastCGI --
347
with WSGI it just works on any gateway interface. The Python standard library
348
contains its own WSGI server :mod:`wsgiref`, which is a small web server that
349
can be used for testing.
351
A really great WSGI feature are the middlewares. Middlewares are layers around
352
your program which can add various functionality to it. There is a `number of
353
middlewares <http://wsgi.org/wsgi/Middleware_and_Utilities>`_ already available.
354
For example, instead of writing your own session management (to identify a user
355
in subsequent requests, as HTTP does not maintain state, so it does now know
356
that the requests belong to the same user) you can just take one middleware,
357
plug it in and you can rely an already existing functionality. The same thing
358
is compression -- say you want to compress your HTML using gzip, to save your
359
server's bandwidth. So you only need to plug-in a middleware and you're done.
360
Authentication is also a problem easily solved using a middleware.
362
So, generally -- although WSGI may seem complex, the initial phase of learning
363
can be very rewarding as WSGI does already have solutions to many problems that
364
might arise while writing web sites.
347
WSGI has already been mentioned several times, so it has to be something
348
important. In fact it really is, and now it is time to explain it.
350
The *Web Server Gateway Interface*, or WSGI for short, is defined in
351
:pep:`333` and is currently the best way to do Python web programming. While
352
it is great for programmers writing frameworks, a normal web developer does not
353
need to get in direct contact with it. When choosing a framework for web
354
development it is a good idea to choose one which supports WSGI.
356
The big benefit of WSGI is the unification of the application programming
357
interface. When your program is compatible with WSGI -- which at the outer
358
level means that the framework you are using has support for WSGI -- your
359
program can be deployed via any web server interface for which there are WSGI
360
wrappers. You do not need to care about whether the application user uses
361
mod_python or FastCGI or mod_wsgi -- with WSGI your application will work on
362
any gateway interface. The Python standard library contains its own WSGI
363
server, :mod:`wsgiref`, which is a small web server that can be used for
366
A really great WSGI feature is middleware. Middleware is a layer around your
367
program which can add various functionality to it. There is quite a bit of
368
`middleware <http://wsgi.org/wsgi/Middleware_and_Utilities>`_ already
369
available. For example, instead of writing your own session management (HTTP
370
is a stateless protocol, so to associate multiple HTTP requests with a single
371
user your application must create and manage such state via a session), you can
372
just download middleware which does that, plug it in, and get on with coding
373
the unique parts of your application. The same thing with compression -- there
374
is existing middleware which handles compressing your HTML using gzip to save
375
on your server's bandwidth. Authentication is another a problem easily solved
376
using existing middleware.
378
Although WSGI may seem complex, the initial phase of learning can be very
379
rewarding because WSGI and the associated middleware already have solutions to
380
many problems that might arise while developing web sites.
370
386
The code that is used to connect to various low level gateways like CGI or
371
mod_python is called *WSGI server*. One of these servers is ``flup`` which was
372
already mentioned and supports FastCGI, SCGI as well as `AJP
387
mod_python is called a *WSGI server*. One of these servers is ``flup``, which
388
supports FastCGI and SCGI, as well as `AJP
373
389
<http://en.wikipedia.org/wiki/Apache_JServ_Protocol>`_. Some of these servers
374
are written in Python as ``flup`` is, but there also exist others which are
390
are written in Python, as ``flup`` is, but there also exist others which are
375
391
written in C and can be used as drop-in replacements.
377
There are quite a lot of servers already available, so a Python web application
378
can be deployed nearly everywhere. This is one big advantage that Python has
379
compared with other web techniques.
393
There are many servers already available, so a Python web application
394
can be deployed nearly anywhere. This is one big advantage that Python has
395
compared with other web technologies.
383
A good overview of all WSGI-related code can be found in the `WSGI wiki
399
A good overview of WSGI-related code can be found in the `WSGI wiki
384
400
<http://wsgi.org/wsgi>`_, which contains an extensive list of `WSGI servers
385
<http://wsgi.org/wsgi/Servers>`_, which can be used by *every* application
401
<http://wsgi.org/wsgi/Servers>`_ which can be used by *any* application
388
404
You might be interested in some WSGI-supporting modules already contained in
394
410
Case study: MoinMoin
395
411
--------------------
397
What does WSGI give the web application developer? Let's take a look on one
398
long existing web application written in Python without using WSGI.
400
One of the most widely used wiki software is `MoinMoin <http://moinmo.in/>`_.
401
It was created in 2000, so it predates WSGI by about three years. While it now
402
includes support for WSGI, older versions needed separate code to run on CGI,
403
mod_python, FastCGI and standalone. Now, this all is possible by using WSGI and
404
the already-written gateways. For running with on FastCGI ``flup`` can be used,
405
for running a standalone server :mod:`wsgiref` is the way to go.
408
Model-view-controller
413
What does WSGI give the web application developer? Let's take a look at
414
an application that's been around for a while, which was written in
415
Python without using WSGI.
417
One of the most widely used wiki software packages is `MoinMoin
418
<http://moinmo.in/>`_. It was created in 2000, so it predates WSGI by about
419
three years. Older versions needed separate code to run on CGI, mod_python,
420
FastCGI and standalone.
422
It now includes support for WSGI. Using WSGI, it is possible to deploy
423
MoinMoin on any WSGI compliant server, with no additional glue code.
424
Unlike the pre-WSGI versions, this could include WSGI servers that the
425
authors of MoinMoin know nothing about.
428
Model-View-Controller
409
429
=====================
411
The term *MVC* is often heard in statements like "framework *foo* supports MVC".
412
While MVC is not really something technical but rather organisational, many web
413
frameworks use this model to help the developer to bring structure into his
414
program. Bigger web applications can have lots of code so it is a good idea to
415
have structure in the program right from the beginnings. That way, even users
416
of other frameworks (or even languages, as MVC is nothing Python-specific) can
417
understand the existing code easier, as they are already familiar with the
431
The term *MVC* is often encountered in statements such as "framework *foo*
432
supports MVC". MVC is more about the overall organization of code, rather than
433
any particular API. Many web frameworks use this model to help the developer
434
bring structure to their program. Bigger web applications can have lots of
435
code, so it is a good idea to have an effective structure right from the beginning.
436
That way, even users of other frameworks (or even other languages, since MVC is
437
not Python-specific) can easily understand the code, given that they are
438
already familiar with the MVC structure.
420
440
MVC stands for three components:
422
* The *model*. This is the data that is meant to modify. In Python frameworks
423
this component is often represented by the classes used by the
424
object-relational mapper. So, all declarations go here.
442
* The *model*. This is the data that will be displayed and modified. In
443
Python frameworks, this component is often represented by the classes used by
444
an object-relational mapper.
425
446
* The *view*. This component's job is to display the data of the model to the
426
user. Typically this component is represented by the templates.
447
user. Typically this component is implemented via templates.
427
449
* The *controller*. This is the layer between the user and the model. The
428
controller reacts on user actions (like opening some specific URL) and tells
429
the model to modify the data if necessary.
450
controller reacts to user actions (like opening some specific URL), tells
451
the model to modify the data if necessary, and tells the view code what to
431
454
While one might think that MVC is a complex design pattern, in fact it is not.
432
455
It is used in Python because it has turned out to be useful for creating clean,
438
461
to create a web site which uses the MVC pattern by separating the data logic
439
462
(the model) from the user interaction logic (the controller) and the
440
463
templates (the view). That's why it is important not to write unnecessary
441
Python code in the templates -- it is against MVC and creates more chaos.
445
The english Wikipedia has an article about the `Model-View-Controller pattern
446
<http://en.wikipedia.org/wiki/Model-view-controller>`_, which includes a long
447
list of web frameworks for different programming languages.
450
Ingredients for web sites
451
=========================
453
Web sites are complex constructs, so tools were created to help the web site
454
developer to make his work maintainable. None of these tools are in any way
455
Python specific, they also exist for other programming languages as well. Of
456
course, developers are not forced to use these tools and often there is no
457
"best" tool, but it is worth informing yourself before choosing something
458
because of the big number of helpers that the developer can use.
463
People have written far more components that can be combined than these
464
presented here. The Python wiki has a page about these components, called
464
Python code in the templates -- it works against the MVC model and creates
465
chaos in the code base, making it harder to understand and modify.
469
The English Wikipedia has an article about the `Model-View-Controller pattern
470
<http://en.wikipedia.org/wiki/Model-view-controller>`_. It includes a long
471
list of web frameworks for various programming languages.
474
Ingredients for Websites
475
========================
477
Websites are complex constructs, so tools have been created to help web
478
developers make their code easier to write and more maintainable. Tools like
479
these exist for all web frameworks in all languages. Developers are not forced
480
to use these tools, and often there is no "best" tool. It is worth learning
481
about the available tools because they can greatly simplify the process of
482
developing a web site.
487
There are far more components than can be presented here. The Python wiki
488
has a page about these components, called
465
489
`Web Components <http://wiki.python.org/moin/WebComponents>`_.
471
Mixing of HTML and Python code is possible with some libraries. While
495
Mixing of HTML and Python code is made possible by a few libraries. While
472
496
convenient at first, it leads to horribly unmaintainable code. That's why
473
497
templates exist. Templates are, in the simplest case, just HTML files with
474
placeholders. The HTML is sent to the user's browser after filling out the
498
placeholders. The HTML is sent to the user's browser after filling in the
477
Python already includes such simple templates::
501
Python already includes a way to build simple templates::
479
503
# a simple template
480
504
template = "<html><body><h1>Hello {who}!</h1></body></html>"
481
505
print(template.format(who="Reader"))
483
The Python standard library also includes some more advanced templates usable
484
through :class:`string.Template`, but in HTML templates it is needed to use
485
conditional and looping contructs like Python's *for* and *if*. So, some
486
*template engine* is needed.
488
Now, Python has a lot of template engines which can be used with or without a
489
`framework`_. Some of these are using a plain-text programming language which
490
is very easy to learn as it is quite limited while others use XML so the
491
template output is always guaranteed to be valid XML. Some `frameworks`_ ship
492
their own template engine or recommend one particular. If one is not yet sure,
493
using these is a good idea.
497
While Python has quite a lot of different template engines it usually does
498
not make sense to use a homebrewed template system. The time needed to
499
evaluate all templating systems is not really worth it, better invest the
500
time in looking through the most popular ones. Some frameworks have their
501
own template engine or have a recommentation for one. It's wise to use
504
Popular template engines include:
507
To generate complex HTML based on non-trivial model data, conditional
508
and looping constructs like Python's *for* and *if* are generally needed.
509
*Template engines* support templates of this complexity.
511
There are a lot of template engines available for Python which can be used with
512
or without a `framework`_. Some of these define a plain-text programming
513
language which is easy to learn, partly because it is limited in scope.
514
Others use XML, and the template output is guaranteed to be always be valid
515
XML. There are many other variations.
517
Some `frameworks`_ ship their own template engine or recommend one in
518
particular. In the absence of a reason to use a different template engine,
519
using the one provided by or recommended by the framework is a good idea.
521
Popular template engines include:
523
* `Mako <http://www.makotemplates.org/>`_
524
* `Genshi <http://genshi.edgewall.org/>`_
525
* `Jinja <http://jinja.pocoo.org/2/>`_
512
Lots of different template engines divide the attention between themselves
513
because it's easy to create them in Python. The page `Templating
529
There are many template engines competing for attention, becuase it is
530
pretty easy to create them in Python. The page `Templating
514
531
<http://wiki.python.org/moin/Templating>`_ in the wiki lists a big,
515
ever-growing number of these.
532
ever-growing number of these. The three listed above are considered "second
533
generation" template engines and are a good place to start.
521
*Data persistence*, while sounding very complicated is just about storing data.
522
This data might be the text of blog entries, the postings of a bulletin board or
523
the text of a wiki page. As always, there are different ways to store
524
informations on a web server.
526
Often relational database engines like `MySQL <http://www.mysql.com/>`_ or
527
`PostgreSQL <http://www.postgresql.org/>`_ are used due to their good
528
performance handling very large databases consisting of up to millions of
529
entries. These are *queried* using a language called `SQL
530
<http://en.wikipedia.org/wiki/SQL>`_. Python programmers in general do not like
531
SQL too much, they prefer to work with objects. It is possible to save Python
532
objects into a database using a technology called `ORM
533
<http://en.wikipedia.org/wiki/Object-relational_mapping>`_. ORM translates all
534
object-oriented access into SQL code under the hood, the user does not need to
535
think about it. Most `frameworks`_ use ORMs and it works quite well.
537
A second possibility is using files that are saved on the hard disk (sometimes
538
called flatfiles). This is very easy, but is not too fast. There is even a
539
small database engine called `SQLite <http://www.sqlite.org/>`_ which is bundled
540
with Python in the :mod:`sqlite` module and uses only one file. This database
541
can be used to store objects via an ORM and has no other dependencies. For
542
smaller sites SQLite is just enough. But it is not the only way in which data
543
can be saved into the file systems. Sometimes normal, plain text files are
546
The third and least used possibility are so-called object oriented databases.
547
These databases store the *actual objects* instead of the relations that
548
OR-mapping creates between rows in a database. This has the advantage that
549
nearly all objects can be saven in a straightforward way, unlike in relational
550
databases where some objects are very hard to represent with ORMs.
552
`Frameworks`_ often give the users hints on which method to choose, it is
553
usually a good idea to stick to these unless there are some special requirements
554
which require to use the one method and not the other.
539
*Data persistence*, while sounding very complicated, is just about storing data.
540
This data might be the text of blog entries, the postings on a bulletin board or
541
the text of a wiki page. There are, of course, a number of different ways to store
542
information on a web server.
544
Often, relational database engines like `MySQL <http://www.mysql.com/>`_ or
545
`PostgreSQL <http://www.postgresql.org/>`_ are used because of their good
546
performance when handling very large databases consisting of millions of
547
entries. There is also a small database engine called `SQLite
548
<http://www.sqlite.org/>`_, which is bundled with Python in the :mod:`sqlite3`
549
module, and which uses only one file. It has no other dependencies. For
550
smaller sites SQLite is just enough.
552
Relational databases are *queried* using a language called `SQL
553
<http://en.wikipedia.org/wiki/SQL>`_. Python programmers in general do not
554
like SQL too much, as they prefer to work with objects. It is possible to save
555
Python objects into a database using a technology called `ORM
556
<http://en.wikipedia.org/wiki/Object-relational_mapping>`_ (Object Relational
557
Mapping). ORM translates all object-oriented access into SQL code under the
558
hood, so the developer does not need to think about it. Most `frameworks`_ use
559
ORMs, and it works quite well.
561
A second possibility is storing data in normal, plain text files (some
562
times called "flat files"). This is very easy for simple sites,
563
but can be difficult to get right if the web site is performing many
564
updates to the stored data.
566
A third possibility are object oriented databases (also called "object
567
databases"). These databases store the object data in a form that closely
568
parallels the way the objects are structured in memory during program
569
execution. (By contrast, ORMs store the object data as rows of data in tables
570
and relations between those rows.) Storing the objects directly has the
571
advantage that nearly all objects can be saved in a straightforward way, unlike
572
in relational databases where some objects are very hard to represent.
574
`Frameworks`_ often give hints on which data storage method to choose. It is
575
usually a good idea to stick to the data store recommended by the framework
576
unless the application has special requirements better satisfied by an
577
alternate storage mechanism.
558
581
* `Persistence Tools <http://wiki.python.org/moin/PersistenceTools>`_ lists
559
possibilities on how to save data in the file system, some of these modules
560
are part of the standard library
582
possibilities on how to save data in the file system. Some of these
583
modules are part of the standard library
561
585
* `Database Programming <http://wiki.python.org/moin/DatabaseProgramming>`_
562
helps on choosing a method on how to save the data
563
* `SQLAlchemy <http://www.sqlalchemy.org/>`_, the most powerful OR-Mapper for
564
Python and `Elixir <http://elixir.ematia.de/>`_ which makes it easier to
586
helps with choosing a method for saving data
588
* `SQLAlchemy <http://www.sqlalchemy.org/>`_, the most powerful OR-Mapper
589
for Python, and `Elixir <http://elixir.ematia.de/>`_, which makes
590
SQLAlchemy easier to use
566
592
* `SQLObject <http://www.sqlobject.org/>`_, another popular OR-Mapper
567
594
* `ZODB <https://launchpad.net/zodb>`_ and `Durus
568
595
<http://www.mems-exchange.org/software/durus/>`_, two object oriented
577
As web sites can easily become quite large, there are so-called frameworks which
578
were created to help the developer with making these sites. Although the most
579
well-known framework is Ruby on Rails, Python does also have its own frameworks
580
which are partly inspired by Rails or which were existing a long time before
583
Two possible approaches to web frameworks exist: the minimalistic approach and
584
the all-inclusive approach (somtimes called *full-stack*). Frameworks which are
585
all-inclusive give you everything you need to start working, like a template
586
engine, some way to save and access data in databases and many features more.
587
Most users are best off using these as they are widely used by lots of other
588
users and well documented in form of books and tutorials. Other web frameworks
589
go the minimalistic approach trying to be as flexible as possible leaving the
590
user the freedom to choose what's best for him.
592
The majority of users is best off with all-inclusive framewors. They bring
593
everything along so a user can just jump in and start to code. While they do
594
have some limitations they can fullfill 80% of what one will ever want to
595
perfectly. They consist of various components which are designed to work
596
together as good as possible.
598
The multitude of web frameworks written in Python demonstrates that it is really
599
easy to write one. One of the most well-known web applications written in
600
Python is `Zope <http://www.zope.org/>`_ which can be regarded as some kind of
601
big framework. But Zope was not the only framework, there were some others
602
which are by now nearly forgotten. These do not need to be mentioned anymore,
603
because most people that used them moved on to newer ones.
604
The process of creating code to run web sites involves writing code to provide
605
various services. The code to provide a particular service often works the
606
same way regardless of the complexity or purpose of the web site in question.
607
Abstracting these common solutions into reusable code produces what are called
608
"frameworks" for web development. Perhaps the most well-known framework for
609
web development is Ruby on Rails, but Python has its own frameworks. Some of
610
these were partly inspired by Rails, or borrowed ideas from Rails, but many
611
existed a long time before Rails.
613
Originally Python web frameworks tended to incorporate all of the services
614
needed to develop web sites as a giant, integrated set of tools. No two web
615
frameworks were interoperable: a program developed for one could not be
616
deployed on a different one without considerable re-engineering work. This led
617
to the development of "minimalist" web frameworks that provided just the tools
618
to communicate between the Python code and the http protocol, with all other
619
services to be added on top via separate components. Some ad hoc standards
620
were developed that allowed for limited interoperability between frameworks,
621
such as a standard that allowed different template engines to be used
624
Since the advent of WSGI, the Python web framework world has been evolving
625
toward interoperability based on the WSGI standard. Now many web frameworks,
626
whether "full stack" (providing all the tools one needs to deploy the most
627
complex web sites) or minimalist, or anything in between, are built from
628
collections of reusable components that can be used with more than one
631
The majority of users will probably want to select a "full stack" framework
632
that has an active community. These frameworks tend to be well documented,
633
and provide the easiest path to producing a fully functional web site in
606
637
Some notable frameworks
607
638
-----------------------
609
There is an incredible number of frameworks, so there is no way to describe them
610
all. It is not even necessary, as most of these frameworks are nothing special
611
and everything that can be done with these can also be done with one of the
640
There are an incredible number of frameworks, so they cannot all be covered
641
here. Instead we will briefly touch on some of the most popular.
657
684
questions. There is also a `TurboGears book <http://turbogearsbook.com/>`_
658
685
published, which is a good starting point.
660
The plan for the next major version of TurboGears, version 2.0 is to switch to a
661
more flexible base provided by another very flexible web framework called
662
`Pylons <http://pylonshq.com/>`_.
687
The newest version of TurboGears, version 2.0, moves even further in direction
688
of WSGI support and a component-based architecture. TurboGears 2 is based on
689
the WSGI stack of another popular component-based web framework, `Pylons
690
<http://pylonshq.com/>`_.
696
The Zope framework is one of the "old original" frameworks. Its current
697
incarnation in Zope2 is a tightly integrated full-stack framework. One of its
698
most interesting feature is its tight integration with a powerful object
699
database called the `ZODB <https://launchpad.net/zodb>`_ (Zope Object Database).
700
Because of its highly integrated nature, Zope wound up in a somewhat isolated
701
ecosystem: code written for Zope wasn't very usable outside of Zope, and
702
vice-versa. To solve this problem the Zope 3 effort was started. Zope 3
703
re-engineers Zope as a set of more cleanly isolated components. This effort
704
was started before the advent of the WSGI standard, but there is WSGI support
705
for Zope 3 from the `Repoze <http://repoze.org/>`_ project. Zope components
706
have many years of production use behind them, and the Zope 3 project gives
707
access to these components to the wider Python community. There is even a
708
separate framework based on the Zope components: `Grok
709
<http://grok.zope.org/>`_.
711
Zope is also the infrastructure used by the `Plone <http://plone.org/>`_ content
712
management system, one of the most powerful and popular content management
665
716
Other notable frameworks
666
717
^^^^^^^^^^^^^^^^^^^^^^^^
668
These two are of course not the only frameworks that are available, there are
669
also some less-popular frameworks worth mentioning.
671
One of these is the already mentioned Zope, which has been around for quite a
672
long time. With Zope 2.x having been known as rather un-pythonic, the newer
673
Zope 3.x tries to change that and therefore gets more acceptance from Python
674
programmers. These efforts already showed results, there is a project which
675
connects Zope with WSGI called `Repoze <http://repoze.org/>`_ and another
676
project called `Grok <http://grok.zope.org/>`_ which makes it possible for
677
"normal" Python programmers use the very mature Zope components.
719
Of course these are not the only frameworks that are available. There are
720
many other frameworks worth mentioning.
679
722
Another framework that's already been mentioned is `Pylons`_. Pylons is much
680
like TurboGears with ab even stronger emphasis on flexibility, which is bought
723
like TurboGears, but with an even stronger emphasis on flexibility, which comes
681
724
at the cost of being more difficult to use. Nearly every component can be
682
725
exchanged, which makes it necessary to use the documentation of every single
683
component, because there are so many Pylons combinations possible that can
684
satisfy every requirement. Pylons builds upon `Paste
726
component, of which there are many. Pylons builds upon `Paste
685
727
<http://pythonpaste.org/>`_, an extensive set of tools which are handy for WSGI.
687
729
And that's still not everything. The most up-to-date information can always be
added
removed
Doc/howto/webservers.rst
