1
.. _howto-custom-model-fields:
3
===========================
4
Writing custom model fields
5
===========================
12
The :ref:`model reference <topics-db-models>` documentation explains how to use
13
Django's standard field classes -- :class:`~django.db.models.CharField`,
14
:class:`~django.db.models.DateField`, etc. For many purposes, those classes are
15
all you'll need. Sometimes, though, the Django version won't meet your precise
16
requirements, or you'll want to use a field that is entirely different from
17
those shipped with Django.
19
Django's built-in field types don't cover every possible database column type --
20
only the common types, such as ``VARCHAR`` and ``INTEGER``. For more obscure
21
column types, such as geographic polygons or even user-created types such as
22
`PostgreSQL custom types`_, you can define your own Django ``Field`` subclasses.
24
.. _PostgreSQL custom types: http://www.postgresql.org/docs/8.2/interactive/sql-createtype.html
26
Alternatively, you may have a complex Python object that can somehow be
27
serialized to fit into a standard database column type. This is another case
28
where a ``Field`` subclass will help you use your object with your models.
33
Creating custom fields requires a bit of attention to detail. To make things
34
easier to follow, we'll use a consistent example throughout this document.
35
Suppose you have a Python object representing the deal of cards in a hand of
36
Bridge_. (Don't worry, you don't know how to play Bridge to follow this
37
example. You only need to know that 52 cards are dealt out equally to four
38
players, who are traditionally called *north*, *east*, *south* and *west*.)
39
Our class looks something like this::
42
def __init__(self, north, east, south, west):
43
# Input parameters are lists of cards ('Ah', '9s', etc)
49
# ... (other possibly useful methods omitted) ...
51
.. _Bridge: http://en.wikipedia.org/wiki/Contract_bridge
53
This is just an ordinary Python class, with nothing Django-specific about it.
54
We'd like to be able to do things like this in our models (we assume the
55
``hand`` attribute on the model is an instance of ``Hand``)::
57
example = MyModel.objects.get(pk=1)
58
print example.hand.north
60
new_hand = Hand(north, east, south, west)
61
example.hand = new_hand
64
We assign to and retrieve from the ``hand`` attribute in our model just like
65
any other Python class. The trick is to tell Django how to handle saving and
66
loading such an object.
68
In order to use the ``Hand`` class in our models, we **do not** have to change
69
this class at all. This is ideal, because it means you can easily write
70
model support for existing classes where you cannot change the source code.
73
You might only be wanting to take advantage of custom database column
74
types and deal with the data as standard Python types in your models;
75
strings, or floats, for example. This case is similar to our ``Hand``
76
example and we'll note any differences as we go along.
84
The simplest way to think of a model field is that it provides a way to take a
85
normal Python object -- string, boolean, ``datetime``, or something more
86
complex like ``Hand`` -- and convert it to and from a format that is useful
87
when dealing with the database (and serialization, but, as we'll see later,
88
that falls out fairly naturally once you have the database side under control).
90
Fields in a model must somehow be converted to fit into an existing database
91
column type. Different databases provide different sets of valid column types,
92
but the rule is still the same: those are the only types you have to work
93
with. Anything you want to store in the database must fit into one of
96
Normally, you're either writing a Django field to match a particular database
97
column type, or there's a fairly straightforward way to convert your data to,
100
For our ``Hand`` example, we could convert the card data to a string of 104
101
characters by concatenating all the cards together in a pre-determined order --
102
say, all the *north* cards first, then the *east*, *south* and *west* cards. So
103
``Hand`` objects can be saved to text or character columns in the database.
105
What does a field class do?
106
---------------------------
108
All of Django's fields (and when we say *fields* in this document, we always
109
mean model fields and not :ref:`form fields <ref-forms-fields>`) are subclasses
110
of :class:`django.db.models.Field`. Most of the information that Django records
111
about a field is common to all fields -- name, help text, uniqueness and so
112
forth. Storing all that information is handled by ``Field``. We'll get into the
113
precise details of what ``Field`` can do later on; for now, suffice it to say
114
that everything descends from ``Field`` and then customizes key pieces of the
117
It's important to realize that a Django field class is not what is stored in
118
your model attributes. The model attributes contain normal Python objects. The
119
field classes you define in a model are actually stored in the ``Meta`` class
120
when the model class is created (the precise details of how this is done are
121
unimportant here). This is because the field classes aren't necessary when
122
you're just creating and modifying attributes. Instead, they provide the
123
machinery for converting between the attribute value and what is stored in the
124
database or sent to the :ref:`serializer <topics-serialization>`.
126
Keep this in mind when creating your own custom fields. The Django ``Field``
127
subclass you write provides the machinery for converting between your Python
128
instances and the database/serializer values in various ways (there are
129
differences between storing a value and using a value for lookups, for
130
example). If this sounds a bit tricky, don't worry -- it will become clearer in
131
the examples below. Just remember that you will often end up creating two
132
classes when you want a custom field:
134
* The first class is the Python object that your users will manipulate.
135
They will assign it to the model attribute, they will read from it for
136
displaying purposes, things like that. This is the ``Hand`` class in our
139
* The second class is the ``Field`` subclass. This is the class that knows
140
how to convert your first class back and forth between its permanent
141
storage form and the Python form.
143
Writing a field subclass
144
========================
146
When planning your :class:`~django.db.models.Field` subclass, first give some
147
thought to which existing :class:`~django.db.models.Field` class your new field
148
is most similar to. Can you subclass an existing Django field and save yourself
149
some work? If not, you should subclass the :class:`~django.db.models.Field`
150
class, from which everything is descended.
152
Initializing your new field is a matter of separating out any arguments that are
153
specific to your case from the common arguments and passing the latter to the
154
:meth:`~django.db.models.Field.__init__` method of
155
:class:`~django.db.models.Field` (or your parent class).
157
In our example, we'll call our field ``HandField``. (It's a good idea to call
158
your :class:`~django.db.models.Field` subclass ``<Something>Field``, so it's
159
easily identifiable as a :class:`~django.db.models.Field` subclass.) It doesn't
160
behave like any existing field, so we'll subclass directly from
161
:class:`~django.db.models.Field`::
163
from django.db import models
165
class HandField(models.Field):
166
def __init__(self, *args, **kwargs):
167
kwargs['max_length'] = 104
168
super(HandField, self).__init__(*args, **kwargs)
170
Our ``HandField`` accept most of the standard field options (see the list
171
below), but we ensure it has a fixed length, since it only needs to hold 52
172
card values plus their suits; 104 characters in total.
175
Many of Django's model fields accept options that they don't do anything
176
with. For example, you can pass both
177
:attr:`~django.db.models.Field.editable` and
178
:attr:`~django.db.models.Field.auto_now` to a
179
:class:`django.db.models.DateField` and it will simply ignore the
180
:attr:`~django.db.models.Field.editable` parameter
181
(:attr:`~django.db.models.Field.auto_now` being set implies
182
``editable=False``). No error is raised in this case.
184
This behavior simplifies the field classes, because they don't need to
185
check for options that aren't necessary. They just pass all the options to
186
the parent class and then don't use them later on. It's up to you whether
187
you want your fields to be more strict about the options they select, or
188
to use the simpler, more permissive behavior of the current fields.
190
The :meth:`~django.db.models.Field.__init__` method takes the following
193
* :attr:`~django.db.models.Field.verbose_name`
194
* :attr:`~django.db.models.Field.name`
195
* :attr:`~django.db.models.Field.primary_key`
196
* :attr:`~django.db.models.Field.max_length`
197
* :attr:`~django.db.models.Field.unique`
198
* :attr:`~django.db.models.Field.blank`
199
* :attr:`~django.db.models.Field.null`
200
* :attr:`~django.db.models.Field.db_index`
201
* :attr:`~django.db.models.Field.core`
202
* :attr:`~django.db.models.Field.rel`: Used for related fields (like
203
:class:`ForeignKey`). For advanced use only.
204
* :attr:`~django.db.models.Field.default`
205
* :attr:`~django.db.models.Field.editable`
206
* :attr:`~django.db.models.Field.serialize`: If ``False``, the field will
207
not be serialized when the model is passed to Django's :ref:`serializers
208
<topics-serialization>`. Defaults to ``True``.
209
* :attr:`~django.db.models.Field.prepopulate_from`
210
* :attr:`~django.db.models.Field.unique_for_date`
211
* :attr:`~django.db.models.Field.unique_for_month`
212
* :attr:`~django.db.models.Field.unique_for_year`
213
* :attr:`~django.db.models.Field.choices`
214
* :attr:`~django.db.models.Field.help_text`
215
* :attr:`~django.db.models.Field.db_column`
216
* :attr:`~django.db.models.Field.db_tablespace`: Currently only used with
217
the Oracle backend and only for index creation. You can usually ignore
220
All of the options without an explanation in the above list have the same
221
meaning they do for normal Django fields. See the :ref:`field documentation
222
<ref-models-fields>` for examples and details.
224
The ``SubfieldBase`` metaclass
225
------------------------------
227
As we indicated in the introduction_, field subclasses are often needed for
228
two reasons: either to take advantage of a custom database column type, or to
229
handle complex Python types. Obviously, a combination of the two is also
230
possible. If you're only working with custom database column types and your
231
model fields appear in Python as standard Python types direct from the
232
database backend, you don't need to worry about this section.
234
If you're handling custom Python types, such as our ``Hand`` class, we need to
235
make sure that when Django initializes an instance of our model and assigns a
236
database value to our custom field attribute, we convert that value into the
237
appropriate Python object. The details of how this happens internally are a
238
little complex, but the code you need to write in your ``Field`` class is
239
simple: make sure your field subclass uses a special metaclass:
241
.. class:: django.db.models.SubfieldBase
245
class HandField(models.Field):
246
__metaclass__ = models.SubfieldBase
248
def __init__(self, *args, **kwargs):
251
This ensures that the :meth:`to_python` method, documented below, will always be
252
called when the attribute is initialized.
257
Once you've created your :class:`~django.db.models.Field` subclass and set up up
258
the ``__metaclass__``, you might consider overriding a few standard methods,
259
depending on your field's behavior. The list of methods below is in
260
approximately decreasing order of importance, so start from the top.
262
Custom database types
263
~~~~~~~~~~~~~~~~~~~~~
265
.. method:: db_type(self)
267
Returns the database column data type for the :class:`~django.db.models.Field`,
268
taking into account the current :setting:`DATABASE_ENGINE` setting.
270
Say you've created a PostgreSQL custom type called ``mytype``. You can use this
271
field with Django by subclassing ``Field`` and implementing the :meth:`db_type`
274
from django.db import models
276
class MytypeField(models.Field):
280
Once you have ``MytypeField``, you can use it in any model, just like any other
283
class Person(models.Model):
284
name = models.CharField(max_length=80)
285
gender = models.CharField(max_length=1)
286
something_else = MytypeField()
288
If you aim to build a database-agnostic application, you should account for
289
differences in database column types. For example, the date/time column type
290
in PostgreSQL is called ``timestamp``, while the same column in MySQL is called
291
``datetime``. The simplest way to handle this in a ``db_type()`` method is to
292
import the Django settings module and check the :setting:`DATABASE_ENGINE` setting.
295
class MyDateField(models.Field):
297
from django.conf import settings
298
if settings.DATABASE_ENGINE == 'mysql':
303
The :meth:`db_type` method is only called by Django when the framework
304
constructs the ``CREATE TABLE`` statements for your application -- that is, when
305
you first create your tables. It's not called at any other time, so it can
306
afford to execute slightly complex code, such as the :setting:`DATABASE_ENGINE`
307
check in the above example.
309
Some database column types accept parameters, such as ``CHAR(25)``, where the
310
parameter ``25`` represents the maximum column length. In cases like these,
311
it's more flexible if the parameter is specified in the model rather than being
312
hard-coded in the ``db_type()`` method. For example, it wouldn't make much
313
sense to have a ``CharMaxlength25Field``, shown here::
315
# This is a silly example of hard-coded parameters.
316
class CharMaxlength25Field(models.Field):
321
class MyModel(models.Model):
323
my_field = CharMaxlength25Field()
325
The better way of doing this would be to make the parameter specifiable at run
326
time -- i.e., when the class is instantiated. To do that, just implement
327
:meth:`django.db.models.Field.__init__`, like so::
329
# This is a much more flexible example.
330
class BetterCharField(models.Field):
331
def __init__(self, max_length, *args, **kwargs):
332
self.max_length = max_length
333
super(BetterCharField, self).__init__(*args, **kwargs)
336
return 'char(%s)' % self.max_length
339
class MyModel(models.Model):
341
my_field = BetterCharField(25)
343
Finally, if your column requires truly complex SQL setup, return ``None`` from
344
:meth:`db_type`. This will cause Django's SQL creation code to skip over this
345
field. You are then responsible for creating the column in the right table in
346
some other way, of course, but this gives you a way to tell Django to get out of
349
Converting database values to Python objects
350
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
352
.. method:: to_python(self, value)
354
Converts a value as returned by your database (or a serializer) to a Python
357
The default implementation simply returns ``value``, for the common case in
358
which the database backend already returns data in the correct format (as a
359
Python string, for example).
361
If your custom :class:`~django.db.models.Field` class deals with data structures
362
that are more complex than strings, dates, integers or floats, then you'll need
363
to override this method. As a general rule, the method should deal gracefully
364
with any of the following arguments:
366
* An instance of the correct type (e.g., ``Hand`` in our ongoing example).
368
* A string (e.g., from a deserializer).
370
* Whatever the database returns for the column type you're using.
372
In our ``HandField`` class, we're storing the data as a VARCHAR field in the
373
database, so we need to be able to process strings and ``Hand`` instances in
378
class HandField(models.Field):
381
def to_python(self, value):
382
if isinstance(value, Hand):
386
p1 = re.compile('.{26}')
387
p2 = re.compile('..')
388
args = [p2.findall(x) for x in p1.findall(value)]
391
Notice that we always return a ``Hand`` instance from this method. That's the
392
Python object type we want to store in the model's attribute.
394
**Remember:** If your custom field needs the :meth:`to_python` method to be
395
called when it is created, you should be using `The SubfieldBase metaclass`_
396
mentioned earlier. Otherwise :meth:`to_python` won't be called automatically.
398
Converting Python objects to database values
399
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
401
.. method:: get_db_prep_value(self, value)
403
This is the reverse of :meth:`to_python` when working with the database backends
404
(as opposed to serialization). The ``value`` parameter is the current value of
405
the model's attribute (a field has no reference to its containing model, so it
406
cannot retrieve the value itself), and the method should return data in a format
407
that can be used as a parameter in a query for the database backend.
411
class HandField(models.Field):
414
def get_db_prep_value(self, value):
415
return ''.join([''.join(l) for l in (value.north,
416
value.east, value.south, value.west)])
418
.. method:: get_db_prep_save(self, value)
420
Same as the above, but called when the Field value must be *saved* to the
421
database. As the default implementation just calls ``get_db_prep_value``, you
422
shouldn't need to implement this method unless your custom field need a special
423
conversion when being saved that is not the same as the used for normal query
424
parameters (which is implemented by ``get_db_prep_value``).
426
Preprocessing values before saving
427
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
429
.. method:: pre_save(self, model_instance, add)
431
This method is called just prior to :meth:`get_db_prep_save` and should return
432
the value of the appropriate attribute from ``model_instance`` for this field.
433
The attribute name is in ``self.attname`` (this is set up by
434
:class:`~django.db.models.Field`). If the model is being saved to the database
435
for the first time, the ``add`` parameter will be ``True``, otherwise it will be
438
You only need to override this method if you want to preprocess the value
439
somehow, just before saving. For example, Django's
440
`:class:`~django.db.models.DateTimeField` uses this method to set the attribute
441
correctly in the case of :attr:`~django.db.models.Field.auto_now` or
442
:attr:`~django.db.models.Field.auto_now_add`.
444
If you do override this method, you must return the value of the attribute at
445
the end. You should also update the model's attribute if you make any changes
446
to the value so that code holding references to the model will always see the
449
Preparing values for use in database lookups
450
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
452
.. method:: get_db_prep_lookup(self, lookup_type, value)
454
Prepares the ``value`` for passing to the database when used in a lookup (a
455
``WHERE`` constraint in SQL). The ``lookup_type`` will be one of the valid
456
Django filter lookups: ``exact``, ``iexact``, ``contains``, ``icontains``,
457
``gt``, ``gte``, ``lt``, ``lte``, ``in``, ``startswith``, ``istartswith``,
458
``endswith``, ``iendswith``, ``range``, ``year``, ``month``, ``day``,
459
``isnull``, ``search``, ``regex``, and ``iregex``.
461
Your method must be prepared to handle all of these ``lookup_type`` values and
462
should raise either a ``ValueError`` if the ``value`` is of the wrong sort (a
463
list when you were expecting an object, for example) or a ``TypeError`` if
464
your field does not support that type of lookup. For many fields, you can get
465
by with handling the lookup types that need special handling for your field
466
and pass the rest of the :meth:`get_db_prep_lookup` method of the parent class.
468
If you needed to implement ``get_db_prep_save()``, you will usually need to
469
implement ``get_db_prep_lookup()``. If you don't, ``get_db_prep_value`` will be
470
called by the default implementation, to manage ``exact``, ``gt``, ``gte``,
471
``lt``, ``lte``, ``in`` and ``range`` lookups.
473
You may also want to implement this method to limit the lookup types that could
474
be used with your custom field type.
476
Note that, for ``range`` and ``in`` lookups, ``get_db_prep_lookup`` will receive
477
a list of objects (presumably of the right type) and will need to convert them
478
to a list of things of the right type for passing to the database. Most of the
479
time, you can reuse ``get_db_prep_value()``, or at least factor out some common
482
For example, the following code implements ``get_db_prep_lookup`` to limit the
483
accepted lookup types to ``exact`` and ``in``::
485
class HandField(models.Field):
488
def get_db_prep_lookup(self, lookup_type, value):
489
# We only handle 'exact' and 'in'. All others are errors.
490
if lookup_type == 'exact':
491
return [self.get_db_prep_value(value)]
492
elif lookup_type == 'in':
493
return [self.get_db_prep_value(v) for v in value]
495
raise TypeError('Lookup type %r not supported.' % lookup_type)
497
Specifying the form field for a model field
498
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
500
.. method:: formfield(self, form_class=forms.CharField, **kwargs)
502
Returns the default form field to use when this field is displayed in a model.
503
This method is called by the :class:`~django.forms.ModelForm` helper.
505
All of the ``kwargs`` dictionary is passed directly to the form field's
506
:meth:`~django.forms.Field__init__` method. Normally, all you need to do is
507
set up a good default for the ``form_class`` argument and then delegate further
508
handling to the parent class. This might require you to write a custom form
509
field (and even a form widget). See the :ref:`forms documentation
510
<topics-forms-index>` for information about this, and take a look at the code in
511
:mod:`django.contrib.localflavor` for some examples of custom widgets.
513
Continuing our ongoing example, we can write the :meth:`formfield` method as::
515
class HandField(models.Field):
518
def formfield(self, **kwargs):
519
# This is a fairly standard way to set up some defaults
520
# while letting the caller override them.
521
defaults = {'form_class': MyFormField}
522
defaults.update(kwargs)
523
return super(HandField, self).formfield(**defaults)
525
This assumes we're imported a ``MyFormField`` field class (which has its own
526
default widget). This document doesn't cover the details of writing custom form
529
.. _helper functions: ../forms/#generating-forms-for-models
530
.. _forms documentation: ../forms/
532
Emulating built-in field types
533
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
535
.. method:: get_internal_type(self)
537
Returns a string giving the name of the :class:`~django.db.models.Field`
538
subclass we are emulating at the database level. This is used to determine the
539
type of database column for simple cases.
541
If you have created a :meth:`db_type` method, you don't need to worry about
542
:meth:`get_internal_type` -- it won't be used much. Sometimes, though, your
543
database storage is similar in type to some other field, so you can use that
544
other field's logic to create the right column.
548
class HandField(models.Field):
551
def get_internal_type(self):
554
No matter which database backend we are using, this will mean that ``syncdb``
555
and other SQL commands create the right column type for storing a string.
557
If :meth:`get_internal_type` returns a string that is not known to Django for
558
the database backend you are using -- that is, it doesn't appear in
559
``django.db.backends.<db_name>.creation.DATA_TYPES`` -- the string will still be
560
used by the serializer, but the default :meth:`db_type` method will return
561
``None``. See the documentation of :meth:`db_type` for reasons why this might be
562
useful. Putting a descriptive string in as the type of the field for the
563
serializer is a useful idea if you're ever going to be using the serializer
564
output in some other place, outside of Django.
566
Converting field data for serialization
567
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
569
.. method:: value_to_string(self, obj)
571
This method is used by the serializers to convert the field into a string for
572
output. Calling :meth:``Field._get_val_from_obj(obj)`` is the best way to get the
573
value to serialize. For example, since our ``HandField`` uses strings for its
574
data storage anyway, we can reuse some existing conversion code::
576
class HandField(models.Field):
579
def value_to_string(self, obj):
580
value = self._get_val_from_obj(obj)
581
return self.get_db_prep_value(value)
586
Writing a custom field can be a tricky process, particularly if you're doing
587
complex conversions between your Python types and your database and
588
serialization formats. Here are a couple of tips to make things go more
591
1. Look at the existing Django fields (in
592
:file:`django/db/models/fields/__init__.py`) for inspiration. Try to find
593
a field that's similar to what you want and extend it a little bit,
594
instead of creating an entirely new field from scratch.
596
2. Put a :meth:`__str__` or :meth:`__unicode__` method on the class you're
597
wrapping up as a field. There are a lot of places where the default
598
behavior of the field code is to call
599
:func:`~django.utils.encoding.force_unicode` on the value. (In our
600
examples in this document, ``value`` would be a ``Hand`` instance, not a
601
``HandField``). So if your :meth:`__unicode__` method automatically
602
converts to the string form of your Python object, you can save yourself
606
Writing a ``FileField`` subclass
607
=================================
609
In addition to the above methods, fields that deal with files have a few other
610
special requirements which must be taken into account. The majority of the
611
mechanics provided by ``FileField``, such as controlling database storage and
612
retrieval, can remain unchanged, leaving subclasses to deal with the challenge
613
of supporting a particular type of file.
615
Django provides a ``File`` class, which is used as a proxy to the file's
616
contents and operations. This can be subclassed to customize how the file is
617
accessed, and what methods are available. It lives at
618
``django.db.models.fields.files``, and its default behavior is explained in the
619
:ref:`file documentation <ref-files-file>`.
621
Once a subclass of ``File`` is created, the new ``FileField`` subclass must be
622
told to use it. To do so, simply assign the new ``File`` subclass to the special
623
``attr_class`` attribute of the ``FileField`` subclass.
628
In addition to the above details, there are a few guidelines which can greatly
629
improve the efficiency and readability of the field's code.
631
1. The source for Django's own ``ImageField`` (in
632
``django/db/models/fields/files.py``) is a great example of how to
633
subclass ``FileField`` to support a particular type of file, as it
634
incorporates all of the techniques described above.
636
2. Cache file attributes wherever possible. Since files may be stored in
637
remote storage systems, retrieving them may cost extra time, or even
638
money, that isn't always necessary. Once a file is retrieved to obtain
639
some data about its content, cache as much of that data as possible to
640
reduce the number of times the file must be retrieved on subsequent
641
calls for that information.
added
removed
docs/howto/custom-model-fields.txt
