1
================================
2
Docutils_ Internationalization
3
================================
6
:Contact: goodger@users.sourceforge.net
7
:Date: $Date: 2004/04/17 17:38:28 $
8
:Revision: $Revision: 1.11 $
9
:Copyright: This document has been placed in the public domain.
15
This document describes the internationalization facilities of the
16
Docutils_ project. `Introduction to i18n`_ by Tomohiro KUBOTA is a
17
good general reference. "Internationalization" is often abbreviated
18
as "i18n": "i" + 18 letters + "n".
22
The i18n facilities of Docutils should be considered a "first
23
draft". They work so far, but improvements are welcome.
24
Specifically, standard i18n facilities like "gettext" have yet to
27
Docutils is designed to work flexibly with text in multiple languages
28
(one language at a time). Language-specific features are (or should
29
be [#]_) fully parameterized. To enable a new language, two modules
30
have to be added to the project: one for Docutils itself (the
31
`Docutils Language Module`_) and one for the reStructuredText parser
32
(the `reStructuredText Language Module`_).
34
.. [#] If anything in Docutils is insufficiently parameterized, it
35
should be considered a bug. Please report bugs to the Docutils
36
project bug tracker on SourceForge at
37
http://sourceforge.net/tracker/?group_id=38414&atid=422030.
39
.. _Docutils: http://docutils.sourceforge.net/
40
.. _Introduction to i18n:
41
http://www.debian.org/doc/manuals/intro-i18n/
47
Language modules are named using a case-insensitive language
48
identifier as defined in `RFC 1766`_, converting hyphens to
49
underscores [#]_. A typical language identifier consists of a
50
2-letter language code from `ISO 639`_ (3-letter codes can be used if
51
no 2-letter code exists; RFC 1766 is currently being revised to allow
52
3-letter codes). The language identifier can have an optional subtag,
53
typically for variations based on country (from `ISO 3166`_ 2-letter
54
country codes). If no language identifier is specified, the default
55
is "en" for English. Examples of module names include ``en.py``,
56
``fr.py``, ``ja.py``, and ``pt_br.py``.
58
.. [#] Subtags are separated from primary tags by underscores instead
59
of hyphens, to conform to Python naming rules.
61
.. _RFC 1766: http://www.faqs.org/rfcs/rfc1766.html
62
.. _ISO 639: http://lcweb.loc.gov/standards/iso639-2/englangn.html
63
.. _ISO 3166: http://www.iso.ch/iso/en/prods-services/iso3166ma/
64
02iso-3166-code-lists/index.html
70
All Python code in Docutils will be ASCII-only. In language modules,
71
Unicode-escapes will have to be used for non-ASCII characters.
72
Although it may be possible for some developers to store non-ASCII
73
characters directly in strings, it will cause problems for other
74
developers whose locales are set up differently.
76
`PEP 263`_ introduces source code encodings to Python modules,
77
implemented beginning in Python 2.3. Until PEP 263 is fully
78
implemented as a well-established convention, proven robust in daily
79
use, and the tools (editors, CVS, email, etc.) recognize this
80
convention, Docutils shall remain conservative.
82
As mentioned in the note above, developers are invited to explore
83
"gettext" and other i18n technologies.
85
.. _PEP 263: http://www.python.org/peps/pep-0263.html
88
Docutils Language Module
89
========================
91
Modules in ``docutils/languages`` contain language mappings for
92
markup-independent language-specific features of Docutils. To make a
93
new language module, just copy the ``en.py`` file, rename it with the
94
code for your language (see `Language Module Names`_ above), and
95
translate the terms as described below.
97
Each Docutils language module contains three module attributes:
100
This is a mapping of node class names to language-dependent
101
boilerplate label text. The label text is used by Writer
102
components when they encounter document tree elements whose class
103
names are the mapping keys.
105
The entry values (*not* the keys) should be translated to the
108
``bibliographic_fields``
109
This is a mapping of language-dependent field names (converted to
110
lower case) to canonical field names (keys of
111
``DocInfo.biblio_notes`` in ``docutils.transforms.frontmatter``).
112
It is used when transforming bibliographic fields.
114
The keys should be translated to the target language.
116
``author_separators``
117
This is a list of strings used to parse the 'Authors'
118
bibliographic field. They separate individual authors' names, and
119
are tried in order (i.e., earlier items take priority, and the
120
first item that matches wins). The English-language module
121
defines them as ``[';', ',']``; semi-colons can be used to
122
separate names like "Arthur Pewtie, Esq.".
124
Most languages won't have to "translate" this list.
127
reStructuredText Language Module
128
================================
130
Modules in ``docutils/parsers/rst/languages`` contain language
131
mappings for language-specific features of the reStructuredText
132
parser. To make a new language module, just copy the ``en.py`` file,
133
rename it with the code for your language (see `Language Module
134
Names`_ above), and translate the terms as described below.
136
Each reStructuredText language module contains just one module
140
This is a mapping from language-dependent directive names to
141
canonical directive names. The canonical directive names are
142
registered in ``docutils/parsers/rst/directives/__init__.py``, in
143
``_directive_registry``.
145
The keys should be translated to the target language. Synonyms
146
(multiple keys with the same values) are allowed; this is useful
150
This is a mapping language-dependent role names to canonical role
151
names for interpreted text. The canonical directive names are
152
registered in ``docutils/parsers/rst/states.py``, in
153
``Inliner._interpreted_roles`` (this may change).
155
The keys should be translated to the target language. Synonyms
156
(multiple keys with the same values) are allowed; this is useful
160
Testing the Language Modules
161
============================
163
Whenever a new language module is added or an existing one modified,
164
the unit tests should be run. The test modules can be found in the
165
docutils/test directory from CVS_ or from the `latest CVS snapshot`_.
167
The ``test_language.py`` module can be run as a script. With no
168
arguments, it will test all language modules. With one or more
169
language codes, it will test just those languages. For example::
171
$ python test_language.py en
173
----------------------------------------
174
Ran 2 tests in 0.095s
178
Use the "alltests.py" script to run all test modules, exhaustively
179
testing the parser and other parts of the Docutils system.
181
.. _CVS: http://sourceforge.net/cvs/?group_id=38414
182
.. _latest CVS snapshot: http://docutils.sf.net/docutils-snapshot.tgz
added
removed
spec/howto/i18n.txt
