5
:Contact: goodger@users.sourceforge.net
6
:Revision: $Revision: 2223 $
7
:Date: $Date: 2004-06-05 21:32:15 +0200 (Sat, 05 Jun 2004) $
8
:Copyright: This document has been placed in the public domain.
10
This document explores issues around extracting and processing
11
docstrings from Python modules.
13
For definitive element hierarchy details, see the "Python Plaintext
14
Document Interface DTD" XML document type definition, pysource.dtd_
15
(which modifies the generic docutils.dtd_). Descriptions below list
16
'DTD elements' (XML 'generic identifiers' or tag names) corresponding
26
The Python Source Reader ("PySource") model that's evolving in my mind
27
goes something like this:
29
1. Extract the docstring/namespace [#]_ tree from the module(s) and/or
32
.. [#] See `Docstring Extractor`_ below.
34
2. Run the parser on each docstring in turn, producing a forest of
35
doctrees (per nodes.py).
37
3. Join the docstring trees together into a single tree, running
42
- create various sections like "Module Attributes", "Functions",
43
"Classes", "Class Attributes", etc.; see pysource.dtd_
44
- convert the above special sections to ordinary doctree nodes
46
4. Run transforms on the combined doctree. Examples: resolving
47
cross-references/hyperlinks (including interpreted text on Python
48
identifiers); footnote auto-numbering; first field list ->
49
bibliographic elements.
51
(Or should step 4's transforms come before step 3?)
53
5. Pass the resulting unified tree to the writer/builder.
55
I've had trouble reconciling the roles of input parser and output
56
writer with the idea of modes ("readers" or "directors"). Does the
57
mode govern the tranformation of the input, the output, or both?
58
Perhaps the mode should be split into two.
60
For example, say the source of our input is a Python module. Our
61
"input mode" should be the "Python Source Reader". It discovers (from
62
``__docformat__``) that the input parser is "reStructuredText". If we
63
want HTML, we'll specify the "HTML" output formatter. But there's a
64
piece missing. What *kind* or *style* of HTML output do we want?
65
PyDoc-style, LibRefMan style, etc. (many people will want to specify
66
and control their own style). Is the output style specific to a
67
particular output format (XML, HTML, etc.)? Is the style specific to
68
the input mode? Or can/should they be independent?
70
I envision interaction between the input parser, an "input mode" , and
71
the output formatter. The same intermediate data format would be used
72
between each of these, being transformed as it progresses.
78
We need code that scans a parsed Python module, and returns an ordered
79
tree containing the names, docstrings (including attribute and
80
additional docstrings), and additional info (in parentheses below) of
81
all of the following objects:
85
- module attributes (+ values)
86
- classes (+ inheritance)
87
- class attributes (+ values)
88
- instance attributes (+ values)
89
- methods (+ formal parameters & defaults)
90
- functions (+ formal parameters & defaults)
92
(Extract comments too? For example, comments at the start of a module
93
would be a good place for bibliographic field lists.)
95
In order to evaluate interpreted text cross-references, namespaces for
96
each of the above will also be required.
98
See python-dev/docstring-develop thread "AST mining", started on
105
DTD elements: package, module, class, method, function,
106
module_attribute, class_attribute, instance_attribute, variable,
107
parameter, type, exception_class, warning_class.
109
To classify identifiers explicitly, the role is given along with the
110
identifier in either prefix or suffix form::
112
Use :method:`Keeper.storedata` to store the object's data in
113
`Keeper.data`:instance_attribute:.
115
The role may be one of 'package', 'module', 'class', 'method',
116
'function', 'module_attribute', 'class_attribute',
117
'instance_attribute', 'variable', 'parameter', 'type',
118
'exception_class', 'exception', 'warning_class', or 'warning'. Other
119
roles may be defined.
121
.. _pysource.dtd: pysource.dtd
122
.. _docutils.dtd: ../ref/docutils.dtd
128
indent-tabs-mode: nil
added
removed
docs/dev/pysource.txt
