~ubuntu-branches/ubuntu/karmic/pypy/karmic

<div id="menubar"><a class="menu" href="http://codespeak.net/pypy/dist/pypy/doc/news.html">news</a> <a class="menu" href="getting-started.html">getting-started</a> <a class="menu" href="index.html">documentation</a> <a class="menu" href="https://codespeak.net/viewvc/pypy/dist/">svn</a> <a class="menu" href="https://codespeak.net/issue/pypy-dev/">issues</a><a class="menu" href="contact.html">contact</a> <a class="menu" href="http://pypy.org/">EU/project</a> </div></div>

<h1 class="title">Independent project ideas relating to PyPy</h1>

<p>PyPy allows experimentation in many directions -- indeed facilitating

experimentation in language implementation was one of the main

motivations for the project. This page is meant to collect some ideas

of experiments that the core developers have not had time to perform

yet and also do not require too much in depth knowledge to get started

with.</p>

<p>Feel free to suggest new ideas and discuss them in #pypy on the freenode IRC

network or the pypy-dev mailing list (see the <a class="reference" href="contact.html">contact</a> page).</p>

<p class="topic-title first"><a id="contents" name="contents">Contents</a></p>

<li><a class="reference" href="#experiment-with-optimizations" id="id1" name="id1">Experiment with optimizations</a></li>

<li><a class="reference" href="#start-or-improve-a-back-end" id="id2" name="id2">Start or improve a back-end</a></li>

<li><a class="reference" href="#improve-javascript-backend" id="id3" name="id3">Improve JavaScript backend</a></li>

<li><a class="reference" href="#improve-one-of-the-jit-back-ends" id="id4" name="id4">Improve one of the JIT back-ends</a></li>

<li><a class="reference" href="#write-a-new-front-end" id="id5" name="id5">Write a new front end</a></li>

<li><a class="reference" href="#investigate-restricted-execution-models" id="id6" name="id6">Investigate restricted execution models</a></li>

<li><a class="reference" href="#experiment-with-distribution-and-persistence" id="id7" name="id7">Experiment with distribution and persistence</a></li>

<li><a class="reference" href="#numeric-numpy-numarray-support" id="id8" name="id8">Numeric/NumPy/numarray support</a></li>

<li><a class="reference" href="#extension-modules" id="id9" name="id9">Extension modules</a></li>

<li><a class="reference" href="#extend-py-execnet-to-a-peer-to-peer-model" id="id10" name="id10">Extend py.execnet to a peer-to-peer model</a></li>

</ul>

</div>

<h1><a class="toc-backref" href="#id1" id="experiment-with-optimizations" name="experiment-with-optimizations">Experiment with optimizations</a></h1>

<p>Although PyPy's Python interpreter is very compatible with CPython, it is not

yet as fast. There are several approaches to making it faster, including the

on-going Just-In-Time compilation efforts and improving the compilation tool

chain, but probably the most suited to being divided into reasonably sized

chunks is to play with alternate implementations of key data structures or

algorithms used by the interpreter. PyPy's structure is designed to make this

straightforward, so it is easy to provide a different implementation of, say,

dictionaries or lists without disturbing any other code.</p>

<p>As examples, we've got working implementations of things like:</p>

<li>lazy string slices (slicing a string gives an object that references a part

of the original string).</li>

<li>lazily concatenated strings (repeated additions and joins are done

incrementally).</li>

<li>dictionaries specialized for string-only keys.</li>

<li>dictionaries which use a different strategy when very small.</li>

<li>caching the lookups of builtin names (by special forms of

dictionaries that can invalidate the caches when they are written to)</li>

</ul>

<p>Things we've thought about but not yet implemented include:</p>

<li>create multiple representations of Unicode string that store the character

data in narrower arrays when they can.</li>

</ul>

<p>Experiments of this kind are really experiments in the sense that we do not know

whether they will work well or not and the only way to find out is to try. A

project of this nature should provide benchmark results (both timing and memory

usage) as much as code.</p>

<p>Some ideas on concrete steps for benchmarking:</p>

<li>find a set of real-world applications that can be used as benchmarks

for pypy (ideas: docutils, <a class="reference" href="http://hachoir.org/">http://hachoir.org/</a>, moinmoin, ...?)</li>

<li>do benchmark runs to see how much speedup the currently written

optimizations give</li>

<li>profile pypy-c and its variants with these benchmarks, identify slow areas</li>

<li>try to come up with optimized implementations for these slow areas</li>

</ul>

</div>

<h1><a class="toc-backref" href="#id2" id="start-or-improve-a-back-end" name="start-or-improve-a-back-end">Start or improve a back-end</a></h1>

<p>PyPy has complete, or nearly so, back-ends for C, LLVM, CLI/.NET and JVM and partial

backends for JavaScript, Common Lisp, Squeak. It would be an

interesting project to improve either of these partial backends, or start one

for another platform (Objective C comes to mind as one that should not be too

terribly hard).</p>

</div>

<h1><a class="toc-backref" href="#id3" id="improve-javascript-backend" name="improve-javascript-backend">Improve JavaScript backend</a></h1>

<p>The JavaScript backend is somehow different from other pypy's backends because

it does not try to support all of PyPy (where it might be run then?), but rather

to compile RPython programs into code that runs in a browser. Some documents

are in <a class="reference" href="js/whatis.html">what is PyPy.js</a> file and <a class="reference" href="js/using.html">using the JavaScript backend</a>. Some project

ideas might be:</p>

<li>Write some examples to show different possibilities of using the backend.</li>

100

<li>Improve the facilities for testing RPython intended to be translated to

101

JavaScript on top of CPython, mostly by improving existing DOM interface.</li>

102

<li>Write down the mochikit bindings (or any other interesting JS effects library),

103

including tests</li>

104

<li>Write down better object layer over DOM in RPython to make writing applications

105

easier</li>

106

</ul>

107

</div>

108

109

<h1><a class="toc-backref" href="#id4" id="improve-one-of-the-jit-back-ends" name="improve-one-of-the-jit-back-ends">Improve one of the JIT back-ends</a></h1>

110

<p>PyPy's Just-In-Time compiler relies on two assembler backends for actual code

111

generation, one for PowerPC and the other for i386. Idea would be start a new backend

112

for ie. mobile device.</p>

113

<p>Another idea in a similar vein would be to use LLVM to re-compile functions that

114

are executed particularly frequently (LLVM cannot be used for <em>all</em> code

115

generation, since it can only work on function at a time).</p>

116

</div>

117

118

<h1><a class="toc-backref" href="#id5" id="write-a-new-front-end" name="write-a-new-front-end">Write a new front end</a></h1>

119

<p>Write an interpreter for <strong>another dynamic language</strong> in the PyPy framework.

120

For example, a Scheme interpreter would be suitable (and it would even be

121

interesting from a semi-academic point of view to see if <tt class="docutils literal"><span class="pre">call/cc</span></tt> can be

122

implemented on top of the primitives the stackless transform provides). Ruby

123

too (though the latter is probably more than two months of work), or Lua, or ...</p>

124

<p>We already have a somewhat usable <a class="reference" href="http://codespeak.net/svn/user/cfbolz/hack/prolog/interpreter">Prolog interpreter</a> and the beginnings of a

125

<a class="reference" href="../../pypy/lang/js">JavaScript interpreter</a>.</p>

126

</div>

127

128

<h1><a class="toc-backref" href="#id6" id="investigate-restricted-execution-models" name="investigate-restricted-execution-models"><span id="security"></span>Investigate restricted execution models</a></h1>

129

<p>Revive <strong>rexec</strong>: implement security checks, sandboxing, or some similar

130

model within PyPy (which, if I may venture an opinion, makes more sense and is

131

more robust than trying to do it in CPython).</p>

132

<p>There are multiple approaches that can be discussed and tried. One of them is

133

about safely executing limited snippets of untrusted RPython code (see

134

<a class="reference" href="http://codespeak.net/pipermail/pypy-dev/2006q2/003131.html">http://codespeak.net/pipermail/pypy-dev/2006q2/003131.html</a>). More general

135

approaches, to execute general but untrusted Python code on top of PyPy,

136

require more design. The object space model of PyPy will easily allow

137

objects to be tagged and tracked. The translation of PyPy would also be a

138

good place to insert e.g. systematic checks around all system calls.</p>

139

</div>

140

141

<h1><a class="toc-backref" href="#id7" id="experiment-with-distribution-and-persistence" name="experiment-with-distribution-and-persistence"><span id="persistence"></span><span id="distribution"></span>Experiment with distribution and persistence</a></h1>

142

<p>One of the advantages of PyPy's implementation is that the Python-level type

143

of an object and its implementation are completely independent. This should

144

allow a much more intuitive interface to, for example, objects that are backed

145

by a persistent store.</p>

146

<p>The <a class="reference" href="objspace-proxies.html#tproxy">transparent proxy</a> objects are a key step in this

147

direction; now all that remains is to implement the interesting bits :-)</p>

148

<p>An example project might be to implement functionality akin to the <a class="reference" href="http://www.zope.org/Documentation/Books/ZDG/current/Persistence.stx">ZODB's

149

Persistent class</a>, without the need for the _p_changed hacks, and in pure

150

Python code (should be relatively easy on top of transparent proxy).</p>

151

</div>

152

153

<h1><a class="toc-backref" href="#id8" id="numeric-numpy-numarray-support" name="numeric-numpy-numarray-support">Numeric/NumPy/numarray support</a></h1>

154

<p>At the EuroPython sprint, some work was done on making RPython's annotator

155

recognise Numeric arrays, with the goal of allowing programs using them to be

156

efficiently translated. It would be a reasonably sized project to finish this

157

work, i.e. allow RPython programs to use some Numeric facilities.

158

Additionally, these facilities could be exposed to applications interpreted by

159

the translated PyPy interpreter.</p>

160

</div>

161

162

<h1><a class="toc-backref" href="#id9" id="extension-modules" name="extension-modules">Extension modules</a></h1>

163

<p>Rewrite one or several CPython extension modules to be based on <strong>ctypes</strong>

164

(integrated in Python 2.5): this is generally useful for Python

165

developers, and it is now the best path to write extension modules that are

166

compatible with both CPython and PyPy. This is done with the <a class="reference" href="extcompiler.html">extension

167

compiler</a> component of PyPy, which will likely require some attention as

168

well.</p>

169

<p>Modules where some work is already done:</p>

170

171

<li><tt class="docutils literal"><span class="pre">_socket</span></tt>, <tt class="docutils literal"><span class="pre">os</span></tt>, <tt class="docutils literal"><span class="pre">select</span></tt> (unfinished yet, feel free to help;

172

see e.g. <a class="reference" href="http://codespeak.net/svn/pypy/dist/pypy/module/_socket/">http://codespeak.net/svn/pypy/dist/pypy/module/_socket/</a>).</li>

173

<li>SSL for socket, <tt class="docutils literal"><span class="pre">bz2</span></tt>, <tt class="docutils literal"><span class="pre">fcntl</span></tt>, <tt class="docutils literal"><span class="pre">mmap</span></tt> and <tt class="docutils literal"><span class="pre">time</span></tt>: part of the

174

Summer of Code project of Lawrence Oluyede

175

(<a class="reference" href="http://codespeak.net/svn/user/rhymes/">http://codespeak.net/svn/user/rhymes/</a>).</li>

176

</ul>

177

<p>You are free to pick any other CPython module, either standard or third-party

178

(if relatively well-known, like gtk bindings).

179

Note that some modules exist in a ctypes version

180

already, which would be a good start for porting them to PyPy's extension

181

compiler.</p>

182

</div>

183

184

<h1><a class="toc-backref" href="#id10" id="extend-py-execnet-to-a-peer-to-peer-model" name="extend-py-execnet-to-a-peer-to-peer-model">Extend py.execnet to a peer-to-peer model</a></h1>

185

186

<li>Work on a P2P model of distributed execution (extending <a class="reference" href="http://codespeak.net/py/current/doc/execnet.html">py.execnet</a>)

187

that allows <a class="reference" href="http://codespeak.net/py/current/doc/test.html">py.test</a> and other upcoming utilities to make use of a

188

network of computers executing python tasks (e.g. tests or PyPy build tasks).</li>

189

<li>Make a client tool and according libraries to instantiate a (dynamic) network

190

of computers executing centrally managed tasks (e.g. build or test ones).

191

(This may make use of a P2P model or not, both is likely feasible).</li>

192

</ul>

193

</div>

194

195

196

197

<li>Constraint programming: <a class="reference" href="http://codespeak.net/svn/pypy/extradoc/soc-2006/constraints.txt">efficient propagators for specialized

198

finite domains</a> (like numbers, sets, intervals).</li>

199

<li>A <a class="reference" href="http://codespeak.net/svn/pypy/extradoc/soc-2006/code-templating.txt">code templating solution</a> for Python code, allowing to extend

200

the language syntax, control flow operators, etc.</li>

201

</ul>

202

<p>...or whatever else interests you!</p>

203

<p>Feel free to mention your interest and discuss these ideas on the <a class="reference" href="http://codespeak.net/mailman/listinfo/pypy-dev">pypy-dev

204

mailing list</a>. You can also have a look around our <a class="reference" href="index.html">documentation</a>.</p>

205

</div>

206

</div>

207

</div></body></html>

b'\\ No newline at end of file'

Older »