1
<html><head><title>A Twisted Web Tutorial</title></head><body>
3
<h1>A Twisted Web Tutorial</h1>
5
<h2>Twisted Web -- The Tutorial</h2><ul>
8
<li>Gimmick -- Buffy quotes</li>
12
<em>Sweet (Once More With Feeling, season 6) -- Showtime</em>
13
<h2>Twisted Web</h2><ul>
14
<li>Web server, using Twisted<ul><li>Like Apache, Zope...</li>
17
<li>Serve static files</li>
21
<li>Other uses...</li>
25
<em>Giles (I Robot -- You Jane, season 1) -- There's a demon in the internet</em>
26
<h2>Short Example: Putting a Server Up</h2><ul>
27
<li>Here's all you need to know to bring up a server</li></ul>
30
% mktap --uid=33 --gid=33 web --path=/var/www/htdocs --port=80
31
% sudo twistd -f web.tap
33
<ul><li>The rest of the talk will explain what that means...</li>
35
<li>...and how to do more complicated things</li>
39
<em>Buffy (Once More, With Feeling, season 6) -- I've got a theory. It doesn't matter.</em>
40
<h2>Setup and Configuration Utilities</h2><ul>
45
<li>websetroot<ul><li>Won't be covered, unless there is time</li>
50
<em>Xander (The Harvest, season 1) -- crosses, garlic, stake through the heart</em>
51
<h2>Digression: What are TAPs</h2><ul>
52
<li>Pickled 'application configuration'</li>
54
<li>Object which contains all the information about application</li>
56
<li>The canonical way to represent configurations in Twisted</li>
58
<li>Machine editable</li>
62
<em>Master (The Wish, season 3) -- Behold the technical wonder</em>
64
<li>General usage</li>
66
<li>Flexibility and Power</li>
70
<em>Buffy (Bad Eggs, season 2) -- I'm gonna need a *big* weapon</em>
71
<h2>mktap web: Common Useful Options</h2><ul>
72
<li>--path: serve from given path</li>
74
<li>--port: listen on given port</li>
76
<li>--user: serve from users' directories and personal servers</li>
78
<li>--logfile: log to NCSA compatible logfile given</li>
80
<li>--processor: add a special processor for a given extension</li>
84
<em>Buffy (Bad Eggs, season 2) -- That's probably not gonna be the winning argument, is it?</em>
87
<li>Start a Twisted Application</li>
89
<li>Loads an instance of twisted.internet.app.Application from a file</li>
91
<li>Daemonizes, binds to appropriate ports, and starts the Twisted mainloop</li>
95
<em>Giles (Teacher's Pet, season 1) -- That's all he said? Fork Guy?</em>
96
<h2>What's a Resource?</h2><ul>
97
<li>Everything represented as twisted.web.resource.Resource</li>
99
<li>Important interface:<ul><li>getChild()</li>
106
<em>Sean (Go Fish, season 3) -- You're soakin' in it, bud.</em>
107
<h2>Resource Examples</h2><ul>
108
<li>Files<ul><li>MIME</li>
113
<li>Others<ul><li>Virtual hosts</li>
115
<li>User directories</li>
120
<em>Xander (As You Were, season 6) -- We have friends, family and demons</em>
121
<h2>Web Development</h2><ul>
122
<li>Processors<ul><li>Inherited from resource.Resource</li>
124
<li>Interpret files as code rather than data</li>
127
<li>Default processors<ul><li>.php -- default PHP</li>
129
<li>.cgi -- Common Gateway Interface</li>
131
<li>.rpy -- Correct way, Python scripting</li>
133
<li>.trp -- Resource pickles</li>
137
<li>You can also write your own</li>
141
<em>Xander (Family, season 5) -- That was a tangled web</em>
142
<h2>Custom Processor</h2><ul>
143
<li>A custom processor to handle Perl CGIs (in a module called PerlScript)</li></ul>
146
from twisted.web import static, twcgi
148
class PerlScript(twcgi.FilteredScript):
149
filter = '/usr/bin/perl' # Points to the perl parser
151
<ul><li>Use:<ul><li>mktap web --path=/home/nafai/public_html --processor=.pl=PerlScript.PerlScript</li>
156
<em>Tara (Family, season 4) -- There was the front of a camel</em>
157
<h2>Resource Scripting</h2><ul>
158
<li>Subclass resource.Resource</li>
160
<li>Write a render(self, request) method<ul><li>Return string for immediate response</li>
162
<li>Return NOT_DONE_YET and write to request</li>
165
<li>Create an .rpy file that sets 'resource' to an instance</li>
169
<em>Tara (Once More, With Feeling, season 6) -- You make me complete</em>
170
<h2>.rpy example</h2>
172
from twisted.web import resource as resourcelib
174
class MyGreatResource(resourcelib.Resource):
175
def render(self, request):
176
return "<html>foo</html>"
178
resource = MyGreatResource()
181
<em>Willow (Welcome to the Hellmouth, season 1) -- It's probably easy for you.</em>
182
<h2>Alternative Configuration Formats</h2><ul>
191
<em>Ben (The Gift, season 5) -- I wish there was another way</em>
192
<h2>Alternative Configuration Formats -- Python</h2><ul>
193
<li>Write manually</li>
195
<li>Just uses twisted.web API</li>
197
<li>Possible to do anything<ul><li>Write loops</li>
199
<li>Read other files</li>
201
<li>(not recommended) Define functions or classes</li>
206
<em>Buffy (The I In Team, season 4) -- But I've learned that it pays to be flexible in life.</em>
208
<h2>Python Configuration Example</h2><ul>
210
<li>Create application</li>
212
<li>Make it listen on port 80 for web requests...</li>
214
<li>...which should be served from /var/www/htdocs</li></ul>
217
from twisted.internet import app
218
from twisted.web import static, server
220
application = app.Application('web')
221
application.listenTCP(80,
222
server.Site(static.File("/var/www/htdocs")))
225
<em>Willow (The Pack, season 1) -- It's simple, really.</em>
227
<h2>Bannerfish -- A Case Study in Deployment</h2><ul>
228
<li>Serves banner ads</li>
230
<li>Has algorithms to maintain randomness and fairness</li>
232
<li><a href="http://itamarst.org/software/bannerfish/">http://itamarst.org/software/bannerfish/</a></li>
234
<li>But how to deploy?</li>
238
<em>Xander (Halloween, season 2) -- Let's move out.</em>
239
<h2>Bannerfish -- Standalone tap</h2><ul>
240
<li>mktap bannerfish</li>
242
<li>twistd -f bannerfish.tap</li>
244
<li>For full list of options<ul><li>mktap --help bannerfish</li>
249
<em>Ethan (Halloween, season 2) -- Don't wish to blow my own trumpet, but --</em>
250
<h2>Bannerfish -- Standalone tap (behind reverse proxy)</h2><ul>
251
<li>mktap bannerfish --port 81 --proxyhost=example.com</li>
253
<li>twistd -f bannerfish.tap</li>
255
<li>Now can work on internal server behind firewall</li>
257
<li>If main server is Twisted Web, following Resource script will serve from bannerfish</li></ul>
260
resource = proxy.ReverseProxyResource('localhost', 81, '/')
263
<em>Buffy (Halloween, season 2) -- You're sweet. A terrible liar, but sweet.</em>
264
<h2>Bannerfish -- Standalone Python</h2>
266
from twisted.internet import app
267
from twisted.cred import authorizer
268
from twisted.web import server
269
from bannerfish import service
271
application = app.Application("bannerfish")
272
auth = authorizer.DefaultAuthorizer(app)
273
svc = service.BannerService('/var/bannerfish',
274
"bannerfish", application, auth)
275
site = server.Site(svc.buildResource(None, None))
276
application.listenTCP(80, site)
279
<em>Spike (Halloween, season 2) -- Shaking. Terrified. Alone. Lost little lamb.</em>
280
<h2>Bannerfish -- /etc/twisted-web/local.d Drop In</h2>
282
from twisted.cred import authorizer
283
from bannerfish import service
285
auth = authorizer.DefaultAuthorizer(app)
286
svc = service.BannerService('/var/bannerfish',
287
"bannerfish", application, auth)
288
resource = svc.buildResource(None, None)
289
default.addChild("bannerfish", resource)
292
<em>Cordelia (Halloween, season 2) -- Well, I guess you better get them back to their parents.</em>
293
<h2>Bannerfish -- Resource Script</h2>
295
from twisted.cred import authorizer
296
from twisted.internet import app
297
from bannerfish import service
299
application = registry.getComponent(app.Application)
300
auth = authorizer.DefaultAuthorizer(application)
301
svc = service.BannerService('/var/bannerfish',
302
"bannerfish", application, auth)
303
resource = svc.buildResource(None, None)
305
<ul><li>But see later, about registry</li>
309
<em>Xander (Innocence, season 2) -- They like to see the big guns.</em>
310
<h2>Bannerfish -- Distributed (Slave)</h2>
313
from twisted.internet import application
314
from twisted.cred import authorizer
315
from twisted.web import server
316
from bannerfish import service
317
application = app.Application("bannerfish")
318
auth = authorizer.DefaultAuthorizer(application)
319
svc = service.BannerService('/var/bannerfish',
320
"bannerfish", application, auth)
321
site = server.Site(svc.buildResource(None, None))
322
fact = pb.BrokerFactory(site)
323
site = server.Site(root)
324
application.listenUNIX('/var/run/bannerfish', fact)
326
<h2>Bannerfish -- Distributed (Master, Resource Script)</h2>
329
from twisted.web import distrib
331
resource = distrib.ResourceSubscription('unix',
332
'/var/run/bannerfish')
335
<em>Oz (Innocence, season 2) -- So, do you guys steal weapons from the Army a lot?</em>
336
<h2>Bannerfish -- Other options</h2><ul>
337
<li>Mix and match possible</li>
339
<li>Can serve same content multiple ways simultaneously</li>
341
<li>Might be useful as a way to serve same ads different ways</li>
343
<li>...or serve ads from several bannerfish servers...</li>
345
<li>...each deployed differently.</li>
349
<em>Buffy (Tabula Rasa, season 6) -- I'm like a superhero or something</em>
350
<h2>Bannerfish -- Conclusions</h2><ul>
351
<li>What are the tradeoffs?</li>
353
<li>Everything works in the simple cases</li>
355
<li>Not enough complicated cases to have data</li>
357
<li>Luckily, easy to move between them</li>
359
<li>Motto -- move deployment choices as late as possible</li>
363
<em>Giles (Killed By Death, season 2) -- Simple enough, but, but</em>
364
<h2>Further Reading</h2><ul>
365
<li>Short overview -- doc/howto/web-overview.html</li>
367
<li>In depth review -- doc/howto/using-twistedweb.html</li>
369
<li>Using databases -- doc/howto/enterprise.html</li>
371
<li>Deferred execution -- doc/howto/deferred.html</li>
373
<li>Resource script examples -- doc/examples/*.rpy.py</li>
377
<em>Giles (I Was Made to Love You, Season 5) -- There's an enormous amount of research we should do before -- no I'm lying</em>
379
<em>Vampire Willow (Dopplegangland, season 3): Questions? Comments?</em>
380
<h2>Bonus Slides</h2>
381
<em>Xander (The Dark Age, season 2) -- A bonus day of class plus Cordelia.</em>
383
<h2>Python Configuration -- Hints</h2><ul>
384
<li>Working with persistence</li>
390
<li>Virtual Hosts</li>
394
<em>Buffy (Phases, season 2) -- Have you dropped any hints?</em>
395
<h2>Python Configuration -- Persistence</h2><ul>
396
<li>Don't define functions or classes</li>
398
<li>Don't modify class attributes</li>
402
<em>Spike (Once More, With Feeling) -- Let me rest in peace</em>
403
<h2>Python Configuration -- Processors</h2>
406
from twisted.internet import app
407
from twisted.web import static, server
408
from twisted.web import twcgi
410
root = static.File("/var/www")
411
root.processors = {".cgi": twcgi.CGIScript}
412
application = app.Application('web')
413
application.listenTCP(80, server.Site(root))
416
<em>Manny (Doublemeat Palace, season 6) -- It's a meat process</em>
417
<h2>Python Configuration -- Indices</h2>
420
root = static.File("/var/www")
421
root.indices = ['index.rpy', 'index.html']
424
<em>Willow (Buffy vs. Dracula, season 5) -- Labelling your amulets and indexing your diaries</em>
425
<h2>Python Configuration -- Virtual Hosts</h2>
428
from twisted.web import vhost
429
default = static.File("/var/www")
430
foo = static.File("/var/foo")
431
root = vhost.NamedVirtualHost(default)
432
root.addHost('foo.com', foo)
435
<em>Fritz (I Robot, You Jane, season 1) -- The only reality is virtual.</em>
436
<h2>Python Configuration -- uber example</h2>
439
from twisted.internet import app
440
from twisted.web import static, server, vhost, script
442
default = static.File("/var/www")
443
default.processors = {".rpy", script.ResourceScript}
444
root = vhost.NamedVirtualHost(default
445
foo = static.File("/var/foo")
446
foo.indices = ['index.xhtml', 'index.html']
447
root.addHost('foo.com', foo)
448
site = server.Site(root)
449
application = app.Application('web')
450
application.listenTCP(80, site, interface='127.0.0.1')
453
<em>Buffy (Potential, season 7) -- It was putting a lot of stock in that uber-vamp</em>
454
<h2>Python Configuration -- Splitting With Reverse Proxy</h2><ul>
455
<li>Original use case - SVN</li></ul>
458
from twisted.web import proxy
461
proxy.ReverseProxyResource('localhost',
465
<em>Buffy (Once More, With Feeling, season 6 -- So I will walk through the fire</em>
466
<h2>mktap examples</h2><ul>
469
<li>mktap web --path=/var/www --logfile=/var/log/twistedweb.log</li>
471
<li>mktap web --port=80 --path=/var/www --mime-type=text/plain</li>
473
<li>mktap web --path=/home/nafai/public_html --processor=.pl=PerlProcessor.PerlProcessor --index=index.pl</li>
477
<em>Anya (I Was Made to Love You, season 4) -- You can also see the website I designed for the magic shop</em>
478
<h2>mktap examples (cont'd)</h2><ul>
479
<li>mktap web --users</li>
481
<li>mktap web --ignore-ext=.cgi</li>
483
<li>mktap web --index=index.cgi --index=index.rpy --index=index.html</li>
487
<em>Buffy (Once More, With Feeling, season 6) -- All the twists and bends</em>
488
<h2>mktap examples (alternate formats)</h2><ul>
489
<li>mktap --type=source web</li>
491
<li>mktap --type=xml web</li>
495
<em>Tara (Seeing Red, season 6) -- It isn't written in any ancient language we could identify.</em>
496
<h2>mktap examples (setting uid)</h2><ul>
497
<li>mktap --uid=33 web</li>
499
<li>mktap --gid=33 web</li>
501
<li>Uid/Gid of www-data on Debian systems</li>
503
<li>Not possible to use username</li>
505
<li>More about this later</li>
509
<em>Buffy (Who Are You?, season 4) -- I would be Buffy</em>
511
<h2>twistd examples</h2><ul>
512
<li>twistd -f web.tap -l /var/log/twisted.log</li>
514
<li>twistd -f web.tap --pidfile /var/run/web.pid</li>
516
<li>twistd -x web.tax<ul><li>For mktap --type=xml</li>
519
<li>twistd -s web.tas<ul><li>For mktap --type=source</li>
524
<em>Xander (Teacher's Pet, season 1) -- How come *that* never came up?</em>
525
<h2>Shutting down twistd</h2><ul>
526
<li>On Unix (in general): <ul><li>kill `cat twistd.pid`</li>
529
<li>On Windows: <ul><li>Cannot daemonize on Windows, so just run twistd in a command prompt</li>
531
<li>Switch to the command prompt, and press Control-C</li>
536
<em>Buffy (Prophecy Girl, season 1) -- I don't wanna die.</em>
537
<h2>Shutdown TAPs</h2><ul>
538
<li>Since TAPs store persistent data for an application, a 'shutdown' TAP is created on twistd shutdown</li>
540
<li>You'll often want to start your Twisted application on subsequent runs with the shutdown TAP</li>
544
<em>Headstone (The Gift, season 5) -- She saved the world. A lot.</em>
545
<h2>twistd and security</h2><ul>
546
<li>When twistd is run as root, it will shed root privileges for the uid and gid of either the user that created the TAP or those specified on the mktap commandline.</li>
550
<em>Buffy (Dopplegangland, season 3) -- I think it's good to be reliable</em>
552
<h2>Resource Call Examples</h2><ul>
553
<li>/foo/bar/baz gets converted to:</li></ul>
556
site.getChild('foo', request
557
).getChild('bar', request
558
).getChild('baz', request
562
<em>Willow/Tara (Afterlife, Part 2, season 6) -- Child of words, hear thy makers</em>
563
<h2>Resource Call Examples (cont'd)</h2><ul>
564
<li>/foo/bar/baz/ gets converted to:</li></ul>
567
site.getChild('foo', request
568
).getChild('bar', request
569
).getChild('baz', request
570
).getChild('', request
574
<em>Buffy (Gone, season 6) -- Stop trying to see me.</em>
575
<h2>Distributed Servers -- Theory</h2><ul>
576
<li>Master is a resource</li>
578
<li>Slave is a server</li>
580
<li>Same server can have both master and slave parts</li>
584
<em>Anya (Once More, With Feeling, season 6) -- I've got a theory, it could be bunnies</em>
585
<h2>Distributed Servers -- Manually</h2>
587
from twisted.internet import app, protocol
588
from twisted.web import server, distrib, static
589
from twisted.spread import pb
591
application = app.Application("silly-web")
592
# The "master" server
593
site = server.Site(distrib.ResourceSubscription('unix', '.rp'))
594
application.listenTCP(19988, site)
596
fact = pb.BrokerFactory(distrib.ResourcePublisher(
597
server.Site(static.File('static'))))
598
application.listenUNIX('./.rp', fact)
601
<em>Buffy (Some Assembly Required, season 2) -- Men dig up the corpses and the women have the babies.</em>
602
<h2>Distributed Servers -- Manual (cont'd)</h2><ul>
603
<li>First Server</li></ul>
606
from twisted.internet import app, protocol
607
from twisted.web import server, distrib, static, vhost
608
from twisted.spread import pb
610
application = app.Application("ping-web")
612
default = static.File("/var/www/foo")
613
root = vhost.NamedVirtualHost(default)
614
root.addVhost("foo.com", default)
615
bar = distrib.ResourceSubscription('unix', '.bar')
616
root.addVhost("bar.com", bar)
618
fact = pb.BrokerFactory(static.Site(default))
619
site = server.Site(root)
620
application.listenTCP(19988, site)
621
application.listenUNIX('./.foo', fact)
624
<em>Buffy (Welcome to the Hellmouth, season 1) -- Now, we can do this the hard way, or...</em>
625
<h2>Distributed Servers -- Manual (cont'd 2)</h2><ul>
626
<li>Second Server</li></ul>
629
from twisted.internet import app, protocol
630
from twisted.web import server, distrib, static, vhost
631
from twisted.spread import pb
633
application = app.Application("pong-web")
635
foo = distrib.ResourceSubscription('unix', '.foo')
636
root = vhost.NamedVirtualHost(foo)
637
root.addVhost("foo.com", foo)
638
bar = static.File("/var/www/bar")
639
root.addVhost("bar.com", bar)
641
fact = pb.BrokerFactory(static.Site(bar))
642
site = server.Site(root)
643
application.listenTCP(19989, site)
644
application.listenUNIX('./.bar', fact)
647
<em>Buffy (Welcome to the Hellmouth, season 1) -- ...well, actually there's just the hard way.</em>
648
<h2>Distributed Servers -- User Directory</h2><ul>
651
<li>Child that looks like 'moshez' -- ~moshez/public_html </li>
653
<li>Child that looks like 'moshez.twistd' -- moshez's personal server</li>
657
<em>Master (The Wish, season 3) -- Mass production!</em>
658
<h2>Distributed Servers -- User Directory Server</h2><ul>
659
<li>With mktap: mktap web --user</li>
661
<li>With Python configuration</li></ul>
664
from twisted.internet import app
665
from twisted.web import static, server, distrib
667
root = static.File("/var/www")
668
root.putChild("users", distrib.UserDirectory())
669
site = server.Site(root)
670
application = app.Application('web')
671
application.listenTCP(80, site)
674
<em>Richard (Reptile Boy, season 2) -- In his name.</em>
675
<h2>Distributed Servers -- Personal Servers</h2><ul>
676
<li>With mktap: mktap web --personal ...</li>
678
<li>With Python configuration</li></ul>
681
from twisted.internet import app
682
from twisted.web import static, server, distrib
683
from twisted.spread import pb
685
root = static.File("/home/moshez/twistd")
686
site = server.Site(root)
688
fact = pb.BrokerFactory(distrib.ResourcePublisher(site))
689
application.listenUNIX('/home/moshez/.twisted-web-pb', fact)
692
<em>Giles (Bargaining, season 6) -- It's my personal collection</em>
693
<h2>Debian Configuration</h2><ul>
694
<li>Inside twisted-web package</li>
696
<li>Goal -- look like other web servers to users</li>
698
<li>Goal -- interoperate easily</li>
700
<li>Goal -- allow users to avoid modifying files</li>
704
<em>Buffy (Bad Girls, season 3) -- We can help each other.</em>
705
<h2>Debian Configuration -- Usage</h2><ul>
706
<li>Changing port -- edit /etc/twisted-web/ports</li>
708
<li>Want to use behind reverse proxy? Use rptwisted</li>
710
<li>Change anything else -- drop files in /etc/twisted-web/local.d</li>
714
<em>Faith (Home Coming, season 3) -- we'll use 'em</em>
715
<h2>Debian Configuration -- Drop In Examples</h2>
717
from twisted.web import static
720
vhostDir = '/var/www/vhost/'
722
for file in os.listdir(vhostDir):
723
root.addHost(file, static.File(os.path.join(vhostDir, file)))
726
<em>Buffy (The Freshman, season 4) -- I just thought I'd drop in</em>
727
<h2>Debian Configuration -- Drop In Examples (cont'd)</h2>
729
from twisted.web import script, static
731
default.processors['.rpy'] = script.ResourceScript
732
default.ignoreExt('rpy')
735
<em>Riley (As You Were, season 6) -- Sorry to just drop in on you</em>
736
<h2>Debian Configuration -- Drop In Examples (cont'd 2)</h2>
738
from twisted.web import vhost
740
default.putChild('vhost', vhost.VHostMonsterResource())
743
<em>Sam (As You Were, season 6) -- a hairy night drop into hostile territory</em>
744
<h2>twistedmatrix.com Configuration</h2><ul>
745
<li>Some highlights</li></ul>
749
indexNames = ['index', 'index.html', 'index.xhtml', 'index.rpy','index.cgi']
751
root.putChild('mailman', twcgi.CGIDirectory('/usr/lib/cgi-bin'))
752
root.putChild('users', distrib.UserDirectory())
753
root.putChild('cgi-bin', twcgi.CGIDirectory('/usr/lib/cgi-bin'))
754
root.putChild('doc', static.File('/usr/share/doc'))
756
uid = pwd.getpwnam('www-data')[2]
757
gid = grp.getgrnam('www-data')[2]
759
top = rewrite.RewriterResource(root, rewrite.tildeToUsers)
761
application = app.Application("web", uid=uid, gid=gid)
764
<em>Xander (The Witch, season 1) -- May all lesser cretins bow before me.</em>
765
<h2>Apache vs. Twisted Web</h2><ul>
766
<li>Apache is faster</li>
768
<li>Apache -- Threads/processes model</li>
770
<li>Twisted -- async model</li>
772
<li>Apache -- has C security holes (buffer overflows)</li>
774
<li>Twisted -- easy to set up</li>
776
<li>Twisted -- built in Python programmability</li>
780
<em>Willow (Buffy vs. Dracular, season 5) -- I think we've just put our finger on why we're the sidekicks</em>
781
<h2>Apache/Twisted Web Integration</h2><ul>
784
<li>Apache's reverse proxy works well</li>
786
<li>Easy to have a site which is partially managed by Apache</li>
788
<li>Documentation has examples of configurations</li>
792
<em>Xander (What's My Line, season 2) -- Angel's our friend! Except I don't like him.</em>
793
<h2>Zope vs. Twisted Web</h2><ul>
794
<li>Zope -- fully editable through the web</li>
796
<li>Zope -- uses ZODB, not file system</li>
798
<li>Twisted -- can integrate with other protocols easily</li>
800
<li>Twisted -- extension code has much less overhead</li>
804
<em>Willow (Dopplegangland, season 3) -- Competition is natural and healthy</em>
805
<h2>Zope/Twisted Web Integration</h2><ul>
806
<li>Possible to use Twisted as Zope's network layer</li>
808
<li>Hackish with Zope2</li>
810
<li>Easier with Zope3</li>
814
<em>Snyder (Dopplegangland, season 3) -- It's a perfect match.</em>
815
<h2>Zope/Twisted Web Integration (cont'd)</h2><ul>
816
<li>Less direct -- use Apache</li>
818
<li>Reverse proxy parts to Zope</li>
820
<li>Reverse proxy parts to Twisted Web</li>
824
<em>Wesley (Dopplegangland, season 3) -- Still a little sloppy, though</em>
825
<h2>Applications Appropriate for Twisted Web</h2><ul>
830
<li>Web/other protocol chat systems</li>
834
<em>Sweet (Once More, With Feeling) -- Why don't you come and play?</em>
835
<h2>Behind Reverse Proxy</h2><ul>
836
<li>Sometimes, we want Twisted to pretend to be another host/port</li>
838
<li>Reverse proxies, NATs, etc.</li>
840
<li>Reverse proxy to /vhost/http/<host:port>/</li>
842
<li>Make sure root has a child called vhost of type twisted.web.vhost.VirtualHostingMonster</li>
846
<em>Jenny (I Robot -- You Jane, season 1) -- The divine exists in cyberspace</em>
847
<h2>Rewrite Rules</h2><ul>
848
<li>Change a URL to another</li>
850
<li>Useful for different treatment from outside resources</li>
852
<li>Wraps a resource</li>
856
<em>Spike (What's My Line, season 2) -- Read it again.</em>
857
<h2>Rewrite Rules -- Example</h2>
860
root = static.File("/var/www")
861
root.putChild("users", distrib.UserDirectory())
862
root = rewrite.RewriterResource(root, rewrite.tildeToUsers)
864
<ul><li>Now, /~moshez/ works</li>
868
<em>Spike (What's My Line, season 2) -- I think it's just enough kill.</em>
869
<h2>websetroot</h2><ul>
870
<li>Used to change what the root of the server points to</li>
872
<li>Set it to a Resource contained either in a Python source file or a Pickle file</li>
876
<em>Manny (DoubleMeat Palace, season 6) -- We have a lot of turnover here</em>
877
<h2>Sample websetroot command lines</h2><ul>
878
<li>websetroot -p 80 -f web.tap --script rootResource.py</li>
880
<li>websetroot -p 8080 -f web.tap --pickle rootPickle</li>
884
<em>Manny (DoubleMeat Palace, season 6) -- You can toss it</em>
892
<li>No smooth reloading</li>
898
<em>Bob (Zeppo, season 3) -- He hasn't been initiated.</em>
899
<h2>Special Bonus - How to Configure <user>.example.com</h2>
902
from twisted.web import resource, error, distrib
903
from twisted.protocols import http
905
class UserNameVirtualHost(resource.Resource):
907
def __init__(self, default, tail):
908
resource.Resource.__init__(self)
909
self.default = default
913
def _getResourceForRequest(self, request):
914
host=request.getHeader('host')
915
if host.endswith(tail):
916
username = host[:-len(tail)]
919
if self.users.has_key(username):
920
return self.users[username]
922
(pw_name, pw_passwd, pw_uid, pw_gid, pw_gecos, pw_dir,
923
pw_shell) = pwd.getpwnam(username)
925
return error.ErrorPage(http.NOT_FOUND,
927
"The user %s was not found on this system." %
929
twistdsock = os.path.join(pw_dir, ".twistd-web-pb")
930
rs = distrib.ResourceSubscription('unix',twistdsock)
931
self.users[username] = rs
934
def render(self, request):
935
resrc = self._getResourceForRequest(request)
936
return resrc.render(request)
938
def getChild(self, path, request):
939
resrc = self._getResourceForRequest(request)
940
request.path=request.path[:-1]
941
request.postpath=request.uri.split('/')[1:]
942
print request, request.path, request.postpath
943
return resrc.getChildForRequest(request)
946
<em>Morgan (The Puppet Show, season 1) -- Weird? What d'you mean?</em>
947
<h2>Special Bonus - How to Configure <user>.example.com (cont'd)</h2><ul>
948
<li>Put above in a module (say, uservhost)</li>
950
<li>Use following configuration file</li></ul>
953
from twisted.internet import app
954
from twisted.web import server
957
root = UserNameVirtualHost("www", "example.com")
958
site = server.Site(root)
959
application = app.Application('web')
960
application.listenTCP(80, site)
963
<em>Snyder (The Puppet Show, season 1) -- You need to integrate into this school, people.</em>
964
<h2>Using the Twisted Registry</h2><ul>
965
<li>Use especially in Resource Scripts</li>
967
<li>Save persistent information</li>
969
<li>Uses Twisted's Componentized to be extensible</li>
973
<em>Angel (Helpless, season 3) -- I wanted to keep it safe</em>
974
<h2>Using the Twisted Registry -- example</h2>
977
from twisted.web import distrib
979
resource = registry.getComponent(distrib.UserDirectory)
981
resource = distrib.UserDirectory()
982
registry.setComponent(distrib.UserDirectory, resource)
985
<em>Paul (The Freshman, season 4) -- Do you know where they're distributing the [...] applications?</em>
986
<h2>Using the Twisted Registry -- problems</h2><ul>
987
<li>In most cases -- need to write a custom class</li>
989
<li>Saves data in-memory</li>
991
<li>Won't work as expected unless -shutdown taps are used</li>
995
<em>Anya (Once More, With Feeling, season 6) -- The only trouble is [pause] I'll never tell.</em>
996
<h2>Alternative Configuration Formats -- XML</h2><ul>
997
<li>Can be generated from mktap</li>
999
<li>Editable with any XML editor</li>
1001
<li>Easy to do easy things<ul><li>Change a port</li>
1004
<li>Nontrivial to do hard things<ul><li>Bind to specific IP</li>
1009
<em>Buffy (Once More, With Feeling, season 6) -- To fit in in this glittering world.</em>
1010
<h2>Alternative Configuration Formats -- XML -- example</h2>
1011
<pre class="python">
1012
<?xml version="1.0"?>
1014
<instance class="twisted.internet.app.Application" reference="1">
1017
<string role="key" value="tcpPorts" />
1020
<int value="80" />
1021
<instance class="twisted.web.server.Site">
1024
<string role="key" value="resource" />
1025
<instance class="twisted.web.static.File">
1028
<string role="key" value="path" />
1029
<string value="/var/www" />
1044
<em>Natalie (Teacher's Pet, season 1) -- There's nothing ugly about these creatures</em>
1045
<h2>Alternative Configuration Formats -- Source</h2><ul>
1046
<li>Can be generated from mktap</li>
1048
<li>Editable with any Python source editor</li>
1050
<li>Easy to do easy things<ul><li>Change a port</li>
1053
<li>Nontrivial to do hard things<ul><li>Bind to specific IP</li>
1058
<em>Willow/Giles/Xander (Primeval, season 4) -- You could never hope to grasp the source</em>
1059
<h2>Alternative Configuration Formats -- Source -- Example</h2>
1060
<pre class="python">
1062
Instance('twisted.internet.app.Application',{
1067
Instance('twisted.web.server.Site',
1069
resource=Instance('twisted.web.static.File',{
1079
<em>Tara (Family, season 5) -- You learn her source, and, uh we'll introduce her to her insect reflection</em>
1080
<h2>Twisted Web - Beginnings</h2>
1081
<pre class="python">
1083
<glyphAtWork> the http server was so we could say "Web!" if we ever did
1084
a freshmeat announcement
1085
<glyphAtWork> this makes people excited
1087
<ul><li>Turned out he was right</li>
1091
<em>Dawn (Get It Done, season 7) -- I think it's an origin myth.</em>
1092
<h2>Woven Overview</h2><ul>
1093
<li>HTML templates</li>
1095
<li>Model/View/Controller architecture</li>
1097
<li>Integrated with deferred</li>
1099
<li>Classical systems work badly with async</li>
1101
<li>More -- beyond scope of this tutorial</li>
1105
<em>Razor (Bargaining, season 6) -- A pretty toy</em>
added
removed
vendor/Twisted-10.0.0/doc/historic/2003/europython/tw-deploy.html
