~blamar/+junk/openstack-api-arrrg

<?xml version="1.0" encoding="utf-8"?><!DOCTYPE html PUBLIC '-//W3C//DTD XHTML 1.0 Transitional//EN' 'http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd'><html lang="en" xmlns="http://www.w3.org/1999/xhtml">

<head>

<title>Twisted Documentation: The Evolution of Finger: adding features to the finger service</title>

</head>

<h1 class="title">The Evolution of Finger: adding features to the finger service</h1>

<div class="toc"><ol><li><a href="#auto0">Introduction</a></li><li><a href="#auto1">Setting Message By Local Users</a></li><li><a href="#auto2">Use Services to Make Dependencies Sane</a></li><li><a href="#auto3">Read Status File</a></li><li><a href="#auto4">Announce on Web, Too</a></li><li><a href="#auto5">Announce on IRC, Too</a></li><li><a href="#auto6">Add XML-RPC Support</a></li></ol></div>

<h2>Introduction<a name="auto0"/></h2>

This is the second part of the Twisted tutorial <a href="index.html" shape="rect">Twisted from Scratch, or The Evolution of Finger</a>.

In this section of the tutorial, our finger server will continue to sprout

features: the ability for users to set finger announces, and using our finger

service to send those announcements on the web, on IRC and over XML-RPC.

Resources and XML-RPC are introduced in the Web Applications portion of

the <a href="../../../web/howto/index.html" shape="rect">Twisted Web howto</a>. More examples

using <code class="API"><a href="http://twistedmatrix.com/documents/10.0.0/api/twisted.words.protocols.irc.html" title="twisted.words.protocols.irc">twisted.words.protocols.irc</a></code> can be found

in <a href="../clients.html" shape="rect">Writing a TCP Client</a> and

the <a href="../../../words/examples/index.html" shape="rect">Twisted Words examples</a>.

<h2>Setting Message By Local Users<a name="auto1"/></h2>

Now that port 1079 is free, maybe we can use it with a different

server, one which will let people set their messages. It does

no access control, so anyone who can login to the machine can

set any message. We assume this is the desired behavior in

our case. Testing it can be done by simply:

% nc localhost 1079 # or telnet localhost 1079

moshez

Giving a tutorial now, sorry!

</pre>

<div class="py-listing"><pre> 1

# But let's try and fix setting away messages, shall we?

from twisted.application import internet, service

100

from twisted.internet import protocol, reactor, defer

101

from twisted.protocols import basic

102

103

class FingerProtocol(basic.LineReceiver):

104

def lineReceived(self, user):

105

d = self.factory.getUser(user)

106

107

def onError(err):

108

return 'Internal error in server'

109

d.addErrback(onError)

110

111

def writeResponse(message):

112

self.transport.write(message + '\r\n')

113

self.transport.loseConnection()

114

d.addCallback(writeResponse)

115

116

class FingerFactory(protocol.ServerFactory):

117

protocol = FingerProtocol

118

119

def __init__(self, **kwargs):

120

self.users = kwargs

121

122

def getUser(self, user):

123

return defer.succeed(self.users.get(user, "No such user"))

124

125

class FingerSetterProtocol(basic.LineReceiver):

126

def connectionMade(self):

127

self.lines = []

128

129

def lineReceived(self, line):

130

self.lines.append(line)

131

132

def connectionLost(self, reason):

133

user = self.lines[0]

134

status = self.lines[1]

135

self.factory.setUser(user, status)

136

137

class FingerSetterFactory(protocol.ServerFactory):

138

protocol = FingerSetterProtocol

139

140

def __init__(self, fingerFactory):

141

self.fingerFactory = fingerFactory

142

143

def setUser(self, user, status):

144

self.fingerFactory.users[user] = status

145

146

ff = FingerFactory(moshez='Happy and well')

147

fsf = FingerSetterFactory(ff)

148

149

application = service.Application('finger', uid=1, gid=1)

150

serviceCollection = service.IServiceCollection(application)

151

internet.TCPServer(79,ff).setServiceParent(serviceCollection)

152

internet.TCPServer(1079,fsf).setServiceParent(serviceCollection)

153

</pre><div class="caption">Source listing - <a href="listings/finger/finger12.tac">listings/finger/finger12.tac</a></div></div>

154

155

This program has two protocol-factory-TCPServer pairs, which are both child

156

services of the application. Specifically, the

157

<code base="API" class="twisted.application.service.Service">setServiceParent</code>

158

method is used to define the two TCPServer services as children of

159

<code>application</code>, which implements

160

<code base="API" class="twisted.application.servce">IServiceCollection</code>.

161

Both services are thus started with the application.

162

163

<h2>Use Services to Make Dependencies Sane<a name="auto2"/></h2>

164

165

The previous version had the setter poke at the innards of the

166

finger factory. This strategy is usually not a good idea: this version makes

167

both factories symmetric by making them both look at a single

168

object. Services are useful for when an object is needed which is

169

not related to a specific network server. Here, we define a common service

170

class with methods that will create factories on the fly. The service

171

also contains methods the factories will depend on.

172

173

The factory-creation methods, <code>getFingerFactory</code> and

174

<code>getFingerSetterFactory</code>, follow this pattern:

175

176

<ol>

177

178

<li>Instantiate a generic server factory,

179

<code>twisted.internet.protocol.ServerFactory</code>.</li>

180

181

<li>Set the protocol class, just like our factory class would have.</li>

182

183

<li>Copy a service method to the factory as a function attribute. The

184

function won't have access to the factory's <code>self</code>, but that's OK

185

because as a bound method it has access to the service's <code>self</code>,

186

which is what it needs. For <code>getUser</code>, a custom method defined in

187

the service gets copied. For <code>setUser</code>, a standard method of the

188

<code>users</code> dictionary is copied.</li>

189

190

</ol>

191

192

Thus, we stopped subclassing: the service simply puts useful methods and

193

attributes inside the factories. We are getting better at protocol design:

194

none of our protocol classes had to be changed, and neither will have to

195

change until the end of the tutorial.

196

197

As an application

198

<code class="API"><a href="http://twistedmatrix.com/documents/10.0.0/api/twisted.application.service.service.html" title="twisted.application.service.service">service</a></code> , this new

199

finger service implements the

200

<code class="API"><a href="http://twistedmatrix.com/documents/10.0.0/api/twisted.application.service.IService.html" title="twisted.application.service.IService">IService</a></code> interface

201

and can be started and stopped in a standardized manner. We'll make use of

202

this in the next example.

203

204

<div class="py-listing"><pre> 1

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

# Fix asymmetry

264

265

266

267

268

class FingerProtocol(basic.LineReceiver):

269

def lineReceived(self, user):

270

271

272

def onError(err):

273

return 'Internal error in server'

274

d.addErrback(onError)

275

276

def writeResponse(message):

277

278

self.transport.loseConnection()

279

d.addCallback(writeResponse)

280

281

class FingerSetterProtocol(basic.LineReceiver):

282

def connectionMade(self):

283

self.lines = []

284

285

def lineReceived(self, line):

286

self.lines.append(line)

287

288

def connectionLost(self,reason):

289

user = self.lines[0]

290

status = self.lines[1]

291

292

293

class FingerService(service.Service):

294

def __init__(self, **kwargs):

295

self.users = kwargs

296

297

def getUser(self, user):

298

299

300

301

self.users[user] = status

302

303

def getFingerFactory(self):

304

f = protocol.ServerFactory()

305

f.protocol = FingerProtocol

306

f.getUser = self.getUser

307

return f

308

309

def getFingerSetterFactory(self):

310

f = protocol.ServerFactory()

311

f.protocol = FingerSetterProtocol

312

f.setUser = self.setUser

313

return f

314

315

316

f = FingerService(moshez='Happy and well')

317

318

internet.TCPServer(79,f.getFingerFactory()

319

).setServiceParent(serviceCollection)

320

internet.TCPServer(1079,f.getFingerSetterFactory()

321

).setServiceParent(serviceCollection)

322

</pre><div class="caption">Source listing - <a href="listings/finger/finger13.tac">listings/finger/finger13.tac</a></div></div>

323

324

325

326

<h2>Read Status File<a name="auto3"/></h2>

327

328

This version shows how, instead of just letting users set their

329

messages, we can read those from a centrally managed file. We cache

330

results, and every 30 seconds we refresh it. Services are useful

331

for such scheduled tasks.

332

333

334

moshez: happy and well

335

shawn: alive

336

</pre><div class="caption">sample /etc/users file - <a href="listings/finger/etc.users">listings/finger/etc.users</a></div></div>

337

338

<div class="py-listing"><pre> 1

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

# Read from file

394

395

396

397

398

class FingerProtocol(basic.LineReceiver):

399

def lineReceived(self, user):

400

401

402

def onError(err):

403

return 'Internal error in server'

404

d.addErrback(onError)

405

406

def writeResponse(message):

407

408

self.transport.loseConnection()

409

d.addCallback(writeResponse)

410

411

class FingerService(service.Service):

412

def __init__(self, filename):

413

self.users = {}

414

self.filename = filename

415

self._read()

416

417

def _read(self):

418

for line in file(self.filename):

419

user, status = line.split(':', 1)

420

user = user.strip()

421

status = status.strip()

422

self.users[user] = status

423

self.call = reactor.callLater(30, self._read)

424

425

def startService(self):

426

self._read()

427

service.Service.startService(self)

428

429

def stopService(self):

430

service.Service.stopService(self)

431

self.call.cancel()

432

433

def getUser(self, user):

434

435

436

def getFingerFactory(self):

437

f = protocol.ServerFactory()

438

f.protocol = FingerProtocol

439

f.getUser = self.getUser

440

return f

441

442

443

f = FingerService('/etc/users')

444

finger = internet.TCPServer(79, f.getFingerFactory())

445

446

finger.setServiceParent(service.IServiceCollection(application))

447

f.setServiceParent(service.IServiceCollection(application))

448

</pre><div class="caption">Source listing - <a href="listings/finger/finger14.tac">listings/finger/finger14.tac</a></div></div>

449

450

Since this verison is reading data from a file (and refreshing the data

451

every 30 seconds), there is no <code>FingerSetterFactory</code> and thus

452

nothing listening on port 1079.

453

454

Here we override the standard

455

<code class="API"><a href="http://twistedmatrix.com/documents/10.0.0/api/twisted.application.service.Service.startService.html" title="twisted.application.service.Service.startService">startService</a></code>

456

and

457

<code class="API"><a href="http://twistedmatrix.com/documents/10.0.0/api/twisted.application.service.Service.stopService.html" title="twisted.application.service.Service.stopService">stopService</a></code>

458

hooks in the Finger service, which is set up as a child service of

459

the application in the last line of the code. <code>startService</code> calls

460

<code>_read</code>, the function responsible for reading the data;

461

<code>reactor.callLater</code> is then used to schedule it to run again after

462

thirty seconds every time it is called. <code>reactor.callLater</code> returns

463

an object that lets us cancel the scheduled run in <code>stopService</code>

464

using its <code>cancel</code> method.

465

466

<h2>Announce on Web, Too<a name="auto4"/></h2>

467

468

The same kind of service can also produce things useful for

469

other protocols. For example, in twisted.web, the factory

470

itself (<code base="API" class="twisted.web.server">Site</code>) is almost

471

never subclassed — instead, it is given a resource, which represents the tree

472

of resources available via URLs. That hierarchy is navigated by

473

474

and overriding it dynamically is possible with

475

<code base="API" class="twisted.web.resource.Resource">getChild</code>.

476

477

To integrate this into the Finger application (just because we can), we set

478

up a new TCPServer that calls the <code base="API" class="twisted.web.server">Site</code> factory and retrieves resources via a

479

new function of <code>FingerService</code> named <code>getResource</code>.

480

This function specifically returns a <code base="API" class="twisted.web.resource">Resource</code> object with an overridden <code base="API" class="twisted.web.resource.Resource">getChild</code> method.

481

482

<div class="py-listing"><pre> 1

483

484

485

486

487

488

489

490

491

492

493

494

495

496

497

498

499

500

501

502

503

504

505

506

507

508

509

510

511

512

513

514

515

516

517

518

519

520

521

522

523

524

525

526

527

528

529

530

531

532

533

534

535

536

537

538

539

540

541

542

543

544

545

546

547

548

549

550

551

552

553

554

555

556

557

558

# Read from file, announce on the web!

559

560

561

562

from twisted.web import resource, server, static

563

import cgi

564

565

class FingerProtocol(basic.LineReceiver):

566

def lineReceived(self, user):

567

568

569

def onError(err):

570

return 'Internal error in server'

571

d.addErrback(onError)

572

573

def writeResponse(message):

574

575

self.transport.loseConnection()

576

d.addCallback(writeResponse)

577

578

class FingerResource(resource.Resource):

579

580

def __init__(self, users):

581

self.users = users

582

resource.Resource.__init__(self)

583

584

# we treat the path as the username

585

def getChild(self, username, request):

586

"""

587

'username' is a string.

588

'request' is a 'twisted.web.server.Request'.

589

"""

590

messagevalue = self.users.get(username)

591

username = cgi.escape(username)

592

if messagevalue is not None:

593

messagevalue = cgi.escape(messagevalue)

594

text = '<h1>%s</h1>%s' % (username,messagevalue)

595

else:

596

text = '<h1>%s</h1>No such user' % username

597

return static.Data(text, 'text/html')

598

599

class FingerService(service.Service):

600

def __init__(self, filename):

601

self.filename = filename

602

self.users = {}

603

self._read()

604

605

def _read(self):

606

self.users.clear()

607

608

609

user = user.strip()

610

status = status.strip()

611

self.users[user] = status

612

613

614

def getUser(self, user):

615

616

617

def getFingerFactory(self):

618

f = protocol.ServerFactory()

619

f.protocol = FingerProtocol

620

f.getUser = self.getUser

621

return f

622

623

def getResource(self):

624

r = FingerResource(self.users)

625

return r

626

627

628

f = FingerService('/etc/users')

629

630

internet.TCPServer(79, f.getFingerFactory()

631

).setServiceParent(serviceCollection)

632

internet.TCPServer(8000, server.Site(f.getResource())

633

).setServiceParent(serviceCollection)

634

</pre><div class="caption">Source listing - <a href="listings/finger/finger15.tac">listings/finger/finger15.tac</a></div></div>

635

636

637

<h2>Announce on IRC, Too<a name="auto5"/></h2>

638

639

This is the first time there is client code. IRC clients often

640

act a lot like servers: responding to events from the network.

641

The reconnecting client factory will make sure that severed links

642

will get re-established, with intelligent tweaked exponential

643

back-off algorithms. The IRC client itself is simple: the only

644

real hack is getting the nickname from the factory in connectionMade.

645

646

<div class="py-listing"><pre> 1

647

648

649

650

651

652

653

654

655

656

657

658

659

660

661

662

663

664

665

666

667

668

669

670

671

672

673

674

675

676

677

678

679

680

681

682

683

684

685

686

687

688

689

690

691

692

693

694

695

696

697

698

699

700

701

702

703

704

705

706

707

708

709

710

711

712

713

714

715

716

717

718

719

720

721

722

723

724

725

726

727

728

729

730

731

732

733

734

735

736

737

# Read from file, announce on the web, irc

738

739

740

from twisted.words.protocols import irc

741

742

743

744

import cgi

745

746

class FingerProtocol(basic.LineReceiver):

747

def lineReceived(self, user):

748

749

750

def onError(err):

751

return 'Internal error in server'

752

d.addErrback(onError)

753

754

def writeResponse(message):

755

756

self.transport.loseConnection()

757

d.addCallback(writeResponse)

758

759

760

class IRCReplyBot(irc.IRCClient):

761

def connectionMade(self):

762

self.nickname = self.factory.nickname

763

irc.IRCClient.connectionMade(self)

764

765

def privmsg(self, user, channel, msg):

766

user = user.split('!')[0]

767

if self.nickname.lower() == channel.lower():

768

769

770

def onError(err):

771

return 'Internal error in server'

772

d.addErrback(onError)

773

774

def writeResponse(message):

775

irc.IRCClient.msg(self, user, msg+': '+message)

776

d.addCallback(writeResponse)

777

778

class FingerService(service.Service):

779

def __init__(self, filename):

780

self.filename = filename

781

self.users = {}

782

self._read()

783

784

def _read(self):

785

self.users.clear()

786

787

788

user = user.strip()

789

status = status.strip()

790

self.users[user] = status

791

792

793

def getUser(self, user):

794

795

796

def getFingerFactory(self):

797

f = protocol.ServerFactory()

798

f.protocol = FingerProtocol

799

f.getUser = self.getUser

800

return f

801

802

def getResource(self):

803

r = resource.Resource()

804

r.getChild = (lambda path, request:

805

static.Data('<h1>%s</h1>%s' %

806

tuple(map(cgi.escape,

807

[path,self.users.get(path,

808

"No such user usage: site/user")])),

809

'text/html'))

810

return r

811

812

def getIRCBot(self, nickname):

813

f = protocol.ReconnectingClientFactory()

814

f.protocol = IRCReplyBot

815

f.nickname = nickname

816

f.getUser = self.getUser

817

return f

818

819

820

f = FingerService('/etc/users')

821

822

internet.TCPServer(79, f.getFingerFactory()

823

).setServiceParent(serviceCollection)

824

825

).setServiceParent(serviceCollection)

826

internet.TCPClient('irc.freenode.org', 6667, f.getIRCBot('fingerbot')

827

).setServiceParent(serviceCollection)

828

</pre><div class="caption">Source listing - <a href="listings/finger/finger16.tac">listings/finger/finger16.tac</a></div></div>

829

830

<code>FingerService</code> now has another new function,

831

<code>getIRCbot</code>, which returns the

832

<code>ReconnectingClientFactory</code>. This factory in turn will instantiate

833

the <code>IRCReplyBot</code> protocol. The IRCBot is configured in the last

834

line to connect to <code>irc.freenode.org</code> with a nickname of

835

<code>fingerbot</code>.

836

837

By overriding <code>irc.IRCClient.connectionMade</code>,

838

<code>IRCReplyBot</code> can access the <code>nickname</code> attribute of the

839

factory that instantiated it.

840

841

<h2>Add XML-RPC Support<a name="auto6"/></h2>

842

843

In Twisted, XML-RPC support is handled just as though it was

844

another resource. That resource will still support GET calls normally

845

through render(), but that is usually left unimplemented. Note

846

that it is possible to return deferreds from XML-RPC methods.

847

The client, of course, will not get the answer until the deferred

848

is triggered.

849

850

<div class="py-listing"><pre> 1

851

852

853

854

855

856

857

858

859

860

861

862

863

864

865

866

867

868

869

870

871

872

873

874

875

876

877

878

879

880

881

882

883

884

885

886

887

888

889

890

891

892

893

894

895

896

897

898

899

900

901

902

903

904

905

906

907

908

909

910

911

912

913

914

915

916

917

918

919

920

921

922

923

924

925

926

927

928

929

930

931

932

933

934

935

936

937

938

939

940

941

# Read from file, announce on the web, irc, xml-rpc

942

943

944

945

946

947

import cgi

948

949

class FingerProtocol(basic.LineReceiver):

950

def lineReceived(self, user):

951

952

953

def onError(err):

954

return 'Internal error in server'

955

d.addErrback(onError)

956

957

def writeResponse(message):

958

959

self.transport.loseConnection()

960

d.addCallback(writeResponse)

961

962

class IRCReplyBot(irc.IRCClient):

963

def connectionMade(self):

964

965

irc.IRCClient.connectionMade(self)

966

967

968

969

970

971

972

def onError(err):

973

return 'Internal error in server'

974

d.addErrback(onError)

975

976

def writeResponse(message):

977

978

d.addCallback(writeResponse)

979

980

class FingerService(service.Service):

981

def __init__(self, filename):

982

self.filename = filename

983

self.users = {}

984

self._read()

985

986

def _read(self):

987

self.users.clear()

988

989

990

user = user.strip()

991

status = status.strip()

992

self.users[user] = status

993

994

995

def getUser(self, user):

996

997

998

def getFingerFactory(self):

999

f = protocol.ServerFactory()

1000

f.protocol = FingerProtocol

1001

f.getUser = self.getUser

1002

return f

1003

1004

def getResource(self):

1005

r = resource.Resource()

1006

1007

static.Data('<h1>%s</h1>%s' %

1008

tuple(map(cgi.escape,

1009

1010

'text/html'))

1011

x = xmlrpc.XMLRPC()

1012

x.xmlrpc_getUser = self.getUser

1013

r.putChild('RPC2', x)

1014

return r

1015

1016

def getIRCBot(self, nickname):

1017

f = protocol.ReconnectingClientFactory()

1018

f.protocol = IRCReplyBot

1019

f.nickname = nickname

1020

f.getUser = self.getUser

1021

return f

1022

1023

1024

f = FingerService('/etc/users')

1025

1026

internet.TCPServer(79, f.getFingerFactory()

1027

).setServiceParent(serviceCollection)

1028

1029

).setServiceParent(serviceCollection)

1030

1031

).setServiceParent(serviceCollection)

1032

</pre><div class="caption">Source listing - <a href="listings/finger/finger17.tac">listings/finger/finger17.tac</a></div></div>

1033

1034

Instead of a web browser, we can test the XMLRPC finger using a simple

1035

client based on Python's built-in <code>xmlrpclib</code>, which will access

1036

the resource we've made available at <code>localhost/RPC2</code>.

1037

1038

<div class="py-listing"><pre>1

1039

1040

1041

1042

1043

# testing xmlrpc finger

1044

1045

import xmlrpclib

1046

server = xmlrpclib.Server('http://127.0.0.1:8000/RPC2')

1047

print server.getUser('moshez')

1048

</pre><div class="caption">Source listing - <a href="listings/finger/fingerXRclient.py">listings/finger/fingerXRclient.py</a></div></div>

1049

1050

</div>

1051

1052

<a href="../index.html">Index</a>

1053

Version: 10.0.0

1054

</body>

1055

</html>

b'\\ No newline at end of file'

Older »