~dkuhlman/python-training-materials/Materials

<a class="reference internal" href="../c-api/structures.html#c.PyObject" title="PyObject"><code class="xref c c-type docutils literal">PyObject*</code></a>, which serves as a “base type” for all Python objects.

<a class="reference internal" href="../c-api/structures.html#c.PyObject" title="PyObject"><code class="xref c c-type docutils literal">PyObject</code></a> itself only contains the refcount and a pointer to the

object’s “type object”. This is where the action is; the type object determines

which (C) functions get called when, for instance, an attribute gets looked

up on an object or it is multiplied by another object. These C functions

are called “type methods”.

So, if you want to define a new object type, you need to create a new type

object.

This sort of thing can only be explained by example, so here’s a minimal, but

complete, module that defines a new type:

<div class="highlight-c"><div class="highlight"><pre>#include <Python.h>

typedef struct {

PyObject_HEAD

100

/* Type-specific fields go here. */

101

} noddy_NoddyObject;

102

103

static PyTypeObject noddy_NoddyType = {

104

PyVarObject_HEAD_INIT(NULL, 0)

105

"noddy.Noddy", /* tp_name */

106

sizeof(noddy_NoddyObject), /* tp_basicsize */

107

0, /* tp_itemsize */

108

0, /* tp_dealloc */

109

0, /* tp_print */

110

0, /* tp_getattr */

111

0, /* tp_setattr */

112

0, /* tp_reserved */

113

0, /* tp_repr */

114

0, /* tp_as_number */

115

0, /* tp_as_sequence */

116

0, /* tp_as_mapping */

117

0, /* tp_hash */

118

0, /* tp_call */

119

0, /* tp_str */

120

0, /* tp_getattro */

121

0, /* tp_setattro */

122

0, /* tp_as_buffer */

123

Py_TPFLAGS_DEFAULT, /* tp_flags */

124

"Noddy objects", /* tp_doc */

125

};

126

127

static PyModuleDef noddymodule = {

128

PyModuleDef_HEAD_INIT,

129

"noddy",

130

"Example module that creates an extension type.",

131

-1,

132

NULL, NULL, NULL, NULL, NULL

133

};

134

135

PyMODINIT_FUNC

136

PyInit_noddy(void)

137

{

138

PyObject* m;

139

140

noddy_NoddyType.tp_new = PyType_GenericNew;

141

if (PyType_Ready(&noddy_NoddyType) < 0)

142

return NULL;

143

144

m = PyModule_Create(&noddymodule);

145

if (m == NULL)

146

return NULL;

147

148

Py_INCREF(&noddy_NoddyType);

149

PyModule_AddObject(m, "Noddy", (PyObject *)&noddy_NoddyType);

150

return m;

151

}

152

</pre></div>

153

</div>

154

Now that’s quite a bit to take in at once, but hopefully bits will seem familiar

155

from the last chapter.

156

The first bit that will be new is:

157

<div class="highlight-c"><div class="highlight"><pre>typedef struct {

158

PyObject_HEAD

159

} noddy_NoddyObject;

160

</pre></div>

161

</div>

162

This is what a Noddy object will contain—in this case, nothing more than what

163

every Python object contains—a refcount and a pointer to a type object.

164

These are the fields the <code class="docutils literal">PyObject_HEAD</code> macro brings in. The reason for the

165

macro is to standardize the layout and to enable special debugging fields in

166

debug builds. Note that there is no semicolon after the <code class="docutils literal">PyObject_HEAD</code>

167

macro; one is included in the macro definition. Be wary of adding one by

168

accident; it’s easy to do from habit, and your compiler might not complain,

169

but someone else’s probably will! (On Windows, MSVC is known to call this an

170

error and refuse to compile the code.)

171

For contrast, let’s take a look at the corresponding definition for standard

172

Python floats:

173

<div class="highlight-c"><div class="highlight"><pre>typedef struct {

174

PyObject_HEAD

175

double ob_fval;

176

} PyFloatObject;

177

</pre></div>

178

</div>

179

Moving on, we come to the crunch — the type object.

180

<div class="highlight-c"><div class="highlight"><pre>static PyTypeObject noddy_NoddyType = {

181

PyVarObject_HEAD_INIT(NULL, 0)

182

"noddy.Noddy", /* tp_name */

183

sizeof(noddy_NoddyObject), /* tp_basicsize */

184

0, /* tp_itemsize */

185

0, /* tp_dealloc */

186

0, /* tp_print */

187

0, /* tp_getattr */

188

0, /* tp_setattr */

189

0, /* tp_as_async */

190

0, /* tp_repr */

191

0, /* tp_as_number */

192

0, /* tp_as_sequence */

193

0, /* tp_as_mapping */

194

0, /* tp_hash */

195

0, /* tp_call */

196

0, /* tp_str */

197

0, /* tp_getattro */

198

0, /* tp_setattro */

199

0, /* tp_as_buffer */

200

Py_TPFLAGS_DEFAULT, /* tp_flags */

201

"Noddy objects", /* tp_doc */

202

};

203

</pre></div>

204

</div>

205

Now if you go and look up the definition of <a class="reference internal" href="../c-api/type.html#c.PyTypeObject" title="PyTypeObject"><code class="xref c c-type docutils literal">PyTypeObject</code></a> in

206

<code class="file docutils literal">object.h</code> you’ll see that it has many more fields that the definition

207

above. The remaining fields will be filled with zeros by the C compiler, and

208

it’s common practice to not specify them explicitly unless you need them.

209

This is so important that we’re going to pick the top of it apart still

210

further:

211

<div class="highlight-c"><div class="highlight"><pre>PyVarObject_HEAD_INIT(NULL, 0)

212

</pre></div>

213

</div>

214

This line is a bit of a wart; what we’d like to write is:

215

<div class="highlight-c"><div class="highlight"><pre>PyVarObject_HEAD_INIT(&PyType_Type, 0)

216

</pre></div>

217

</div>

218

as the type of a type object is “type”, but this isn’t strictly conforming C and

219

some compilers complain. Fortunately, this member will be filled in for us by

220

<a class="reference internal" href="../c-api/type.html#c.PyType_Ready" title="PyType_Ready"><code class="xref c c-func docutils literal">PyType_Ready()</code></a>.

221

<div class="highlight-c"><div class="highlight"><pre>"noddy.Noddy", /* tp_name */

222

</pre></div>

223

</div>

224

The name of our type. This will appear in the default textual representation of

225

our objects and in some error messages, for example:

226

<div class="highlight-c"><div class="highlight"><pre>>>> "" + noddy.new_noddy()

227

Traceback (most recent call last):

228

File "<stdin>", line 1, in ?

229

TypeError: cannot add type "noddy.Noddy" to string

230

</pre></div>

231

</div>

232

Note that the name is a dotted name that includes both the module name and the

233

name of the type within the module. The module in this case is <code class="xref py py-mod docutils literal">noddy</code> and

234

the type is <code class="xref py py-class docutils literal">Noddy</code>, so we set the type name to <code class="xref py py-class docutils literal">noddy.Noddy</code>.

235

<div class="highlight-c"><div class="highlight"><pre>sizeof(noddy_NoddyObject), /* tp_basicsize */

236

</pre></div>

237

</div>

238

This is so that Python knows how much memory to allocate when you call

239

<a class="reference internal" href="../c-api/allocation.html#c.PyObject_New" title="PyObject_New"><code class="xref c c-func docutils literal">PyObject_New()</code></a>.

240

241

Note

242

If you want your type to be subclassable from Python, and your type has the same

243

<a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_basicsize" title="PyTypeObject.tp_basicsize"><code class="xref c c-member docutils literal">tp_basicsize</code></a> as its base type, you may have problems with multiple

244

inheritance. A Python subclass of your type will have to list your type first

245

in its <a class="reference internal" href="../library/stdtypes.html#class.__bases__" title="class.__bases__"><code class="xref py py-attr docutils literal">__bases__</code></a>, or else it will not be able to call your type’s

246

<a class="reference internal" href="../reference/datamodel.html#object.__new__" title="object.__new__"><code class="xref py py-meth docutils literal">__new__()</code></a> method without getting an error. You can avoid this problem by

247

ensuring that your type has a larger value for <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_basicsize" title="PyTypeObject.tp_basicsize"><code class="xref c c-member docutils literal">tp_basicsize</code></a> than its

248

base type does. Most of the time, this will be true anyway, because either your

249

base type will be <a class="reference internal" href="../library/functions.html#object" title="object"><code class="xref py py-class docutils literal">object</code></a>, or else you will be adding data members to

250

your base type, and therefore increasing its size.

251

</div>

252

<div class="highlight-c"><div class="highlight"><pre>0, /* tp_itemsize */

253

</pre></div>

254

</div>

255

This has to do with variable length objects like lists and strings. Ignore this

256

for now.

257

Skipping a number of type methods that we don’t provide, we set the class flags

258

to <a class="reference internal" href="../c-api/typeobj.html#Py_TPFLAGS_DEFAULT" title="Py_TPFLAGS_DEFAULT"><code class="xref py py-const docutils literal">Py_TPFLAGS_DEFAULT</code></a>.

259

<div class="highlight-c"><div class="highlight"><pre>Py_TPFLAGS_DEFAULT, /* tp_flags */

260

</pre></div>

261

</div>

262

All types should include this constant in their flags. It enables all of the

263

members defined until at least Python 3.3. If you need further members,

264

you will need to OR the corresponding flags.

265

We provide a doc string for the type in <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_doc" title="PyTypeObject.tp_doc"><code class="xref c c-member docutils literal">tp_doc</code></a>.

266

<div class="highlight-c"><div class="highlight"><pre>"Noddy objects", /* tp_doc */

267

</pre></div>

268

</div>

269

Now we get into the type methods, the things that make your objects different

270

from the others. We aren’t going to implement any of these in this version of

271

the module. We’ll expand this example later to have more interesting behavior.

272

For now, all we want to be able to do is to create new <code class="xref py py-class docutils literal">Noddy</code> objects.

273

To enable object creation, we have to provide a <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_new" title="PyTypeObject.tp_new"><code class="xref c c-member docutils literal">tp_new</code></a> implementation.

274

In this case, we can just use the default implementation provided by the API

275

function <a class="reference internal" href="../c-api/type.html#c.PyType_GenericNew" title="PyType_GenericNew"><code class="xref c c-func docutils literal">PyType_GenericNew()</code></a>. We’d like to just assign this to the

276

277

compilers, we can’t statically initialize a structure member with a function

278

defined in another C module, so, instead, we’ll assign the <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_new" title="PyTypeObject.tp_new"><code class="xref c c-member docutils literal">tp_new</code></a> slot

279

in the module initialization function just before calling

280

<a class="reference internal" href="../c-api/type.html#c.PyType_Ready" title="PyType_Ready"><code class="xref c c-func docutils literal">PyType_Ready()</code></a>:

281

<div class="highlight-c"><div class="highlight"><pre>noddy_NoddyType.tp_new = PyType_GenericNew;

282

283

return;

284

</pre></div>

285

</div>

286

All the other type methods are NULL, so we’ll go over them later — that’s

287

for a later section!

288

Everything else in the file should be familiar, except for some code in

289

<code class="xref c c-func docutils literal">PyInit_noddy()</code>:

290

<div class="highlight-c"><div class="highlight"><pre>if (PyType_Ready(&noddy_NoddyType) < 0)

291

return;

292

</pre></div>

293

</div>

294

This initializes the <code class="xref py py-class docutils literal">Noddy</code> type, filing in a number of members,

295

including <code class="xref py py-attr docutils literal">ob_type</code> that we initially set to NULL.

296

<div class="highlight-c"><div class="highlight"><pre>PyModule_AddObject(m, "Noddy", (PyObject *)&noddy_NoddyType);

297

</pre></div>

298

</div>

299

This adds the type to the module dictionary. This allows us to create

300

<code class="xref py py-class docutils literal">Noddy</code> instances by calling the <code class="xref py py-class docutils literal">Noddy</code> class:

301

<div class="highlight-c"><div class="highlight"><pre>>>> import noddy

302

>>> mynoddy = noddy.Noddy()

303

</pre></div>

304

</div>

305

That’s it! All that remains is to build it; put the above code in a file called

306

<code class="file docutils literal">noddy.c</code> and

307

<div class="highlight-c"><div class="highlight"><pre>from distutils.core import setup, Extension

308

setup(name="noddy", version="1.0",

309

ext_modules=[Extension("noddy", ["noddy.c"])])

310

</pre></div>

311

</div>

312

in a file called <code class="file docutils literal">setup.py</code>; then typing

313

<div class="highlight-c"><div class="highlight"><pre>$ python setup.py build

314

</pre></div>

315

</div>

316

at a shell should produce a file <code class="file docutils literal">noddy.so</code> in a subdirectory; move to

317

that directory and fire up Python — you should be able to <code class="docutils literal">import noddy</code> and

318

play around with Noddy objects.

319

That wasn’t so hard, was it?

320

Of course, the current Noddy type is pretty uninteresting. It has no data and

321

doesn’t do anything. It can’t even be subclassed.

322

323

<h3>2.1.1. Adding data and methods to the Basic example<a class="headerlink" href="#adding-data-and-methods-to-the-basic-example" title="Permalink to this headline">¶</a></h3>

324

Let’s extend the basic example to add some data and methods. Let’s also make

325

the type usable as a base class. We’ll create a new module, <code class="xref py py-mod docutils literal">noddy2</code> that

326

adds these capabilities:

327

<div class="highlight-c"><div class="highlight"><pre>#include <Python.h>

328

#include "structmember.h"

329

330

typedef struct {

331

PyObject_HEAD

332

PyObject *first; /* first name */

333

PyObject *last; /* last name */

334

int number;

335

} Noddy;

336

337

static void

338

Noddy_dealloc(Noddy* self)

339

{

340

Py_XDECREF(self->first);

341

Py_XDECREF(self->last);

342

Py_TYPE(self)->tp_free((PyObject*)self);

343

}

344

345

static PyObject *

346

Noddy_new(PyTypeObject *type, PyObject *args, PyObject *kwds)

347

{

348

Noddy *self;

349

350

self = (Noddy *)type->tp_alloc(type, 0);

351

if (self != NULL) {

352

self->first = PyUnicode_FromString("");

353

if (self->first == NULL) {

354

Py_DECREF(self);

355

return NULL;

356

}

357

358

self->last = PyUnicode_FromString("");

359

if (self->last == NULL) {

360

Py_DECREF(self);

361

return NULL;

362

}

363

364

self->number = 0;

365

}

366

367

return (PyObject *)self;

368

}

369

370

static int

371

Noddy_init(Noddy *self, PyObject *args, PyObject *kwds)

372

{

373

PyObject *first=NULL, *last=NULL, *tmp;

374

375

static char *kwlist[] = {"first", "last", "number", NULL};

376

377

if (! PyArg_ParseTupleAndKeywords(args, kwds, "|OOi", kwlist,

378

&first, &last,

379

&self->number))

380

return -1;

381

382

if (first) {

383

tmp = self->first;

384

Py_INCREF(first);

385

self->first = first;

386

Py_XDECREF(tmp);

387

}

388

389

if (last) {

390

tmp = self->last;

391

Py_INCREF(last);

392

self->last = last;

393

Py_XDECREF(tmp);

394

}

395

396

return 0;

397

}

398

399

400

static PyMemberDef Noddy_members[] = {

401

{"first", T_OBJECT_EX, offsetof(Noddy, first), 0,

402

"first name"},

403

{"last", T_OBJECT_EX, offsetof(Noddy, last), 0,

404

"last name"},

405

{"number", T_INT, offsetof(Noddy, number), 0,

406

"noddy number"},

407

{NULL} /* Sentinel */

408

};

409

410

static PyObject *

411

Noddy_name(Noddy* self)

412

{

413

414

PyErr_SetString(PyExc_AttributeError, "first");

415

return NULL;

416

}

417

418

419

PyErr_SetString(PyExc_AttributeError, "last");

420

return NULL;

421

}

422

423

return PyUnicode_FromFormat("%S %S", self->first, self->last);

424

}

425

426

static PyMethodDef Noddy_methods[] = {

427

{"name", (PyCFunction)Noddy_name, METH_NOARGS,

428

"Return the name, combining the first and last name"

429

},

430

{NULL} /* Sentinel */

431

};

432

433

static PyTypeObject NoddyType = {

434

PyVarObject_HEAD_INIT(NULL, 0)

435

"noddy.Noddy", /* tp_name */

436

sizeof(Noddy), /* tp_basicsize */

437

0, /* tp_itemsize */

438

(destructor)Noddy_dealloc, /* tp_dealloc */

439

0, /* tp_print */

440

0, /* tp_getattr */

441

0, /* tp_setattr */

442

0, /* tp_reserved */

443

0, /* tp_repr */

444

0, /* tp_as_number */

445

0, /* tp_as_sequence */

446

0, /* tp_as_mapping */

447

0, /* tp_hash */

448

0, /* tp_call */

449

0, /* tp_str */

450

0, /* tp_getattro */

451

0, /* tp_setattro */

452

0, /* tp_as_buffer */

453

Py_TPFLAGS_DEFAULT |

454

Py_TPFLAGS_BASETYPE, /* tp_flags */

455

"Noddy objects", /* tp_doc */

456

0, /* tp_traverse */

457

0, /* tp_clear */

458

0, /* tp_richcompare */

459

0, /* tp_weaklistoffset */

460

0, /* tp_iter */

461

0, /* tp_iternext */

462

Noddy_methods, /* tp_methods */

463

Noddy_members, /* tp_members */

464

0, /* tp_getset */

465

0, /* tp_base */

466

0, /* tp_dict */

467

0, /* tp_descr_get */

468

0, /* tp_descr_set */

469

0, /* tp_dictoffset */

470

(initproc)Noddy_init, /* tp_init */

471

0, /* tp_alloc */

472

Noddy_new, /* tp_new */

473

};

474

475

static PyModuleDef noddy2module = {

476

PyModuleDef_HEAD_INIT,

477

"noddy2",

478

"Example module that creates an extension type.",

479

-1,

480

481

};

482

483

PyMODINIT_FUNC

484

PyInit_noddy2(void)

485

{

486

PyObject* m;

487

488

if (PyType_Ready(&NoddyType) < 0)

489

return NULL;

490

491

m = PyModule_Create(&noddy2module);

492

if (m == NULL)

493

return NULL;

494

495

Py_INCREF(&NoddyType);

496

497

return m;

498

}

499

</pre></div>

500

</div>

501

This version of the module has a number of changes.

502

We’ve added an extra include:

503

<div class="highlight-c"><div class="highlight"><pre>#include <structmember.h>

504

</pre></div>

505

</div>

506

This include provides declarations that we use to handle attributes, as

507

described a bit later.

508

The name of the <code class="xref py py-class docutils literal">Noddy</code> object structure has been shortened to

509

<code class="xref py py-class docutils literal">Noddy</code>. The type object name has been shortened to <code class="xref py py-class docutils literal">NoddyType</code>.

510

The <code class="xref py py-class docutils literal">Noddy</code> type now has three data attributes, first, last, and

511

number. The first and last variables are Python strings containing first

512

and last names. The number attribute is an integer.

513

The object structure is updated accordingly:

514

<div class="highlight-c"><div class="highlight"><pre>typedef struct {

515

PyObject_HEAD

516

PyObject *first;

517

PyObject *last;

518

int number;

519

} Noddy;

520

</pre></div>

521

</div>

522

Because we now have data to manage, we have to be more careful about object

523

allocation and deallocation. At a minimum, we need a deallocation method:

524

<div class="highlight-c"><div class="highlight"><pre>static void

525

Noddy_dealloc(Noddy* self)

526

{

527

Py_XDECREF(self->first);

528

Py_XDECREF(self->last);

529

530

}

531

</pre></div>

532

</div>

533

which is assigned to the <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_dealloc" title="PyTypeObject.tp_dealloc"><code class="xref c c-member docutils literal">tp_dealloc</code></a> member:

534

<div class="highlight-c"><div class="highlight"><pre>(destructor)Noddy_dealloc, /*tp_dealloc*/

535

</pre></div>

536

</div>

537

This method decrements the reference counts of the two Python attributes. We use

538

<a class="reference internal" href="../c-api/refcounting.html#c.Py_XDECREF" title="Py_XDECREF"><code class="xref c c-func docutils literal">Py_XDECREF()</code></a> here because the <code class="xref py py-attr docutils literal">first</code> and <code class="xref py py-attr docutils literal">last</code> members

539

could be NULL. It then calls the <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_free" title="PyTypeObject.tp_free"><code class="xref c c-member docutils literal">tp_free</code></a> member of the object’s type

540

to free the object’s memory. Note that the object’s type might not be

541

<code class="xref py py-class docutils literal">NoddyType</code>, because the object may be an instance of a subclass.

542

We want to make sure that the first and last names are initialized to empty

543

strings, so we provide a new method:

544

<div class="highlight-c"><div class="highlight"><pre>static PyObject *

545

546

{

547

Noddy *self;

548

549

550

if (self != NULL) {

551

552

553

Py_DECREF(self);

554

return NULL;

555

}

556

557

558

559

Py_DECREF(self);

560

return NULL;

561

}

562

563

self->number = 0;

564

}

565

566

return (PyObject *)self;

567

}

568

</pre></div>

569

</div>

570

and install it in the <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_new" title="PyTypeObject.tp_new"><code class="xref c c-member docutils literal">tp_new</code></a> member:

571

<div class="highlight-c"><div class="highlight"><pre>Noddy_new, /* tp_new */

572

</pre></div>

573

</div>

574

The new member is responsible for creating (as opposed to initializing) objects

575

of the type. It is exposed in Python as the <a class="reference internal" href="../reference/datamodel.html#object.__new__" title="object.__new__"><code class="xref py py-meth docutils literal">__new__()</code></a> method. See the

576

paper titled “Unifying types and classes in Python” for a detailed discussion of

577

the <a class="reference internal" href="../reference/datamodel.html#object.__new__" title="object.__new__"><code class="xref py py-meth docutils literal">__new__()</code></a> method. One reason to implement a new method is to assure

578

the initial values of instance variables. In this case, we use the new method

579

to make sure that the initial values of the members <code class="xref py py-attr docutils literal">first</code> and

580

<code class="xref py py-attr docutils literal">last</code> are not NULL. If we didn’t care whether the initial values were

581

NULL, we could have used <a class="reference internal" href="../c-api/type.html#c.PyType_GenericNew" title="PyType_GenericNew"><code class="xref c c-func docutils literal">PyType_GenericNew()</code></a> as our new method, as we

582

did before. <a class="reference internal" href="../c-api/type.html#c.PyType_GenericNew" title="PyType_GenericNew"><code class="xref c c-func docutils literal">PyType_GenericNew()</code></a> initializes all of the instance variable

583

members to NULL.

584

The new method is a static method that is passed the type being instantiated and

585

any arguments passed when the type was called, and that returns the new object

586

created. New methods always accept positional and keyword arguments, but they

587

often ignore the arguments, leaving the argument handling to initializer

588

methods. Note that if the type supports subclassing, the type passed may not be

589

the type being defined. The new method calls the <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_alloc" title="PyTypeObject.tp_alloc"><code class="xref c c-member docutils literal">tp_alloc</code></a> slot to

590

allocate memory. We don’t fill the <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_alloc" title="PyTypeObject.tp_alloc"><code class="xref c c-member docutils literal">tp_alloc</code></a> slot ourselves. Rather

591

<a class="reference internal" href="../c-api/type.html#c.PyType_Ready" title="PyType_Ready"><code class="xref c c-func docutils literal">PyType_Ready()</code></a> fills it for us by inheriting it from our base class,

592

which is <a class="reference internal" href="../library/functions.html#object" title="object"><code class="xref py py-class docutils literal">object</code></a> by default. Most types use the default allocation.

593

594

Note

595

If you are creating a co-operative <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_new" title="PyTypeObject.tp_new"><code class="xref c c-member docutils literal">tp_new</code></a> (one that calls a base type’s

596

<a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_new" title="PyTypeObject.tp_new"><code class="xref c c-member docutils literal">tp_new</code></a> or <a class="reference internal" href="../reference/datamodel.html#object.__new__" title="object.__new__"><code class="xref py py-meth docutils literal">__new__()</code></a>), you must not try to determine what method

597

to call using method resolution order at runtime. Always statically determine

598

what type you are going to call, and call its <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_new" title="PyTypeObject.tp_new"><code class="xref c c-member docutils literal">tp_new</code></a> directly, or via

599

<code class="docutils literal">type->tp_base->tp_new</code>. If you do not do this, Python subclasses of your

600

type that also inherit from other Python-defined classes may not work correctly.

601

(Specifically, you may not be able to create instances of such subclasses

602

without getting a <a class="reference internal" href="../library/exceptions.html#TypeError" title="TypeError"><code class="xref py py-exc docutils literal">TypeError</code></a>.)

603

</div>

604

We provide an initialization function:

605

<div class="highlight-c"><div class="highlight"><pre>static int

606

607

{

608

609

610

611

612

613

&first, &last,

614

&self->number))

615

return -1;

616

617

if (first) {

618

tmp = self->first;

619

Py_INCREF(first);

620

self->first = first;

621

Py_XDECREF(tmp);

622

}

623

624

if (last) {

625

tmp = self->last;

626

Py_INCREF(last);

627

self->last = last;

628

Py_XDECREF(tmp);

629

}

630

631

return 0;

632

}

633

</pre></div>

634

</div>

635

by filling the <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_init" title="PyTypeObject.tp_init"><code class="xref c c-member docutils literal">tp_init</code></a> slot.

636

<div class="highlight-c"><div class="highlight"><pre>(initproc)Noddy_init, /* tp_init */

637

</pre></div>

638

</div>

639

The <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_init" title="PyTypeObject.tp_init"><code class="xref c c-member docutils literal">tp_init</code></a> slot is exposed in Python as the <a class="reference internal" href="../reference/datamodel.html#object.__init__" title="object.__init__"><code class="xref py py-meth docutils literal">__init__()</code></a> method. It

640

is used to initialize an object after it’s created. Unlike the new method, we

641

can’t guarantee that the initializer is called. The initializer isn’t called

642

when unpickling objects and it can be overridden. Our initializer accepts

643

arguments to provide initial values for our instance. Initializers always accept

644

positional and keyword arguments. Initializers should return either 0 on

645

success or -1 on error.

646

Initializers can be called multiple times. Anyone can call the <a class="reference internal" href="../reference/datamodel.html#object.__init__" title="object.__init__"><code class="xref py py-meth docutils literal">__init__()</code></a>

647

method on our objects. For this reason, we have to be extra careful when

648

assigning the new values. We might be tempted, for example to assign the

649

<code class="xref py py-attr docutils literal">first</code> member like this:

650

<div class="highlight-c"><div class="highlight"><pre>if (first) {

651

Py_XDECREF(self->first);

652

Py_INCREF(first);

653

self->first = first;

654

}

655

</pre></div>

656

</div>

657

But this would be risky. Our type doesn’t restrict the type of the

658

<code class="xref py py-attr docutils literal">first</code> member, so it could be any kind of object. It could have a

659

destructor that causes code to be executed that tries to access the

660

<code class="xref py py-attr docutils literal">first</code> member. To be paranoid and protect ourselves against this

661

possibility, we almost always reassign members before decrementing their

662

reference counts. When don’t we have to do this?

663

664

<li>when we absolutely know that the reference count is greater than 1</li>

665

<li>when we know that deallocation of the object <a class="footnote-reference" href="#id7" id="id2">[1]</a> will not cause any calls

666

back into our type’s code</li>

667

<li>when decrementing a reference count in a <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_dealloc" title="PyTypeObject.tp_dealloc"><code class="xref c c-member docutils literal">tp_dealloc</code></a> handler when

668

garbage-collections is not supported <a class="footnote-reference" href="#id8" id="id3">[2]</a></li>

669

</ul>

670

We want to expose our instance variables as attributes. There are a

671

number of ways to do that. The simplest way is to define member definitions:

672

<div class="highlight-c"><div class="highlight"><pre>static PyMemberDef Noddy_members[] = {

673

674

"first name"},

675

676

"last name"},

677

678

"noddy number"},

679

{NULL} /* Sentinel */

680

};

681

</pre></div>

682

</div>

683

and put the definitions in the <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_members" title="PyTypeObject.tp_members"><code class="xref c c-member docutils literal">tp_members</code></a> slot:

684

<div class="highlight-c"><div class="highlight"><pre>Noddy_members, /* tp_members */

685

</pre></div>

686

</div>

687

Each member definition has a member name, type, offset, access flags and

688

documentation string. See the <a class="reference internal" href="#generic-attribute-management">Generic Attribute Management</a> section below for

689

details.

690

A disadvantage of this approach is that it doesn’t provide a way to restrict the

691

types of objects that can be assigned to the Python attributes. We expect the

692

first and last names to be strings, but any Python objects can be assigned.

693

Further, the attributes can be deleted, setting the C pointers to NULL. Even

694

though we can make sure the members are initialized to non-NULL values, the

695

members can be set to NULL if the attributes are deleted.

696

We define a single method, <code class="xref py py-meth docutils literal">name()</code>, that outputs the objects name as the

697

concatenation of the first and last names.

698

<div class="highlight-c"><div class="highlight"><pre>static PyObject *

699

Noddy_name(Noddy* self)

700

{

701

702

PyErr_SetString(PyExc_AttributeError, "first");

703

return NULL;

704

}

705

706

707

PyErr_SetString(PyExc_AttributeError, "last");

708

return NULL;

709

}

710

711

712

}

713

</pre></div>

714

</div>

715

The method is implemented as a C function that takes a <code class="xref py py-class docutils literal">Noddy</code> (or

716

<code class="xref py py-class docutils literal">Noddy</code> subclass) instance as the first argument. Methods always take an

717

instance as the first argument. Methods often take positional and keyword

718

arguments as well, but in this case we don’t take any and don’t need to accept

719

a positional argument tuple or keyword argument dictionary. This method is

720

equivalent to the Python method:

721

<div class="highlight-c"><div class="highlight"><pre>def name(self):

722

return "%s %s" % (self.first, self.last)

723

</pre></div>

724

</div>

725

Note that we have to check for the possibility that our <code class="xref py py-attr docutils literal">first</code> and

726

<code class="xref py py-attr docutils literal">last</code> members are NULL. This is because they can be deleted, in which

727

case they are set to NULL. It would be better to prevent deletion of these

728

attributes and to restrict the attribute values to be strings. We’ll see how to

729

do that in the next section.

730

Now that we’ve defined the method, we need to create an array of method

731

definitions:

732

<div class="highlight-c"><div class="highlight"><pre>static PyMethodDef Noddy_methods[] = {

733

734

"Return the name, combining the first and last name"

735

},

736

{NULL} /* Sentinel */

737

};

738

</pre></div>

739

</div>

740

and assign them to the <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_methods" title="PyTypeObject.tp_methods"><code class="xref c c-member docutils literal">tp_methods</code></a> slot:

741

<div class="highlight-c"><div class="highlight"><pre>Noddy_methods, /* tp_methods */

742

</pre></div>

743

</div>

744

Note that we used the <a class="reference internal" href="../c-api/structures.html#METH_NOARGS" title="METH_NOARGS"><code class="xref py py-const docutils literal">METH_NOARGS</code></a> flag to indicate that the method is

745

passed no arguments.

746

Finally, we’ll make our type usable as a base class. We’ve written our methods

747

carefully so far so that they don’t make any assumptions about the type of the

748

object being created or used, so all we need to do is to add the

749

<a class="reference internal" href="../c-api/typeobj.html#Py_TPFLAGS_BASETYPE" title="Py_TPFLAGS_BASETYPE"><code class="xref py py-const docutils literal">Py_TPFLAGS_BASETYPE</code></a> to our class flag definition:

750

<div class="highlight-c"><div class="highlight"><pre>Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /*tp_flags*/

751

</pre></div>

752

</div>

753

We rename <code class="xref c c-func docutils literal">PyInit_noddy()</code> to <code class="xref c c-func docutils literal">PyInit_noddy2()</code> and update the module

754

name in the <a class="reference internal" href="../c-api/module.html#c.PyModuleDef" title="PyModuleDef"><code class="xref c c-type docutils literal">PyModuleDef</code></a> struct.

755

Finally, we update our <code class="file docutils literal">setup.py</code> file to build the new module:

756

757

758

ext_modules=[

759

Extension("noddy", ["noddy.c"]),

760

Extension("noddy2", ["noddy2.c"]),

761

])

762

</pre></div>

763

</div>

764

</div>

765

766

<h3>2.1.2. Providing finer control over data attributes<a class="headerlink" href="#providing-finer-control-over-data-attributes" title="Permalink to this headline">¶</a></h3>

767

In this section, we’ll provide finer control over how the <code class="xref py py-attr docutils literal">first</code> and

768

<code class="xref py py-attr docutils literal">last</code> attributes are set in the <code class="xref py py-class docutils literal">Noddy</code> example. In the previous

769

version of our module, the instance variables <code class="xref py py-attr docutils literal">first</code> and <code class="xref py py-attr docutils literal">last</code>

770

could be set to non-string values or even deleted. We want to make sure that

771

these attributes always contain strings.

772

<div class="highlight-c"><div class="highlight"><pre>#include <Python.h>

773

#include "structmember.h"

774

775

typedef struct {

776

PyObject_HEAD

777

PyObject *first;

778

PyObject *last;

779

int number;

780

} Noddy;

781

782

static void

783

Noddy_dealloc(Noddy* self)

784

{

785

Py_XDECREF(self->first);

786

Py_XDECREF(self->last);

787

788

}

789

790

static PyObject *

791

792

{

793

Noddy *self;

794

795

796

if (self != NULL) {

797

798

799

Py_DECREF(self);

800

return NULL;

801

}

802

803

804

805

Py_DECREF(self);

806

return NULL;

807

}

808

809

self->number = 0;

810

}

811

812

return (PyObject *)self;

813

}

814

815

static int

816

817

{

818

819

820

821

822

if (! PyArg_ParseTupleAndKeywords(args, kwds, "|SSi", kwlist,

823

&first, &last,

824

&self->number))

825

return -1;

826

827

if (first) {

828

tmp = self->first;

829

Py_INCREF(first);

830

self->first = first;

831

Py_DECREF(tmp);

832

}

833

834

if (last) {

835

tmp = self->last;

836

Py_INCREF(last);

837

self->last = last;

838

Py_DECREF(tmp);

839

}

840

841

return 0;

842

}

843

844

static PyMemberDef Noddy_members[] = {

845

846

"noddy number"},

847

{NULL} /* Sentinel */

848

};

849

850

static PyObject *

851

Noddy_getfirst(Noddy *self, void *closure)

852

{

853

Py_INCREF(self->first);

854

return self->first;

855

}

856

857

static int

858

Noddy_setfirst(Noddy *self, PyObject *value, void *closure)

859

{

860

if (value == NULL) {

861

PyErr_SetString(PyExc_TypeError, "Cannot delete the first attribute");

862

return -1;

863

}

864

865

if (! PyUnicode_Check(value)) {

866

PyErr_SetString(PyExc_TypeError,

867

"The first attribute value must be a string");

868

return -1;

869

}

870

871

Py_DECREF(self->first);

872

Py_INCREF(value);

873

self->first = value;

874

875

return 0;

876

}

877

878

static PyObject *

879

Noddy_getlast(Noddy *self, void *closure)

880

{

881

Py_INCREF(self->last);

882

return self->last;

883

}

884

885

static int

886

Noddy_setlast(Noddy *self, PyObject *value, void *closure)

887

{

888

if (value == NULL) {

889

890

return -1;

891

}

892

893

894

PyErr_SetString(PyExc_TypeError,

895

"The last attribute value must be a string");

896

return -1;

897

}

898

899

Py_DECREF(self->last);

900

Py_INCREF(value);

901

self->last = value;

902

903

return 0;

904

}

905

906

static PyGetSetDef Noddy_getseters[] = {

907

{"first",

908

(getter)Noddy_getfirst, (setter)Noddy_setfirst,

909

"first name",

910

NULL},

911

{"last",

912

(getter)Noddy_getlast, (setter)Noddy_setlast,

913

"last name",

914

NULL},

915

{NULL} /* Sentinel */

916

};

917

918

static PyObject *

919

Noddy_name(Noddy* self)

920

{

921

922

}

923

924

static PyMethodDef Noddy_methods[] = {

925

926

"Return the name, combining the first and last name"

927

},

928

{NULL} /* Sentinel */

929

};

930

931

static PyTypeObject NoddyType = {

932

PyVarObject_HEAD_INIT(NULL, 0)

933

"noddy.Noddy", /* tp_name */

934

sizeof(Noddy), /* tp_basicsize */

935

0, /* tp_itemsize */

936

(destructor)Noddy_dealloc, /* tp_dealloc */

937

0, /* tp_print */

938

0, /* tp_getattr */

939

0, /* tp_setattr */

940

0, /* tp_reserved */

941

0, /* tp_repr */

942

0, /* tp_as_number */

943

0, /* tp_as_sequence */

944

0, /* tp_as_mapping */

945

0, /* tp_hash */

946

0, /* tp_call */

947

0, /* tp_str */

948

0, /* tp_getattro */

949

0, /* tp_setattro */

950

0, /* tp_as_buffer */

951

Py_TPFLAGS_DEFAULT |

952

Py_TPFLAGS_BASETYPE, /* tp_flags */

953

"Noddy objects", /* tp_doc */

954

0, /* tp_traverse */

955

0, /* tp_clear */

956

0, /* tp_richcompare */

957

0, /* tp_weaklistoffset */

958

0, /* tp_iter */

959

0, /* tp_iternext */

960

Noddy_methods, /* tp_methods */

961

Noddy_members, /* tp_members */

962

Noddy_getseters, /* tp_getset */

963

0, /* tp_base */

964

0, /* tp_dict */

965

0, /* tp_descr_get */

966

0, /* tp_descr_set */

967

0, /* tp_dictoffset */

968

(initproc)Noddy_init, /* tp_init */

969

0, /* tp_alloc */

970

Noddy_new, /* tp_new */

971

};

972

973

static PyModuleDef noddy3module = {

974

PyModuleDef_HEAD_INIT,

975

"noddy3",

976

"Example module that creates an extension type.",

977

-1,

978

979

};

980

981

PyMODINIT_FUNC

982

PyInit_noddy3(void)

983

{

984

PyObject* m;

985

986

987

return NULL;

988

989

m = PyModule_Create(&noddy3module);

990

if (m == NULL)

991

return NULL;

992

993

Py_INCREF(&NoddyType);

994

995

return m;

996

}

997

</pre></div>

998

</div>

999

To provide greater control, over the <code class="xref py py-attr docutils literal">first</code> and <code class="xref py py-attr docutils literal">last</code> attributes,

1000

we’ll use custom getter and setter functions. Here are the functions for

1001

getting and setting the <code class="xref py py-attr docutils literal">first</code> attribute:

1002

<div class="highlight-c"><div class="highlight"><pre>Noddy_getfirst(Noddy *self, void *closure)

1003

{

1004

Py_INCREF(self->first);

1005

return self->first;

1006

}

1007

1008

static int

1009

Noddy_setfirst(Noddy *self, PyObject *value, void *closure)

1010

{

1011

if (value == NULL) {

1012

1013

return -1;

1014

}

1015

1016

1017

PyErr_SetString(PyExc_TypeError,

1018

"The first attribute value must be a str");

1019

return -1;

1020

}

1021

1022

Py_DECREF(self->first);

1023

Py_INCREF(value);

1024

self->first = value;

1025

1026

return 0;

1027

}

1028

</pre></div>

1029

</div>

1030

The getter function is passed a <code class="xref py py-class docutils literal">Noddy</code> object and a “closure”, which is

1031

void pointer. In this case, the closure is ignored. (The closure supports an

1032

advanced usage in which definition data is passed to the getter and setter. This

1033

could, for example, be used to allow a single set of getter and setter functions

1034

that decide the attribute to get or set based on data in the closure.)

1035

The setter function is passed the <code class="xref py py-class docutils literal">Noddy</code> object, the new value, and the

1036

closure. The new value may be NULL, in which case the attribute is being

1037

deleted. In our setter, we raise an error if the attribute is deleted or if the

1038

attribute value is not a string.

1039

We create an array of <code class="xref c c-type docutils literal">PyGetSetDef</code> structures:

1040

<div class="highlight-c"><div class="highlight"><pre>static PyGetSetDef Noddy_getseters[] = {

1041

{"first",

1042

1043

"first name",

1044

NULL},

1045

{"last",

1046

1047

"last name",

1048

NULL},

1049

{NULL} /* Sentinel */

1050

};

1051

</pre></div>

1052

</div>

1053

and register it in the <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_getset" title="PyTypeObject.tp_getset"><code class="xref c c-member docutils literal">tp_getset</code></a> slot:

1054

<div class="highlight-c"><div class="highlight"><pre>Noddy_getseters, /* tp_getset */

1055

</pre></div>

1056

</div>

1057

to register our attribute getters and setters.

1058

The last item in a <code class="xref c c-type docutils literal">PyGetSetDef</code> structure is the closure mentioned

1059

above. In this case, we aren’t using the closure, so we just pass NULL.

1060

We also remove the member definitions for these attributes:

1061

1062

1063

"noddy number"},

1064

{NULL} /* Sentinel */

1065

};

1066

</pre></div>

1067

</div>

1068

We also need to update the <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_init" title="PyTypeObject.tp_init"><code class="xref c c-member docutils literal">tp_init</code></a> handler to only allow strings <a class="footnote-reference" href="#id9" id="id4">[3]</a> to

1069

be passed:

1070

<div class="highlight-c"><div class="highlight"><pre>static int

1071

1072

{

1073

1074

1075

1076

1077

if (! PyArg_ParseTupleAndKeywords(args, kwds, "|SSi", kwlist,

1078

&first, &last,

1079

&self->number))

1080

return -1;

1081

1082

if (first) {

1083

tmp = self->first;

1084

Py_INCREF(first);

1085

self->first = first;

1086

Py_DECREF(tmp);

1087

}

1088

1089

if (last) {

1090

tmp = self->last;

1091

Py_INCREF(last);

1092

self->last = last;

1093

Py_DECREF(tmp);

1094

}

1095

1096

return 0;

1097

}

1098

</pre></div>

1099

</div>

1100

With these changes, we can assure that the <code class="xref py py-attr docutils literal">first</code> and <code class="xref py py-attr docutils literal">last</code>

1101

members are never NULL so we can remove checks for NULL values in almost all

1102

cases. This means that most of the <a class="reference internal" href="../c-api/refcounting.html#c.Py_XDECREF" title="Py_XDECREF"><code class="xref c c-func docutils literal">Py_XDECREF()</code></a> calls can be converted to

1103

<a class="reference internal" href="../c-api/refcounting.html#c.Py_DECREF" title="Py_DECREF"><code class="xref c c-func docutils literal">Py_DECREF()</code></a> calls. The only place we can’t change these calls is in the

1104

deallocator, where there is the possibility that the initialization of these

1105

members failed in the constructor.

1106

We also rename the module initialization function and module name in the

1107

initialization function, as we did before, and we add an extra definition to the

1108

<code class="file docutils literal">setup.py</code> file.

1109

</div>

1110

1111

<h3>2.1.3. Supporting cyclic garbage collection<a class="headerlink" href="#supporting-cyclic-garbage-collection" title="Permalink to this headline">¶</a></h3>

1112

Python has a cyclic-garbage collector that can identify unneeded objects even

1113

when their reference counts are not zero. This can happen when objects are

1114

involved in cycles. For example, consider:

1115

<div class="highlight-c"><div class="highlight"><pre>>>> l = []

1116

>>> l.append(l)

1117

>>> del l

1118

</pre></div>

1119

</div>

1120

In this example, we create a list that contains itself. When we delete it, it

1121

still has a reference from itself. Its reference count doesn’t drop to zero.

1122

Fortunately, Python’s cyclic-garbage collector will eventually figure out that

1123

the list is garbage and free it.

1124

In the second version of the <code class="xref py py-class docutils literal">Noddy</code> example, we allowed any kind of

1125

object to be stored in the <code class="xref py py-attr docutils literal">first</code> or <code class="xref py py-attr docutils literal">last</code> attributes. <a class="footnote-reference" href="#id10" id="id5">[4]</a> This

1126

means that <code class="xref py py-class docutils literal">Noddy</code> objects can participate in cycles:

1127

<div class="highlight-c"><div class="highlight"><pre>>>> import noddy2

1128

>>> n = noddy2.Noddy()

1129

>>> l = [n]

1130

>>> n.first = l

1131

</pre></div>

1132

</div>

1133

This is pretty silly, but it gives us an excuse to add support for the

1134

cyclic-garbage collector to the <code class="xref py py-class docutils literal">Noddy</code> example. To support cyclic

1135

garbage collection, types need to fill two slots and set a class flag that

1136

enables these slots:

1137

<div class="highlight-c"><div class="highlight"><pre>#include <Python.h>

1138

#include "structmember.h"

1139

1140

typedef struct {

1141

PyObject_HEAD

1142

PyObject *first;

1143

PyObject *last;

1144

int number;

1145

} Noddy;

1146

1147

static int

1148

Noddy_traverse(Noddy *self, visitproc visit, void *arg)

1149

{

1150

int vret;

1151

1152

if (self->first) {

1153

vret = visit(self->first, arg);

1154

if (vret != 0)

1155

return vret;

1156

}

1157

if (self->last) {

1158

vret = visit(self->last, arg);

1159

if (vret != 0)

1160

return vret;

1161

}

1162

1163

return 0;

1164

}

1165

1166

static int

1167

Noddy_clear(Noddy *self)

1168

{

1169

PyObject *tmp;

1170

1171

tmp = self->first;

1172

self->first = NULL;

1173

Py_XDECREF(tmp);

1174

1175

tmp = self->last;

1176

self->last = NULL;

1177

Py_XDECREF(tmp);

1178

1179

return 0;

1180

}

1181

1182

static void

1183

Noddy_dealloc(Noddy* self)

1184

{

1185

Noddy_clear(self);

1186

1187

}

1188

1189

static PyObject *

1190

1191

{

1192

Noddy *self;

1193

1194

1195

if (self != NULL) {

1196

1197

1198

Py_DECREF(self);

1199

return NULL;

1200

}

1201

1202

1203

1204

Py_DECREF(self);

1205

return NULL;

1206

}

1207

1208

self->number = 0;

1209

}

1210

1211

return (PyObject *)self;

1212

}

1213

1214

static int

1215

1216

{

1217

1218

1219

1220

1221

1222

&first, &last,

1223

&self->number))

1224

return -1;

1225

1226

if (first) {

1227

tmp = self->first;

1228

Py_INCREF(first);

1229

self->first = first;

1230

Py_XDECREF(tmp);

1231

}

1232

1233

if (last) {

1234

tmp = self->last;

1235

Py_INCREF(last);

1236

self->last = last;

1237

Py_XDECREF(tmp);

1238

}

1239

1240

return 0;

1241

}

1242

1243

1244

static PyMemberDef Noddy_members[] = {

1245

1246

"first name"},

1247

1248

"last name"},

1249

1250

"noddy number"},

1251

{NULL} /* Sentinel */

1252

};

1253

1254

static PyObject *

1255

Noddy_name(Noddy* self)

1256

{

1257

1258

PyErr_SetString(PyExc_AttributeError, "first");

1259

return NULL;

1260

}

1261

1262

1263

PyErr_SetString(PyExc_AttributeError, "last");

1264

return NULL;

1265

}

1266

1267

1268

}

1269

1270

static PyMethodDef Noddy_methods[] = {

1271

1272

"Return the name, combining the first and last name"

1273

},

1274

{NULL} /* Sentinel */

1275

};

1276

1277

static PyTypeObject NoddyType = {

1278

PyVarObject_HEAD_INIT(NULL, 0)

1279

"noddy.Noddy", /* tp_name */

1280

sizeof(Noddy), /* tp_basicsize */

1281

0, /* tp_itemsize */

1282

(destructor)Noddy_dealloc, /* tp_dealloc */

1283

0, /* tp_print */

1284

0, /* tp_getattr */

1285

0, /* tp_setattr */

1286

0, /* tp_reserved */

1287

0, /* tp_repr */

1288

0, /* tp_as_number */

1289

0, /* tp_as_sequence */

1290

0, /* tp_as_mapping */

1291

0, /* tp_hash */

1292

0, /* tp_call */

1293

0, /* tp_str */

1294

0, /* tp_getattro */

1295

0, /* tp_setattro */

1296

0, /* tp_as_buffer */

1297

Py_TPFLAGS_DEFAULT |

1298

Py_TPFLAGS_BASETYPE |

1299

Py_TPFLAGS_HAVE_GC, /* tp_flags */

1300

"Noddy objects", /* tp_doc */

1301

(traverseproc)Noddy_traverse, /* tp_traverse */

1302

(inquiry)Noddy_clear, /* tp_clear */

1303

0, /* tp_richcompare */

1304

0, /* tp_weaklistoffset */

1305

0, /* tp_iter */

1306

0, /* tp_iternext */

1307

Noddy_methods, /* tp_methods */

1308

Noddy_members, /* tp_members */

1309

0, /* tp_getset */

1310

0, /* tp_base */

1311

0, /* tp_dict */

1312

0, /* tp_descr_get */

1313

0, /* tp_descr_set */

1314

0, /* tp_dictoffset */

1315

(initproc)Noddy_init, /* tp_init */

1316

0, /* tp_alloc */

1317

Noddy_new, /* tp_new */

1318

};

1319

1320

static PyModuleDef noddy4module = {

1321

PyModuleDef_HEAD_INIT,

1322

"noddy4",

1323

"Example module that creates an extension type.",

1324

-1,

1325

1326

};

1327

1328

PyMODINIT_FUNC

1329

PyInit_noddy4(void)

1330

{

1331

PyObject* m;

1332

1333

1334

return NULL;

1335

1336

m = PyModule_Create(&noddy4module);

1337

if (m == NULL)

1338

return NULL;

1339

1340

Py_INCREF(&NoddyType);

1341

1342

return m;

1343

}

1344

</pre></div>

1345

</div>

1346

The traversal method provides access to subobjects that could participate in

1347

cycles:

1348

<div class="highlight-c"><div class="highlight"><pre>static int

1349

1350

{

1351

int vret;

1352

1353

if (self->first) {

1354

1355

if (vret != 0)

1356

return vret;

1357

}

1358

if (self->last) {

1359

vret = visit(self->last, arg);

1360

if (vret != 0)

1361

return vret;

1362

}

1363

1364

return 0;

1365

}

1366

</pre></div>

1367

</div>

1368

For each subobject that can participate in cycles, we need to call the

1369

<code class="xref c c-func docutils literal">visit()</code> function, which is passed to the traversal method. The

1370

<code class="xref c c-func docutils literal">visit()</code> function takes as arguments the subobject and the extra argument

1371

arg passed to the traversal method. It returns an integer value that must be

1372

returned if it is non-zero.

1373

Python provides a <a class="reference internal" href="../c-api/gcsupport.html#c.Py_VISIT" title="Py_VISIT"><code class="xref c c-func docutils literal">Py_VISIT()</code></a> macro that automates calling visit

1374

functions. With <a class="reference internal" href="../c-api/gcsupport.html#c.Py_VISIT" title="Py_VISIT"><code class="xref c c-func docutils literal">Py_VISIT()</code></a>, <code class="xref c c-func docutils literal">Noddy_traverse()</code> can be simplified:

1375

<div class="highlight-c"><div class="highlight"><pre>static int

1376

1377

{

1378

Py_VISIT(self->first);

1379

Py_VISIT(self->last);

1380

return 0;

1381

}

1382

</pre></div>

1383

</div>

1384

1385

Note

1386

Note that the <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_traverse" title="PyTypeObject.tp_traverse"><code class="xref c c-member docutils literal">tp_traverse</code></a> implementation must name its arguments exactly

1387

visit and arg in order to use <a class="reference internal" href="../c-api/gcsupport.html#c.Py_VISIT" title="Py_VISIT"><code class="xref c c-func docutils literal">Py_VISIT()</code></a>. This is to encourage

1388

uniformity across these boring implementations.

1389

</div>

1390

We also need to provide a method for clearing any subobjects that can

1391

participate in cycles. We implement the method and reimplement the deallocator

1392

to use it:

1393

<div class="highlight-c"><div class="highlight"><pre>static int

1394

Noddy_clear(Noddy *self)

1395

{

1396

PyObject *tmp;

1397

1398

tmp = self->first;

1399

self->first = NULL;

1400

Py_XDECREF(tmp);

1401

1402

tmp = self->last;

1403

self->last = NULL;

1404

Py_XDECREF(tmp);

1405

1406

return 0;

1407

}

1408

1409

static void

1410

Noddy_dealloc(Noddy* self)

1411

{

1412

Noddy_clear(self);

1413

1414

}

1415

</pre></div>

1416

</div>

1417

Notice the use of a temporary variable in <code class="xref c c-func docutils literal">Noddy_clear()</code>. We use the

1418

temporary variable so that we can set each member to NULL before decrementing

1419

its reference count. We do this because, as was discussed earlier, if the

1420

reference count drops to zero, we might cause code to run that calls back into

1421

the object. In addition, because we now support garbage collection, we also

1422

have to worry about code being run that triggers garbage collection. If garbage

1423

collection is run, our <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_traverse" title="PyTypeObject.tp_traverse"><code class="xref c c-member docutils literal">tp_traverse</code></a> handler could get called. We can’t

1424

take a chance of having <code class="xref c c-func docutils literal">Noddy_traverse()</code> called when a member’s reference

1425

count has dropped to zero and its value hasn’t been set to NULL.

1426

Python provides a <a class="reference internal" href="../c-api/refcounting.html#c.Py_CLEAR" title="Py_CLEAR"><code class="xref c c-func docutils literal">Py_CLEAR()</code></a> that automates the careful decrementing of

1427

reference counts. With <a class="reference internal" href="../c-api/refcounting.html#c.Py_CLEAR" title="Py_CLEAR"><code class="xref c c-func docutils literal">Py_CLEAR()</code></a>, the <code class="xref c c-func docutils literal">Noddy_clear()</code> function can

1428

be simplified:

1429

<div class="highlight-c"><div class="highlight"><pre>static int

1430

Noddy_clear(Noddy *self)

1431

{

1432

Py_CLEAR(self->first);

1433

Py_CLEAR(self->last);

1434

return 0;

1435

}

1436

</pre></div>

1437

</div>

1438

Finally, we add the <a class="reference internal" href="../c-api/typeobj.html#Py_TPFLAGS_HAVE_GC" title="Py_TPFLAGS_HAVE_GC"><code class="xref py py-const docutils literal">Py_TPFLAGS_HAVE_GC</code></a> flag to the class flags:

1439

<div class="highlight-c"><div class="highlight"><pre>Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC, /* tp_flags */

1440

</pre></div>

1441

</div>

1442

That’s pretty much it. If we had written custom <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_alloc" title="PyTypeObject.tp_alloc"><code class="xref c c-member docutils literal">tp_alloc</code></a> or

1443

<a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_free" title="PyTypeObject.tp_free"><code class="xref c c-member docutils literal">tp_free</code></a> slots, we’d need to modify them for cyclic-garbage collection.

1444

Most extensions will use the versions automatically provided.

1445

</div>

1446

1447

<h3>2.1.4. Subclassing other types<a class="headerlink" href="#subclassing-other-types" title="Permalink to this headline">¶</a></h3>

1448

It is possible to create new extension types that are derived from existing

1449

types. It is easiest to inherit from the built in types, since an extension can

1450

easily use the <code class="xref py py-class docutils literal">PyTypeObject</code> it needs. It can be difficult to share

1451

these <code class="xref py py-class docutils literal">PyTypeObject</code> structures between extension modules.

1452

In this example we will create a <code class="xref py py-class docutils literal">Shoddy</code> type that inherits from the

1453

built-in <a class="reference internal" href="../library/stdtypes.html#list" title="list"><code class="xref py py-class docutils literal">list</code></a> type. The new type will be completely compatible with

1454

regular lists, but will have an additional <code class="xref py py-meth docutils literal">increment()</code> method that

1455

increases an internal counter.

1456

<div class="highlight-c"><div class="highlight"><pre>>>> import shoddy

1457

>>> s = shoddy.Shoddy(range(3))

1458

>>> s.extend(s)

1459

>>> print(len(s))

1460

6

1461

>>> print(s.increment())

1462

1

1463

>>> print(s.increment())

1464

2

1465

</pre></div>

1466

</div>

1467

<div class="highlight-c"><div class="highlight"><pre>#include <Python.h>

1468

1469

typedef struct {

1470

PyListObject list;

1471

int state;

1472

} Shoddy;

1473

1474

1475

static PyObject *

1476

Shoddy_increment(Shoddy *self, PyObject *unused)

1477

{

1478

self->state++;

1479

return PyLong_FromLong(self->state);

1480

}

1481

1482

1483

static PyMethodDef Shoddy_methods[] = {

1484

{"increment", (PyCFunction)Shoddy_increment, METH_NOARGS,

1485

PyDoc_STR("increment state counter")},

1486

{NULL, NULL},

1487

};

1488

1489

static int

1490

Shoddy_init(Shoddy *self, PyObject *args, PyObject *kwds)

1491

{

1492

if (PyList_Type.tp_init((PyObject *)self, args, kwds) < 0)

1493

return -1;

1494

self->state = 0;

1495

return 0;

1496

}

1497

1498

1499

static PyTypeObject ShoddyType = {

1500

PyObject_HEAD_INIT(NULL)

1501

"shoddy.Shoddy", /* tp_name */

1502

sizeof(Shoddy), /* tp_basicsize */

1503

0, /* tp_itemsize */

1504

0, /* tp_dealloc */

1505

0, /* tp_print */

1506

0, /* tp_getattr */

1507

0, /* tp_setattr */

1508

0, /* tp_reserved */

1509

0, /* tp_repr */

1510

0, /* tp_as_number */

1511

0, /* tp_as_sequence */

1512

0, /* tp_as_mapping */

1513

0, /* tp_hash */

1514

0, /* tp_call */

1515

0, /* tp_str */

1516

0, /* tp_getattro */

1517

0, /* tp_setattro */

1518

0, /* tp_as_buffer */

1519

Py_TPFLAGS_DEFAULT |

1520

Py_TPFLAGS_BASETYPE, /* tp_flags */

1521

0, /* tp_doc */

1522

0, /* tp_traverse */

1523

0, /* tp_clear */

1524

0, /* tp_richcompare */

1525

0, /* tp_weaklistoffset */

1526

0, /* tp_iter */

1527

0, /* tp_iternext */

1528

Shoddy_methods, /* tp_methods */

1529

0, /* tp_members */

1530

0, /* tp_getset */

1531

0, /* tp_base */

1532

0, /* tp_dict */

1533

0, /* tp_descr_get */

1534

0, /* tp_descr_set */

1535

0, /* tp_dictoffset */

1536

(initproc)Shoddy_init, /* tp_init */

1537

0, /* tp_alloc */

1538

0, /* tp_new */

1539

};

1540

1541

static PyModuleDef shoddymodule = {

1542

PyModuleDef_HEAD_INIT,

1543

"shoddy",

1544

"Shoddy module",

1545

-1,

1546

1547

};

1548

1549

PyMODINIT_FUNC

1550

PyInit_shoddy(void)

1551

{

1552

PyObject *m;

1553

1554

ShoddyType.tp_base = &PyList_Type;

1555

if (PyType_Ready(&ShoddyType) < 0)

1556

return NULL;

1557

1558

m = PyModule_Create(&shoddymodule);

1559

if (m == NULL)

1560

return NULL;

1561

1562

Py_INCREF(&ShoddyType);

1563

PyModule_AddObject(m, "Shoddy", (PyObject *) &ShoddyType);

1564

return m;

1565

}

1566

</pre></div>

1567

</div>

1568

As you can see, the source code closely resembles the <code class="xref py py-class docutils literal">Noddy</code> examples in

1569

previous sections. We will break down the main differences between them.

1570

<div class="highlight-c"><div class="highlight"><pre>typedef struct {

1571

PyListObject list;

1572

int state;

1573

} Shoddy;

1574

</pre></div>

1575

</div>

1576

The primary difference for derived type objects is that the base type’s object

1577

structure must be the first value. The base type will already include the

1578

<a class="reference internal" href="../c-api/structures.html#c.PyObject_HEAD" title="PyObject_HEAD"><code class="xref c c-func docutils literal">PyObject_HEAD()</code></a> at the beginning of its structure.

1579

When a Python object is a <code class="xref py py-class docutils literal">Shoddy</code> instance, its PyObject* pointer can

1580

be safely cast to both PyListObject* and Shoddy*.

1581

<div class="highlight-c"><div class="highlight"><pre>static int

1582

1583

{

1584

1585

return -1;

1586

self->state = 0;

1587

return 0;

1588

}

1589

</pre></div>

1590

</div>

1591

In the <code class="xref py py-attr docutils literal">__init__</code> method for our type, we can see how to call through to

1592

the <code class="xref py py-attr docutils literal">__init__</code> method of the base type.

1593

This pattern is important when writing a type with custom <code class="xref py py-attr docutils literal">new</code> and

1594

<code class="xref py py-attr docutils literal">dealloc</code> methods. The <code class="xref py py-attr docutils literal">new</code> method should not actually create the

1595

memory for the object with <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_alloc" title="PyTypeObject.tp_alloc"><code class="xref c c-member docutils literal">tp_alloc</code></a>, that will be handled by the base

1596

class when calling its <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_new" title="PyTypeObject.tp_new"><code class="xref c c-member docutils literal">tp_new</code></a>.

1597

When filling out the <a class="reference internal" href="../c-api/type.html#c.PyTypeObject" title="PyTypeObject"><code class="xref c c-func docutils literal">PyTypeObject()</code></a> for the <code class="xref py py-class docutils literal">Shoddy</code> type, you see

1598

a slot for <code class="xref c c-func docutils literal">tp_base()</code>. Due to cross platform compiler issues, you can’t

1599

fill that field directly with the <a class="reference internal" href="../c-api/list.html#c.PyList_Type" title="PyList_Type"><code class="xref c c-func docutils literal">PyList_Type()</code></a>; it can be done later in

1600

the module’s <code class="xref c c-func docutils literal">init()</code> function.

1601

<div class="highlight-c"><div class="highlight"><pre>PyMODINIT_FUNC

1602

PyInit_shoddy(void)

1603

{

1604

PyObject *m;

1605

1606

ShoddyType.tp_base = &PyList_Type;

1607

1608

return NULL;

1609

1610

m = PyModule_Create(&shoddymodule);

1611

if (m == NULL)

1612

return NULL;

1613

1614

Py_INCREF(&ShoddyType);

1615

1616

return m;

1617

}

1618

</pre></div>

1619

</div>

1620

Before calling <a class="reference internal" href="../c-api/type.html#c.PyType_Ready" title="PyType_Ready"><code class="xref c c-func docutils literal">PyType_Ready()</code></a>, the type structure must have the

1621

<a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_base" title="PyTypeObject.tp_base"><code class="xref c c-member docutils literal">tp_base</code></a> slot filled in. When we are deriving a new type, it is not

1622

necessary to fill out the <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_alloc" title="PyTypeObject.tp_alloc"><code class="xref c c-member docutils literal">tp_alloc</code></a> slot with <a class="reference internal" href="../c-api/type.html#c.PyType_GenericNew" title="PyType_GenericNew"><code class="xref c c-func docutils literal">PyType_GenericNew()</code></a>

1623

– the allocate function from the base type will be inherited.

1624

After that, calling <a class="reference internal" href="../c-api/type.html#c.PyType_Ready" title="PyType_Ready"><code class="xref c c-func docutils literal">PyType_Ready()</code></a> and adding the type object to the

1625

module is the same as with the basic <code class="xref py py-class docutils literal">Noddy</code> examples.

1626

</div>

1627

</div>

1628

1629

<h2>2.2. Type Methods<a class="headerlink" href="#type-methods" title="Permalink to this headline">¶</a></h2>

1630

This section aims to give a quick fly-by on the various type methods you can

1631

implement and what they do.

1632

Here is the definition of <a class="reference internal" href="../c-api/type.html#c.PyTypeObject" title="PyTypeObject"><code class="xref c c-type docutils literal">PyTypeObject</code></a>, with some fields only used in

1633

debug builds omitted:

1634

<div class="highlight-c"><div class="highlight"><pre>typedef struct _typeobject {

1635

PyObject_VAR_HEAD

1636

const char *tp_name; /* For printing, in format "<module>.<name>" */

1637

Py_ssize_t tp_basicsize, tp_itemsize; /* For allocation */

1638

1639

/* Methods to implement standard operations */

1640

1641

destructor tp_dealloc;

1642

printfunc tp_print;

1643

getattrfunc tp_getattr;

1644

setattrfunc tp_setattr;

1645

PyAsyncMethods *tp_as_async; /* formerly known as tp_compare (Python 2)

1646

or tp_reserved (Python 3) */

1647

reprfunc tp_repr;

1648

1649

/* Method suites for standard classes */

1650

1651

PyNumberMethods *tp_as_number;

1652

PySequenceMethods *tp_as_sequence;

1653

PyMappingMethods *tp_as_mapping;

1654

1655

/* More standard operations (here for binary compatibility) */

1656

1657

hashfunc tp_hash;

1658

ternaryfunc tp_call;

1659

reprfunc tp_str;

1660

getattrofunc tp_getattro;

1661

setattrofunc tp_setattro;

1662

1663

/* Functions to access object as input/output buffer */

1664

PyBufferProcs *tp_as_buffer;

1665

1666

/* Flags to define presence of optional/expanded features */

1667

unsigned long tp_flags;

1668

1669

const char *tp_doc; /* Documentation string */

1670

1671

/* call function for all accessible objects */

1672

traverseproc tp_traverse;

1673

1674

/* delete references to contained objects */

1675

inquiry tp_clear;

1676

1677

/* rich comparisons */

1678

richcmpfunc tp_richcompare;

1679

1680

/* weak reference enabler */

1681

Py_ssize_t tp_weaklistoffset;

1682

1683

/* Iterators */

1684

getiterfunc tp_iter;

1685

iternextfunc tp_iternext;

1686

1687

/* Attribute descriptor and subclassing stuff */

1688

struct PyMethodDef *tp_methods;

1689

struct PyMemberDef *tp_members;

1690

struct PyGetSetDef *tp_getset;

1691

struct _typeobject *tp_base;

1692

PyObject *tp_dict;

1693

descrgetfunc tp_descr_get;

1694

descrsetfunc tp_descr_set;

1695

Py_ssize_t tp_dictoffset;

1696

initproc tp_init;

1697

allocfunc tp_alloc;

1698

newfunc tp_new;

1699

freefunc tp_free; /* Low-level free-memory routine */

1700

inquiry tp_is_gc; /* For PyObject_IS_GC */

1701

PyObject *tp_bases;

1702

PyObject *tp_mro; /* method resolution order */

1703

PyObject *tp_cache;

1704

PyObject *tp_subclasses;

1705

PyObject *tp_weaklist;

1706

destructor tp_del;

1707

1708

/* Type attribute cache version tag. Added in version 2.6 */

1709

unsigned int tp_version_tag;

1710

1711

destructor tp_finalize;

1712

1713

} PyTypeObject;

1714

</pre></div>

1715

</div>

1716

Now that’s a lot of methods. Don’t worry too much though - if you have a type

1717

you want to define, the chances are very good that you will only implement a

1718

handful of these.

1719

As you probably expect by now, we’re going to go over this and give more

1720

information about the various handlers. We won’t go in the order they are

1721

defined in the structure, because there is a lot of historical baggage that

1722

impacts the ordering of the fields; be sure your type initialization keeps the

1723

fields in the right order! It’s often easiest to find an example that includes

1724

all the fields you need (even if they’re initialized to <code class="docutils literal">0</code>) and then change

1725

the values to suit your new type.

1726

<div class="highlight-c"><div class="highlight"><pre>const char *tp_name; /* For printing */

1727

</pre></div>

1728

</div>

1729

The name of the type - as mentioned in the last section, this will appear in

1730

various places, almost entirely for diagnostic purposes. Try to choose something

1731

that will be helpful in such a situation!

1732

<div class="highlight-c"><div class="highlight"><pre>Py_ssize_t tp_basicsize, tp_itemsize; /* For allocation */

1733

</pre></div>

1734

</div>

1735

These fields tell the runtime how much memory to allocate when new objects of

1736

this type are created. Python has some built-in support for variable length

1737

structures (think: strings, lists) which is where the <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_itemsize" title="PyTypeObject.tp_itemsize"><code class="xref c c-member docutils literal">tp_itemsize</code></a> field

1738

comes in. This will be dealt with later.

1739

<div class="highlight-c"><div class="highlight"><pre>const char *tp_doc;

1740

</pre></div>

1741

</div>

1742

Here you can put a string (or its address) that you want returned when the

1743

Python script references <code class="docutils literal">obj.__doc__</code> to retrieve the doc string.

1744

Now we come to the basic type methods—the ones most extension types will

1745

implement.

1746

1747

<h3>2.2.1. Finalization and De-allocation<a class="headerlink" href="#finalization-and-de-allocation" title="Permalink to this headline">¶</a></h3>

1748

<div class="highlight-c" id="index-0"><div class="highlight"><pre>destructor tp_dealloc;

1749

</pre></div>

1750

</div>

1751

This function is called when the reference count of the instance of your type is

1752

reduced to zero and the Python interpreter wants to reclaim it. If your type

1753

has memory to free or other clean-up to perform, you can put it here. The

1754

object itself needs to be freed here as well. Here is an example of this

1755

function:

1756

<div class="highlight-c"><div class="highlight"><pre>static void

1757

newdatatype_dealloc(newdatatypeobject * obj)

1758

{

1759

free(obj->obj_UnderlyingDatatypePtr);

1760

Py_TYPE(obj)->tp_free(obj);

1761

}

1762

</pre></div>

1763

</div>

1764

One important requirement of the deallocator function is that it leaves any

1765

pending exceptions alone. This is important since deallocators are frequently

1766

called as the interpreter unwinds the Python stack; when the stack is unwound

1767

due to an exception (rather than normal returns), nothing is done to protect the

1768

deallocators from seeing that an exception has already been set. Any actions

1769

which a deallocator performs which may cause additional Python code to be

1770

executed may detect that an exception has been set. This can lead to misleading

1771

errors from the interpreter. The proper way to protect against this is to save

1772

a pending exception before performing the unsafe action, and restoring it when

1773

done. This can be done using the <a class="reference internal" href="../c-api/exceptions.html#c.PyErr_Fetch" title="PyErr_Fetch"><code class="xref c c-func docutils literal">PyErr_Fetch()</code></a> and

1774

<a class="reference internal" href="../c-api/exceptions.html#c.PyErr_Restore" title="PyErr_Restore"><code class="xref c c-func docutils literal">PyErr_Restore()</code></a> functions:

1775

<div class="highlight-c"><div class="highlight"><pre>static void

1776

my_dealloc(PyObject *obj)

1777

{

1778

MyObject *self = (MyObject *) obj;

1779

PyObject *cbresult;

1780

1781

if (self->my_callback != NULL) {

1782

PyObject *err_type, *err_value, *err_traceback;

1783

1784

/* This saves the current exception state */

1785

PyErr_Fetch(&err_type, &err_value, &err_traceback);

1786

1787

cbresult = PyObject_CallObject(self->my_callback, NULL);

1788

if (cbresult == NULL)

1789

PyErr_WriteUnraisable(self->my_callback);

1790

else

1791

Py_DECREF(cbresult);

1792

1793

/* This restores the saved exception state */

1794

PyErr_Restore(err_type, err_value, err_traceback);

1795

1796

Py_DECREF(self->my_callback);

1797

}

1798

Py_TYPE(obj)->tp_free((PyObject*)self);

1799

}

1800

</pre></div>

1801

</div>

1802

1803

Note

1804

There are limitations to what you can safely do in a deallocator function.

1805

First, if your type supports garbage collection (using <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_traverse" title="PyTypeObject.tp_traverse"><code class="xref c c-member docutils literal">tp_traverse</code></a>

1806

and/or <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_clear" title="PyTypeObject.tp_clear"><code class="xref c c-member docutils literal">tp_clear</code></a>), some of the object’s members can have been

1807

cleared or finalized by the time <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_dealloc" title="PyTypeObject.tp_dealloc"><code class="xref c c-member docutils literal">tp_dealloc</code></a> is called. Second, in

1808

<a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_dealloc" title="PyTypeObject.tp_dealloc"><code class="xref c c-member docutils literal">tp_dealloc</code></a>, your object is in an unstable state: its reference

1809

count is equal to zero. Any call to a non-trivial object or API (as in the

1810

example above) might end up calling <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_dealloc" title="PyTypeObject.tp_dealloc"><code class="xref c c-member docutils literal">tp_dealloc</code></a> again, causing a

1811

double free and a crash.

1812

Starting with Python 3.4, it is recommended not to put any complex

1813

finalization code in <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_dealloc" title="PyTypeObject.tp_dealloc"><code class="xref c c-member docutils literal">tp_dealloc</code></a>, and instead use the new

1814

<a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_finalize" title="PyTypeObject.tp_finalize"><code class="xref c c-member docutils literal">tp_finalize</code></a> type method.

1815

1816

See also

1817

<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0442">PEP 442</a> explains the new finalization scheme.

1818

</div>

1819

</div>

1820

</div>

1821

1822

<h3>2.2.2. Object Presentation<a class="headerlink" href="#object-presentation" title="Permalink to this headline">¶</a></h3>

1823

In Python, there are two ways to generate a textual representation of an object:

1824

the <a class="reference internal" href="../library/functions.html#repr" title="repr"><code class="xref py py-func docutils literal">repr()</code></a> function, and the <a class="reference internal" href="../library/stdtypes.html#str" title="str"><code class="xref py py-func docutils literal">str()</code></a> function. (The <a class="reference internal" href="../library/functions.html#print" title="print"><code class="xref py py-func docutils literal">print()</code></a>

1825

function just calls <a class="reference internal" href="../library/stdtypes.html#str" title="str"><code class="xref py py-func docutils literal">str()</code></a>.) These handlers are both optional.

1826

<div class="highlight-c"><div class="highlight"><pre>reprfunc tp_repr;

1827

reprfunc tp_str;

1828

</pre></div>

1829

</div>

1830

The <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_repr" title="PyTypeObject.tp_repr"><code class="xref c c-member docutils literal">tp_repr</code></a> handler should return a string object containing a

1831

representation of the instance for which it is called. Here is a simple

1832

example:

1833

<div class="highlight-c"><div class="highlight"><pre>static PyObject *

1834

newdatatype_repr(newdatatypeobject * obj)

1835

{

1836

return PyUnicode_FromFormat("Repr-ified_newdatatype{{size:\%d}}",

1837

obj->obj_UnderlyingDatatypePtr->size);

1838

}

1839

</pre></div>

1840

</div>

1841

If no <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_repr" title="PyTypeObject.tp_repr"><code class="xref c c-member docutils literal">tp_repr</code></a> handler is specified, the interpreter will supply a

1842

representation that uses the type’s <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_name" title="PyTypeObject.tp_name"><code class="xref c c-member docutils literal">tp_name</code></a> and a uniquely-identifying

1843

value for the object.

1844

The <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_str" title="PyTypeObject.tp_str"><code class="xref c c-member docutils literal">tp_str</code></a> handler is to <a class="reference internal" href="../library/stdtypes.html#str" title="str"><code class="xref py py-func docutils literal">str()</code></a> what the <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_repr" title="PyTypeObject.tp_repr"><code class="xref c c-member docutils literal">tp_repr</code></a> handler

1845

described above is to <a class="reference internal" href="../library/functions.html#repr" title="repr"><code class="xref py py-func docutils literal">repr()</code></a>; that is, it is called when Python code calls

1846

<a class="reference internal" href="../library/stdtypes.html#str" title="str"><code class="xref py py-func docutils literal">str()</code></a> on an instance of your object. Its implementation is very similar

1847

to the <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_repr" title="PyTypeObject.tp_repr"><code class="xref c c-member docutils literal">tp_repr</code></a> function, but the resulting string is intended for human

1848

consumption. If <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_str" title="PyTypeObject.tp_str"><code class="xref c c-member docutils literal">tp_str</code></a> is not specified, the <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_repr" title="PyTypeObject.tp_repr"><code class="xref c c-member docutils literal">tp_repr</code></a> handler is

1849

used instead.

1850

Here is a simple example:

1851

<div class="highlight-c"><div class="highlight"><pre>static PyObject *

1852

newdatatype_str(newdatatypeobject * obj)

1853

{

1854

return PyUnicode_FromFormat("Stringified_newdatatype{{size:\%d}}",

1855

obj->obj_UnderlyingDatatypePtr->size);

1856

}

1857

</pre></div>

1858

</div>

1859

</div>

1860

1861

<h3>2.2.3. Attribute Management<a class="headerlink" href="#attribute-management" title="Permalink to this headline">¶</a></h3>

1862

For every object which can support attributes, the corresponding type must

1863

provide the functions that control how the attributes are resolved. There needs

1864

to be a function which can retrieve attributes (if any are defined), and another

1865

to set attributes (if setting attributes is allowed). Removing an attribute is

1866

a special case, for which the new value passed to the handler is NULL.

1867

Python supports two pairs of attribute handlers; a type that supports attributes

1868

only needs to implement the functions for one pair. The difference is that one

1869

pair takes the name of the attribute as a <code class="xref c c-type docutils literal">char*</code>, while the other

1870

accepts a <a class="reference internal" href="../c-api/structures.html#c.PyObject" title="PyObject"><code class="xref c c-type docutils literal">PyObject*</code></a>. Each type can use whichever pair makes more

1871

sense for the implementation’s convenience.

1872

<div class="highlight-c"><div class="highlight"><pre>getattrfunc tp_getattr; /* char * version */

1873

setattrfunc tp_setattr;

1874

/* ... */

1875

getattrofunc tp_getattro; /* PyObject * version */

1876

setattrofunc tp_setattro;

1877

</pre></div>

1878

</div>

1879

If accessing attributes of an object is always a simple operation (this will be

1880

explained shortly), there are generic implementations which can be used to

1881

provide the <a class="reference internal" href="../c-api/structures.html#c.PyObject" title="PyObject"><code class="xref c c-type docutils literal">PyObject*</code></a> version of the attribute management functions.

1882

The actual need for type-specific attribute handlers almost completely

1883

disappeared starting with Python 2.2, though there are many examples which have

1884

not been updated to use some of the new generic mechanism that is available.

1885

1886

<h4>2.2.3.1. Generic Attribute Management<a class="headerlink" href="#generic-attribute-management" title="Permalink to this headline">¶</a></h4>

1887

Most extension types only use simple attributes. So, what makes the

1888

attributes simple? There are only a couple of conditions that must be met:

1889

1890

<li>The name of the attributes must be known when <a class="reference internal" href="../c-api/type.html#c.PyType_Ready" title="PyType_Ready"><code class="xref c c-func docutils literal">PyType_Ready()</code></a> is

1891

called.</li>

1892

<li>No special processing is needed to record that an attribute was looked up or

1893

set, nor do actions need to be taken based on the value.</li>

1894

</ol>

1895

Note that this list does not place any restrictions on the values of the

1896

attributes, when the values are computed, or how relevant data is stored.

1897

When <a class="reference internal" href="../c-api/type.html#c.PyType_Ready" title="PyType_Ready"><code class="xref c c-func docutils literal">PyType_Ready()</code></a> is called, it uses three tables referenced by the

1898

type object to create <a class="reference internal" href="../glossary.html#term-descriptor">descriptor</a>s which are placed in the dictionary of the

1899

type object. Each descriptor controls access to one attribute of the instance

1900

object. Each of the tables is optional; if all three are NULL, instances of

1901

the type will only have attributes that are inherited from their base type, and

1902

should leave the <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_getattro" title="PyTypeObject.tp_getattro"><code class="xref c c-member docutils literal">tp_getattro</code></a> and <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_setattro" title="PyTypeObject.tp_setattro"><code class="xref c c-member docutils literal">tp_setattro</code></a> fields NULL as

1903

well, allowing the base type to handle attributes.

1904

The tables are declared as three fields of the type object:

1905

<div class="highlight-c"><div class="highlight"><pre>struct PyMethodDef *tp_methods;

1906

struct PyMemberDef *tp_members;

1907

struct PyGetSetDef *tp_getset;

1908

</pre></div>

1909

</div>

1910

If <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_methods" title="PyTypeObject.tp_methods"><code class="xref c c-member docutils literal">tp_methods</code></a> is not NULL, it must refer to an array of

1911

<a class="reference internal" href="../c-api/structures.html#c.PyMethodDef" title="PyMethodDef"><code class="xref c c-type docutils literal">PyMethodDef</code></a> structures. Each entry in the table is an instance of this

1912

structure:

1913

<div class="highlight-c"><div class="highlight"><pre>typedef struct PyMethodDef {

1914

char *ml_name; /* method name */

1915

PyCFunction ml_meth; /* implementation function */

1916

int ml_flags; /* flags */

1917

char *ml_doc; /* docstring */

1918

} PyMethodDef;

1919

</pre></div>

1920

</div>

1921

One entry should be defined for each method provided by the type; no entries are

1922

needed for methods inherited from a base type. One additional entry is needed

1923

at the end; it is a sentinel that marks the end of the array. The

1924

<code class="xref py py-attr docutils literal">ml_name</code> field of the sentinel must be NULL.

1925

The second table is used to define attributes which map directly to data stored

1926

in the instance. A variety of primitive C types are supported, and access may

1927

be read-only or read-write. The structures in the table are defined as:

1928

<div class="highlight-c"><div class="highlight"><pre>typedef struct PyMemberDef {

1929

char *name;

1930

int type;

1931

int offset;

1932

int flags;

1933

char *doc;

1934

} PyMemberDef;

1935

</pre></div>

1936

</div>

1937

For each entry in the table, a <a class="reference internal" href="../glossary.html#term-descriptor">descriptor</a> will be constructed and added to the

1938

type which will be able to extract a value from the instance structure. The

1939

<a class="reference internal" href="../library/functions.html#type" title="type"><code class="xref py py-attr docutils literal">type</code></a> field should contain one of the type codes defined in the

1940

<code class="file docutils literal">structmember.h</code> header; the value will be used to determine how to

1941

convert Python values to and from C values. The <code class="xref py py-attr docutils literal">flags</code> field is used to

1942

store flags which control how the attribute can be accessed.

1943

The following flag constants are defined in <code class="file docutils literal">structmember.h</code>; they may be

1944

combined using bitwise-OR.

1945

1946

1947

1948

1949

</colgroup>

1950

1951

<tr class="row-odd"><th class="head">Constant</th>

1952

<th class="head">Meaning</th>

1953

</tr>

1954

</thead>

1955

1956

<tr class="row-even"><td><code class="xref py py-const docutils literal">READONLY</code></td>

1957

<td>Never writable.</td>

1958

</tr>

1959

<tr class="row-odd"><td><code class="xref py py-const docutils literal">READ_RESTRICTED</code></td>

1960

<td>Not readable in restricted mode.</td>

1961

</tr>

1962

<tr class="row-even"><td><code class="xref py py-const docutils literal">WRITE_RESTRICTED</code></td>

1963

<td>Not writable in restricted mode.</td>

1964

</tr>

1965

<tr class="row-odd"><td><code class="xref py py-const docutils literal">RESTRICTED</code></td>

1966

<td>Not readable or writable in restricted mode.</td>

1967

</tr>

1968

</tbody>

1969

</table>

1970

An interesting advantage of using the <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_members" title="PyTypeObject.tp_members"><code class="xref c c-member docutils literal">tp_members</code></a> table to build

1971

descriptors that are used at runtime is that any attribute defined this way can

1972

have an associated doc string simply by providing the text in the table. An

1973

application can use the introspection API to retrieve the descriptor from the

1974

class object, and get the doc string using its <code class="xref py py-attr docutils literal">__doc__</code> attribute.

1975

As with the <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_methods" title="PyTypeObject.tp_methods"><code class="xref c c-member docutils literal">tp_methods</code></a> table, a sentinel entry with a <code class="xref py py-attr docutils literal">name</code> value

1976

of NULL is required.

1977

</div>

1978

1979

<h4>2.2.3.2. Type-specific Attribute Management<a class="headerlink" href="#type-specific-attribute-management" title="Permalink to this headline">¶</a></h4>

1980

For simplicity, only the <code class="xref c c-type docutils literal">char*</code> version will be demonstrated here; the

1981

type of the name parameter is the only difference between the <code class="xref c c-type docutils literal">char*</code>

1982

and <a class="reference internal" href="../c-api/structures.html#c.PyObject" title="PyObject"><code class="xref c c-type docutils literal">PyObject*</code></a> flavors of the interface. This example effectively does

1983

the same thing as the generic example above, but does not use the generic

1984

support added in Python 2.2. It explains how the handler functions are

1985

called, so that if you do need to extend their functionality, you’ll understand

1986

what needs to be done.

1987

The <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_getattr" title="PyTypeObject.tp_getattr"><code class="xref c c-member docutils literal">tp_getattr</code></a> handler is called when the object requires an attribute

1988

look-up. It is called in the same situations where the <a class="reference internal" href="../reference/datamodel.html#object.__getattr__" title="object.__getattr__"><code class="xref py py-meth docutils literal">__getattr__()</code></a>

1989

method of a class would be called.

1990

Here is an example:

1991

<div class="highlight-c"><div class="highlight"><pre>static PyObject *

1992

newdatatype_getattr(newdatatypeobject *obj, char *name)

1993

{

1994

if (strcmp(name, "data") == 0)

1995

{

1996

return PyLong_FromLong(obj->data);

1997

}

1998

1999

PyErr_Format(PyExc_AttributeError,

2000

"'%.50s' object has no attribute '%.400s'",

2001

tp->tp_name, name);

2002

return NULL;

2003

}

2004

</pre></div>

2005

</div>

2006

The <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_setattr" title="PyTypeObject.tp_setattr"><code class="xref c c-member docutils literal">tp_setattr</code></a> handler is called when the <a class="reference internal" href="../reference/datamodel.html#object.__setattr__" title="object.__setattr__"><code class="xref py py-meth docutils literal">__setattr__()</code></a> or

2007

<a class="reference internal" href="../reference/datamodel.html#object.__delattr__" title="object.__delattr__"><code class="xref py py-meth docutils literal">__delattr__()</code></a> method of a class instance would be called. When an

2008

attribute should be deleted, the third parameter will be NULL. Here is an

2009

example that simply raises an exception; if this were really all you wanted, the

2010

<a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_setattr" title="PyTypeObject.tp_setattr"><code class="xref c c-member docutils literal">tp_setattr</code></a> handler should be set to NULL.

2011

<div class="highlight-c"><div class="highlight"><pre>static int

2012

newdatatype_setattr(newdatatypeobject *obj, char *name, PyObject *v)

2013

{

2014

(void)PyErr_Format(PyExc_RuntimeError, "Read-only attribute: \%s", name);

2015

return -1;

2016

}

2017

</pre></div>

2018

</div>

2019

</div>

2020

</div>

2021

2022

<h3>2.2.4. Object Comparison<a class="headerlink" href="#object-comparison" title="Permalink to this headline">¶</a></h3>

2023

<div class="highlight-c"><div class="highlight"><pre>richcmpfunc tp_richcompare;

2024

</pre></div>

2025

</div>

2026

The <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_richcompare" title="PyTypeObject.tp_richcompare"><code class="xref c c-member docutils literal">tp_richcompare</code></a> handler is called when comparisons are needed. It is

2027

analogous to the <a class="reference internal" href="../reference/datamodel.html#richcmpfuncs">rich comparison methods</a>, like

2028

<a class="reference internal" href="../reference/datamodel.html#object.__lt__" title="object.__lt__"><code class="xref py py-meth docutils literal">__lt__()</code></a>, and also called by <a class="reference internal" href="../c-api/object.html#c.PyObject_RichCompare" title="PyObject_RichCompare"><code class="xref c c-func docutils literal">PyObject_RichCompare()</code></a> and

2029

<a class="reference internal" href="../c-api/object.html#c.PyObject_RichCompareBool" title="PyObject_RichCompareBool"><code class="xref c c-func docutils literal">PyObject_RichCompareBool()</code></a>.

2030

This function is called with two Python objects and the operator as arguments,

2031

where the operator is one of <code class="docutils literal">Py_EQ</code>, <code class="docutils literal">Py_NE</code>, <code class="docutils literal">Py_LE</code>, <code class="docutils literal">Py_GT</code>,

2032

<code class="docutils literal">Py_LT</code> or <code class="docutils literal">Py_GT</code>. It should compare the two objects with respect to the

2033

specified operator and return <code class="docutils literal">Py_True</code> or <code class="docutils literal">Py_False</code> if the comparison is

2034

successful, <code class="docutils literal">Py_NotImplemented</code> to indicate that comparison is not

2035

implemented and the other object’s comparison method should be tried, or NULL

2036

if an exception was set.

2037

Here is a sample implementation, for a datatype that is considered equal if the

2038

size of an internal pointer is equal:

2039

<div class="highlight-c"><div class="highlight"><pre>static PyObject *

2040

newdatatype_richcmp(PyObject *obj1, PyObject *obj2, int op)

2041

{

2042

PyObject *result;

2043

int c, size1, size2;

2044

2045

/* code to make sure that both arguments are of type

2046

newdatatype omitted */

2047

2048

size1 = obj1->obj_UnderlyingDatatypePtr->size;

2049

size2 = obj2->obj_UnderlyingDatatypePtr->size;

2050

2051

switch (op) {

2052

case Py_LT: c = size1 < size2; break;

2053

case Py_LE: c = size1 <= size2; break;

2054

case Py_EQ: c = size1 == size2; break;

2055

case Py_NE: c = size1 != size2; break;

2056

case Py_GT: c = size1 > size2; break;

2057

case Py_GE: c = size1 >= size2; break;

2058

}

2059

result = c ? Py_True : Py_False;

2060

Py_INCREF(result);

2061

return result;

2062

}

2063

</pre></div>

2064

</div>

2065

</div>

2066

2067

<h3>2.2.5. Abstract Protocol Support<a class="headerlink" href="#abstract-protocol-support" title="Permalink to this headline">¶</a></h3>

2068

Python supports a variety of abstract ‘protocols;’ the specific interfaces

2069

provided to use these interfaces are documented in <a class="reference internal" href="../c-api/abstract.html#abstract">Abstract Objects Layer</a>.

2070

A number of these abstract interfaces were defined early in the development of

2071

the Python implementation. In particular, the number, mapping, and sequence

2072

protocols have been part of Python since the beginning. Other protocols have

2073

been added over time. For protocols which depend on several handler routines

2074

from the type implementation, the older protocols have been defined as optional

2075

blocks of handlers referenced by the type object. For newer protocols there are

2076

additional slots in the main type object, with a flag bit being set to indicate

2077

that the slots are present and should be checked by the interpreter. (The flag

2078

bit does not indicate that the slot values are non-NULL. The flag may be set

2079

to indicate the presence of a slot, but a slot may still be unfilled.)

2080

<div class="highlight-c"><div class="highlight"><pre>PyNumberMethods *tp_as_number;

2081

PySequenceMethods *tp_as_sequence;

2082

PyMappingMethods *tp_as_mapping;

2083

</pre></div>

2084

</div>

2085

If you wish your object to be able to act like a number, a sequence, or a

2086

mapping object, then you place the address of a structure that implements the C

2087

type <a class="reference internal" href="../c-api/typeobj.html#c.PyNumberMethods" title="PyNumberMethods"><code class="xref c c-type docutils literal">PyNumberMethods</code></a>, <a class="reference internal" href="../c-api/typeobj.html#c.PySequenceMethods" title="PySequenceMethods"><code class="xref c c-type docutils literal">PySequenceMethods</code></a>, or

2088

<a class="reference internal" href="../c-api/typeobj.html#c.PyMappingMethods" title="PyMappingMethods"><code class="xref c c-type docutils literal">PyMappingMethods</code></a>, respectively. It is up to you to fill in this

2089

structure with appropriate values. You can find examples of the use of each of

2090

these in the <code class="file docutils literal">Objects</code> directory of the Python source distribution.

2091

<div class="highlight-c"><div class="highlight"><pre>hashfunc tp_hash;

2092

</pre></div>

2093

</div>

2094

This function, if you choose to provide it, should return a hash number for an

2095

instance of your data type. Here is a moderately pointless example:

2096

<div class="highlight-c"><div class="highlight"><pre>static long

2097

newdatatype_hash(newdatatypeobject *obj)

2098

{

2099

long result;

2100

result = obj->obj_UnderlyingDatatypePtr->size;

2101

result = result * 3;

2102

return result;

2103

}

2104

</pre></div>

2105

</div>

2106

<div class="highlight-c"><div class="highlight"><pre>ternaryfunc tp_call;

2107

</pre></div>

2108

</div>

2109

This function is called when an instance of your data type is “called”, for

2110

example, if <code class="docutils literal">obj1</code> is an instance of your data type and the Python script

2111

contains <code class="docutils literal">obj1('hello')</code>, the <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_call" title="PyTypeObject.tp_call"><code class="xref c c-member docutils literal">tp_call</code></a> handler is invoked.

2112

This function takes three arguments:

2113

2114

<li>arg1 is the instance of the data type which is the subject of the call. If

2115

the call is <code class="docutils literal">obj1('hello')</code>, then arg1 is <code class="docutils literal">obj1</code>.</li>

2116

<li>arg2 is a tuple containing the arguments to the call. You can use

2117

<a class="reference internal" href="../c-api/arg.html#c.PyArg_ParseTuple" title="PyArg_ParseTuple"><code class="xref c c-func docutils literal">PyArg_ParseTuple()</code></a> to extract the arguments.</li>

2118

<li>arg3 is a dictionary of keyword arguments that were passed. If this is

2119

non-NULL and you support keyword arguments, use

2120

<a class="reference internal" href="../c-api/arg.html#c.PyArg_ParseTupleAndKeywords" title="PyArg_ParseTupleAndKeywords"><code class="xref c c-func docutils literal">PyArg_ParseTupleAndKeywords()</code></a> to extract the arguments. If you do not

2121

want to support keyword arguments and this is non-NULL, raise a

2122

<a class="reference internal" href="../library/exceptions.html#TypeError" title="TypeError"><code class="xref py py-exc docutils literal">TypeError</code></a> with a message saying that keyword arguments are not supported.</li>

2123

</ol>

2124

Here is a desultory example of the implementation of the call function.

2125

<div class="highlight-c"><div class="highlight"><pre>/* Implement the call function.

2126

* obj1 is the instance receiving the call.

2127

* obj2 is a tuple containing the arguments to the call, in this

2128

* case 3 strings.

2129

*/

2130

static PyObject *

2131

newdatatype_call(newdatatypeobject *obj, PyObject *args, PyObject *other)

2132

{

2133

PyObject *result;

2134

char *arg1;

2135

char *arg2;

2136

char *arg3;

2137

2138

if (!PyArg_ParseTuple(args, "sss:call", &arg1, &arg2, &arg3)) {

2139

return NULL;

2140

}

2141

result = PyUnicode_FromFormat(

2142

"Returning -- value: [\%d] arg1: [\%s] arg2: [\%s] arg3: [\%s]\n",

2143

obj->obj_UnderlyingDatatypePtr->size,

2144

arg1, arg2, arg3);

2145

return result;

2146

}

2147

</pre></div>

2148

</div>

2149

<div class="highlight-c"><div class="highlight"><pre>/* Iterators */

2150

getiterfunc tp_iter;

2151

iternextfunc tp_iternext;

2152

</pre></div>

2153

</div>

2154

These functions provide support for the iterator protocol. Any object which

2155

wishes to support iteration over its contents (which may be generated during

2156

iteration) must implement the <code class="docutils literal">tp_iter</code> handler. Objects which are returned

2157

by a <code class="docutils literal">tp_iter</code> handler must implement both the <code class="docutils literal">tp_iter</code> and <code class="docutils literal">tp_iternext</code>

2158

handlers. Both handlers take exactly one parameter, the instance for which they

2159

are being called, and return a new reference. In the case of an error, they

2160

should set an exception and return NULL.

2161

For an object which represents an iterable collection, the <code class="docutils literal">tp_iter</code> handler

2162

must return an iterator object. The iterator object is responsible for

2163

maintaining the state of the iteration. For collections which can support

2164

multiple iterators which do not interfere with each other (as lists and tuples

2165

do), a new iterator should be created and returned. Objects which can only be

2166

iterated over once (usually due to side effects of iteration) should implement

2167

this handler by returning a new reference to themselves, and should also

2168

implement the <code class="docutils literal">tp_iternext</code> handler. File objects are an example of such an

2169

iterator.

2170

Iterator objects should implement both handlers. The <code class="docutils literal">tp_iter</code> handler should

2171

return a new reference to the iterator (this is the same as the <code class="docutils literal">tp_iter</code>

2172

handler for objects which can only be iterated over destructively). The

2173

<code class="docutils literal">tp_iternext</code> handler should return a new reference to the next object in the

2174

iteration if there is one. If the iteration has reached the end, it may return

2175

NULL without setting an exception or it may set <a class="reference internal" href="../library/exceptions.html#StopIteration" title="StopIteration"><code class="xref py py-exc docutils literal">StopIteration</code></a>; avoiding

2176

the exception can yield slightly better performance. If an actual error occurs,

2177

it should set an exception and return NULL.

2178

</div>

2179

2180

<h3>2.2.6. Weak Reference Support<a class="headerlink" href="#weak-reference-support" title="Permalink to this headline">¶</a></h3>

2181

One of the goals of Python’s weak-reference implementation is to allow any type

2182

to participate in the weak reference mechanism without incurring the overhead on

2183

those objects which do not benefit by weak referencing (such as numbers).

2184

For an object to be weakly referencable, the extension must include a

2185

<a class="reference internal" href="../c-api/structures.html#c.PyObject" title="PyObject"><code class="xref c c-type docutils literal">PyObject*</code></a> field in the instance structure for the use of the weak

2186

reference mechanism; it must be initialized to NULL by the object’s

2187

constructor. It must also set the <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_weaklistoffset" title="PyTypeObject.tp_weaklistoffset"><code class="xref c c-member docutils literal">tp_weaklistoffset</code></a> field of the

2188

corresponding type object to the offset of the field. For example, the instance

2189

type is defined with the following structure:

2190

<div class="highlight-c"><div class="highlight"><pre>typedef struct {

2191

PyObject_HEAD

2192

PyClassObject *in_class; /* The class object */

2193

PyObject *in_dict; /* A dictionary */

2194

PyObject *in_weakreflist; /* List of weak references */

2195

} PyInstanceObject;

2196

</pre></div>

2197

</div>

2198

The statically-declared type object for instances is defined this way:

2199

<div class="highlight-c"><div class="highlight"><pre>PyTypeObject PyInstance_Type = {

2200

PyVarObject_HEAD_INIT(&PyType_Type, 0)

2201

0,

2202

"module.instance",

2203

2204

/* Lots of stuff omitted for brevity... */

2205

2206

Py_TPFLAGS_DEFAULT, /* tp_flags */

2207

0, /* tp_doc */

2208

0, /* tp_traverse */

2209

0, /* tp_clear */

2210

0, /* tp_richcompare */

2211

offsetof(PyInstanceObject, in_weakreflist), /* tp_weaklistoffset */

2212

};

2213

</pre></div>

2214

</div>

2215

The type constructor is responsible for initializing the weak reference list to

2216

NULL:

2217

<div class="highlight-c"><div class="highlight"><pre>static PyObject *

2218

instance_new() {

2219

/* Other initialization stuff omitted for brevity */

2220

2221

self->in_weakreflist = NULL;

2222

2223

return (PyObject *) self;

2224

}

2225

</pre></div>

2226

</div>

2227

The only further addition is that the destructor needs to call the weak

2228

reference manager to clear any weak references. This is only required if the

2229

weak reference list is non-NULL:

2230

<div class="highlight-c"><div class="highlight"><pre>static void

2231

instance_dealloc(PyInstanceObject *inst)

2232

{

2233

/* Allocate temporaries if needed, but do not begin

2234

destruction just yet.

2235

*/

2236

2237

if (inst->in_weakreflist != NULL)

2238

PyObject_ClearWeakRefs((PyObject *) inst);

2239

2240

/* Proceed with object destruction normally. */

2241

}

2242

</pre></div>

2243

</div>

2244

</div>

2245

2246

<h3>2.2.7. More Suggestions<a class="headerlink" href="#more-suggestions" title="Permalink to this headline">¶</a></h3>

2247

Remember that you can omit most of these functions, in which case you provide

2248

<code class="docutils literal">0</code> as a value. There are type definitions for each of the functions you must

2249

provide. They are in <code class="file docutils literal">object.h</code> in the Python include directory that

2250

comes with the source distribution of Python.

2251

In order to learn how to implement any specific method for your new data type,

2252

do the following: Download and unpack the Python source distribution. Go to

2253

the <code class="file docutils literal">Objects</code> directory, then search the C source files for <code class="docutils literal">tp_</code> plus

2254

the function you want (for example, <code class="docutils literal">tp_richcompare</code>). You will find examples

2255

of the function you want to implement.

2256

When you need to verify that an object is an instance of the type you are

2257

implementing, use the <a class="reference internal" href="../c-api/object.html#c.PyObject_TypeCheck" title="PyObject_TypeCheck"><code class="xref c c-func docutils literal">PyObject_TypeCheck()</code></a> function. A sample of its use

2258

might be something like the following:

2259

<div class="highlight-c"><div class="highlight"><pre>if (! PyObject_TypeCheck(some_object, &MyType)) {

2260

PyErr_SetString(PyExc_TypeError, "arg #1 not a mything");

2261

return NULL;

2262

}

2263

</pre></div>

2264

</div>

2265

Footnotes

2266

2267

2268

2269

<tr><td class="label"><a class="fn-backref" href="#id2">[1]</a></td><td>This is true when we know that the object is a basic type, like a string or a

2270

float.</td></tr>

2271

</tbody>

2272

</table>

2273

2274

2275

2276

<tr><td class="label"><a class="fn-backref" href="#id3">[2]</a></td><td>We relied on this in the <a class="reference internal" href="../c-api/typeobj.html#c.PyTypeObject.tp_dealloc" title="PyTypeObject.tp_dealloc"><code class="xref c c-member docutils literal">tp_dealloc</code></a> handler in this example, because our

2277

type doesn’t support garbage collection. Even if a type supports garbage

2278

collection, there are calls that can be made to “untrack” the object from

2279

garbage collection, however, these calls are advanced and not covered here.</td></tr>

2280

</tbody>

2281

</table>

2282

2283

2284

2285

<tr><td class="label"><a class="fn-backref" href="#id4">[3]</a></td><td>We now know that the first and last members are strings, so perhaps we could be

2286

less careful about decrementing their reference counts, however, we accept

2287

instances of string subclasses. Even though deallocating normal strings won’t

2288

call back into our objects, we can’t guarantee that deallocating an instance of

2289

a string subclass won’t call back into our objects.</td></tr>

2290

</tbody>

2291

</table>

2292

2293

2294

2295

<tr><td class="label"><a class="fn-backref" href="#id5">[4]</a></td><td>Even in the third version, we aren’t guaranteed to avoid cycles. Instances of

2296

string subclasses are allowed and string subclasses could allow cycles even if

2297

normal strings don’t.</td></tr>

2298

</tbody>

2299

</table>

2300

</div>

2301

</div>

2302

</div>

2303

2304

2305

</div>

2306

</div>

2307

</div>

2308

2309

2310

<h3><a href="../contents.html">Table Of Contents</a></h3>

2311

<ul>

2312

<li><a class="reference internal" href="#">2. Defining New Types</a><ul>

2313

<li><a class="reference internal" href="#the-basics">2.1. The Basics</a><ul>

2314

<li><a class="reference internal" href="#adding-data-and-methods-to-the-basic-example">2.1.1. Adding data and methods to the Basic example</a></li>

2315

<li><a class="reference internal" href="#providing-finer-control-over-data-attributes">2.1.2. Providing finer control over data attributes</a></li>

2316

<li><a class="reference internal" href="#supporting-cyclic-garbage-collection">2.1.3. Supporting cyclic garbage collection</a></li>

2317

<li><a class="reference internal" href="#subclassing-other-types">2.1.4. Subclassing other types</a></li>

2318

</ul>

2319

</li>

2320

<li><a class="reference internal" href="#type-methods">2.2. Type Methods</a><ul>

2321

<li><a class="reference internal" href="#finalization-and-de-allocation">2.2.1. Finalization and De-allocation</a></li>

2322

<li><a class="reference internal" href="#object-presentation">2.2.2. Object Presentation</a></li>

2323

<li><a class="reference internal" href="#attribute-management">2.2.3. Attribute Management</a><ul>

2324

<li><a class="reference internal" href="#generic-attribute-management">2.2.3.1. Generic Attribute Management</a></li>

2325

<li><a class="reference internal" href="#type-specific-attribute-management">2.2.3.2. Type-specific Attribute Management</a></li>

2326

</ul>

2327

</li>

2328

<li><a class="reference internal" href="#object-comparison">2.2.4. Object Comparison</a></li>

2329

<li><a class="reference internal" href="#abstract-protocol-support">2.2.5. Abstract Protocol Support</a></li>

2330

<li><a class="reference internal" href="#weak-reference-support">2.2.6. Weak Reference Support</a></li>

2331

<li><a class="reference internal" href="#more-suggestions">2.2.7. More Suggestions</a></li>

2332

</ul>

2333

</li>

2334

</ul>

2335

</li>

2336

</ul>

2337

2338

<h4>Previous topic</h4>

2339

<a href="extending.html"

2340

title="previous chapter">1. Extending Python with C or C++</a>

2341

<h4>Next topic</h4>

2342

<a href="building.html"

2343

title="next chapter">3. Building C and C++ Extensions</a>

2344

2345

2346

<li><a href="../bugs.html">Report a Bug</a></li>

2347

<li><a href="../_sources/extending/newtypes.txt"

2348

rel="nofollow">Show Source</a></li>

2349

</ul>

2350

2351

2352

<h3>Quick search</h3>

2353

2354

2355

2356

2357

2358

</form>

2359

2360

Enter search terms or a module, class or function name.

2361

2362

</div>

2363

2364

</div>

2365

</div>

2366

2367

</div>

2368

2369

<h3>Navigation</h3>

2370

<ul>

2371

2372

<a href="../genindex.html" title="General Index"

2373

>index</a></li>

2374

2375

<a href="../py-modindex.html" title="Python Module Index"

2376

>modules</a> |</li>

2377

2378

<a href="building.html" title="3. Building C and C++ Extensions"

2379

>next</a> |</li>

2380

2381

<a href="extending.html" title="1. Extending Python with C or C++"

2382

>previous</a> |</li>

2383

<li><img src="../_static/py.png" alt=""

2384

style="vertical-align: middle; margin-top: -1px"/></li>

2385

<li><a href="https://www.python.org/">Python</a> »</li>

2386

<li>

2387

3.5.1

2388

<a href="../index.html">Documentation </a> »

2389

</li>

2390

2391

<li class="nav-item nav-item-1"><a href="index.html" >Extending and Embedding the Python Interpreter</a> »</li>

2392

</ul>

2393

</div>

2394

2395

2396

2397

The Python Software Foundation is a non-profit corporation.

2398

<a href="https://www.python.org/psf/donations/">Please donate.</a>

2399

2400

Last updated on Jan 22, 2016.

2401

<a href="../bugs.html">Found a bug</a>?

2402

2403

Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.3.3.

2404

</div>

2405

2406

</body>

2407

</html>

b'\\ No newline at end of file'

Older »