163
163
<p>Provides an API for creation of custom ClauseElements and compilers.</p>
164
164
<div class="section" id="synopsis">
165
165
<h2>Synopsis<a class="headerlink" href="#synopsis" title="Permalink to this headline">¶</a></h2>
166
<p>Usage involves the creation of one or more <a class="reference internal" href="expression_api.html#sqlalchemy.sql.expression.ClauseElement" title="sqlalchemy.sql.expression.ClauseElement"><tt class="xref py py-class docutils literal"><span class="pre">ClauseElement</span></tt></a>
167
subclasses and one or more callables defining its compilation:</p>
166
<p>Usage involves the creation of one or more
167
<a class="reference internal" href="expression_api.html#sqlalchemy.sql.expression.ClauseElement" title="sqlalchemy.sql.expression.ClauseElement"><tt class="xref py py-class docutils literal"><span class="pre">ClauseElement</span></tt></a> subclasses and one or
168
more callables defining its compilation:</p>
168
169
<div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">sqlalchemy.ext.compiler</span> <span class="kn">import</span> <span class="n">compiles</span>
169
170
<span class="kn">from</span> <span class="nn">sqlalchemy.sql.expression</span> <span class="kn">import</span> <span class="n">ColumnClause</span>
208
209
<span class="k">def</span> <span class="nf">visit_alter_column</span><span class="p">(</span><span class="n">element</span><span class="p">,</span> <span class="n">compiler</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">):</span>
209
210
<span class="k">return</span> <span class="s">"ALTER TABLE </span><span class="si">%s</span><span class="s"> ALTER COLUMN </span><span class="si">%s</span><span class="s"> ..."</span> <span class="o">%</span> <span class="p">(</span><span class="n">element</span><span class="o">.</span><span class="n">table</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">element</span><span class="o">.</span><span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">)</span></pre></div>
211
<p>The second <tt class="docutils literal"><span class="pre">visit_alter_table</span></tt> will be invoked when any <tt class="docutils literal"><span class="pre">postgresql</span></tt> dialect is used.</p>
212
<p>The second <tt class="docutils literal"><span class="pre">visit_alter_table</span></tt> will be invoked when any <tt class="docutils literal"><span class="pre">postgresql</span></tt>
213
215
<div class="section" id="compiling-sub-elements-of-a-custom-expression-construct">
214
216
<h2>Compiling sub-elements of a custom expression construct<a class="headerlink" href="#compiling-sub-elements-of-a-custom-expression-construct" title="Permalink to this headline">¶</a></h2>
215
<p>The <tt class="docutils literal"><span class="pre">compiler</span></tt> argument is the <a class="reference internal" href="internals.html#sqlalchemy.engine.base.Compiled" title="sqlalchemy.engine.base.Compiled"><tt class="xref py py-class docutils literal"><span class="pre">Compiled</span></tt></a>
216
object in use. This object can be inspected for any information about the
217
in-progress compilation, including <tt class="docutils literal"><span class="pre">compiler.dialect</span></tt>,
218
<tt class="docutils literal"><span class="pre">compiler.statement</span></tt> etc. The <a class="reference internal" href="internals.html#sqlalchemy.sql.compiler.SQLCompiler" title="sqlalchemy.sql.compiler.SQLCompiler"><tt class="xref py py-class docutils literal"><span class="pre">SQLCompiler</span></tt></a>
219
and <a class="reference internal" href="internals.html#sqlalchemy.sql.compiler.DDLCompiler" title="sqlalchemy.sql.compiler.DDLCompiler"><tt class="xref py py-class docutils literal"><span class="pre">DDLCompiler</span></tt></a> both include a <tt class="docutils literal"><span class="pre">process()</span></tt>
217
<p>The <tt class="docutils literal"><span class="pre">compiler</span></tt> argument is the
218
<a class="reference internal" href="internals.html#sqlalchemy.engine.interfaces.Compiled" title="sqlalchemy.engine.interfaces.Compiled"><tt class="xref py py-class docutils literal"><span class="pre">Compiled</span></tt></a> object in use. This object
219
can be inspected for any information about the in-progress compilation,
220
including <tt class="docutils literal"><span class="pre">compiler.dialect</span></tt>, <tt class="docutils literal"><span class="pre">compiler.statement</span></tt> etc. The
221
<a class="reference internal" href="internals.html#sqlalchemy.sql.compiler.SQLCompiler" title="sqlalchemy.sql.compiler.SQLCompiler"><tt class="xref py py-class docutils literal"><span class="pre">SQLCompiler</span></tt></a> and
222
<a class="reference internal" href="internals.html#sqlalchemy.sql.compiler.DDLCompiler" title="sqlalchemy.sql.compiler.DDLCompiler"><tt class="xref py py-class docutils literal"><span class="pre">DDLCompiler</span></tt></a> both include a <tt class="docutils literal"><span class="pre">process()</span></tt>
220
223
method which can be used for compilation of embedded attributes:</p>
221
224
<div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">sqlalchemy.sql.expression</span> <span class="kn">import</span> <span class="n">Executable</span><span class="p">,</span> <span class="n">ClauseElement</span>
246
249
<div class="section" id="cross-compiling-between-sql-and-ddl-compilers">
247
250
<h3>Cross Compiling between SQL and DDL compilers<a class="headerlink" href="#cross-compiling-between-sql-and-ddl-compilers" title="Permalink to this headline">¶</a></h3>
248
<p>SQL and DDL constructs are each compiled using different base compilers - <tt class="docutils literal"><span class="pre">SQLCompiler</span></tt>
249
and <tt class="docutils literal"><span class="pre">DDLCompiler</span></tt>. A common need is to access the compilation rules of SQL expressions
250
from within a DDL expression. The <tt class="docutils literal"><span class="pre">DDLCompiler</span></tt> includes an accessor <tt class="docutils literal"><span class="pre">sql_compiler</span></tt> for this reason, such as below where we generate a CHECK
251
constraint that embeds a SQL expression:</p>
251
<p>SQL and DDL constructs are each compiled using different base compilers -
252
<tt class="docutils literal"><span class="pre">SQLCompiler</span></tt> and <tt class="docutils literal"><span class="pre">DDLCompiler</span></tt>. A common need is to access the
253
compilation rules of SQL expressions from within a DDL expression. The
254
<tt class="docutils literal"><span class="pre">DDLCompiler</span></tt> includes an accessor <tt class="docutils literal"><span class="pre">sql_compiler</span></tt> for this reason, such as
255
below where we generate a CHECK constraint that embeds a SQL expression:</p>
252
256
<div class="highlight-python"><div class="highlight"><pre><span class="nd">@compiles</span><span class="p">(</span><span class="n">MyConstraint</span><span class="p">)</span>
253
257
<span class="k">def</span> <span class="nf">compile_my_constraint</span><span class="p">(</span><span class="n">constraint</span><span class="p">,</span> <span class="n">ddlcompiler</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">):</span>
254
258
<span class="k">return</span> <span class="s">"CONSTRAINT </span><span class="si">%s</span><span class="s"> CHECK (</span><span class="si">%s</span><span class="s">)"</span> <span class="o">%</span> <span class="p">(</span>
261
265
<div class="section" id="enabling-autocommit-on-a-construct">
262
266
<span id="enabling-compiled-autocommit"></span><h2>Enabling Autocommit on a Construct<a class="headerlink" href="#enabling-autocommit-on-a-construct" title="Permalink to this headline">¶</a></h2>
263
<p>Recall from the section <a class="reference internal" href="connections.html#autocommit"><em>Understanding Autocommit</em></a> that the <a class="reference internal" href="connections.html#sqlalchemy.engine.base.Engine" title="sqlalchemy.engine.base.Engine"><tt class="xref py py-class docutils literal"><span class="pre">Engine</span></tt></a>, when asked to execute
264
a construct in the absence of a user-defined transaction, detects if the given
265
construct represents DML or DDL, that is, a data modification or data definition statement, which
266
requires (or may require, in the case of DDL) that the transaction generated by the DBAPI be committed
267
(recall that DBAPI always has a transaction going on regardless of what SQLAlchemy does). Checking
268
for this is actually accomplished
269
by checking for the “autocommit” execution option on the construct. When building a construct like
270
an INSERT derivation, a new DDL type, or perhaps a stored procedure that alters data, the “autocommit”
271
option needs to be set in order for the statement to function with “connectionless” execution
267
<p>Recall from the section <a class="reference internal" href="connections.html#autocommit"><em>Understanding Autocommit</em></a> that the <a class="reference internal" href="connections.html#sqlalchemy.engine.Engine" title="sqlalchemy.engine.Engine"><tt class="xref py py-class docutils literal"><span class="pre">Engine</span></tt></a>, when
268
asked to execute a construct in the absence of a user-defined transaction,
269
detects if the given construct represents DML or DDL, that is, a data
270
modification or data definition statement, which requires (or may require,
271
in the case of DDL) that the transaction generated by the DBAPI be committed
272
(recall that DBAPI always has a transaction going on regardless of what
273
SQLAlchemy does). Checking for this is actually accomplished by checking for
274
the “autocommit” execution option on the construct. When building a
275
construct like an INSERT derivation, a new DDL type, or perhaps a stored
276
procedure that alters data, the “autocommit” option needs to be set in order
277
for the statement to function with “connectionless” execution
272
278
(as described in <a class="reference internal" href="connections.html#dbengine-implicit"><em>Connectionless Execution, Implicit Execution</em></a>).</p>
273
<p>Currently a quick way to do this is to subclass <a class="reference internal" href="expression_api.html#sqlalchemy.sql.expression.Executable" title="sqlalchemy.sql.expression.Executable"><tt class="xref py py-class docutils literal"><span class="pre">Executable</span></tt></a>, then add the “autocommit” flag
274
to the <tt class="docutils literal"><span class="pre">_execution_options</span></tt> dictionary (note this is a “frozen” dictionary which supplies a generative
275
<tt class="docutils literal"><span class="pre">union()</span></tt> method):</p>
279
<p>Currently a quick way to do this is to subclass <a class="reference internal" href="expression_api.html#sqlalchemy.sql.expression.Executable" title="sqlalchemy.sql.expression.Executable"><tt class="xref py py-class docutils literal"><span class="pre">Executable</span></tt></a>, then
280
add the “autocommit” flag to the <tt class="docutils literal"><span class="pre">_execution_options</span></tt> dictionary (note this
281
is a “frozen” dictionary which supplies a generative <tt class="docutils literal"><span class="pre">union()</span></tt> method):</p>
276
282
<div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">sqlalchemy.sql.expression</span> <span class="kn">import</span> <span class="n">Executable</span><span class="p">,</span> <span class="n">ClauseElement</span>
278
284
<span class="k">class</span> <span class="nc">MyInsertThing</span><span class="p">(</span><span class="n">Executable</span><span class="p">,</span> <span class="n">ClauseElement</span><span class="p">):</span>
279
285
<span class="n">_execution_options</span> <span class="o">=</span> \
280
286
<span class="n">Executable</span><span class="o">.</span><span class="n">_execution_options</span><span class="o">.</span><span class="n">union</span><span class="p">({</span><span class="s">'autocommit'</span><span class="p">:</span> <span class="bp">True</span><span class="p">})</span></pre></div>
282
<p>More succinctly, if the construct is truly similar to an INSERT, UPDATE, or DELETE, <a class="reference internal" href="expression_api.html#sqlalchemy.sql.expression.UpdateBase" title="sqlalchemy.sql.expression.UpdateBase"><tt class="xref py py-class docutils literal"><span class="pre">UpdateBase</span></tt></a>
283
can be used, which already is a subclass of <a class="reference internal" href="expression_api.html#sqlalchemy.sql.expression.Executable" title="sqlalchemy.sql.expression.Executable"><tt class="xref py py-class docutils literal"><span class="pre">Executable</span></tt></a>, <a class="reference internal" href="expression_api.html#sqlalchemy.sql.expression.ClauseElement" title="sqlalchemy.sql.expression.ClauseElement"><tt class="xref py py-class docutils literal"><span class="pre">ClauseElement</span></tt></a> and includes the
288
<p>More succinctly, if the construct is truly similar to an INSERT, UPDATE, or
289
DELETE, <a class="reference internal" href="expression_api.html#sqlalchemy.sql.expression.UpdateBase" title="sqlalchemy.sql.expression.UpdateBase"><tt class="xref py py-class docutils literal"><span class="pre">UpdateBase</span></tt></a> can be used, which already is a subclass
290
of <a class="reference internal" href="expression_api.html#sqlalchemy.sql.expression.Executable" title="sqlalchemy.sql.expression.Executable"><tt class="xref py py-class docutils literal"><span class="pre">Executable</span></tt></a>, <a class="reference internal" href="expression_api.html#sqlalchemy.sql.expression.ClauseElement" title="sqlalchemy.sql.expression.ClauseElement"><tt class="xref py py-class docutils literal"><span class="pre">ClauseElement</span></tt></a> and includes the
284
291
<tt class="docutils literal"><span class="pre">autocommit</span></tt> flag:</p>
285
292
<div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">sqlalchemy.sql.expression</span> <span class="kn">import</span> <span class="n">UpdateBase</span>
288
295
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">...</span><span class="p">):</span>
289
296
<span class="o">...</span></pre></div>
291
<p>DDL elements that subclass <a class="reference internal" href="schema.html#sqlalchemy.schema.DDLElement" title="sqlalchemy.schema.DDLElement"><tt class="xref py py-class docutils literal"><span class="pre">DDLElement</span></tt></a> already have the “autocommit” flag turned on.</p>
298
<p>DDL elements that subclass <a class="reference internal" href="schema.html#sqlalchemy.schema.DDLElement" title="sqlalchemy.schema.DDLElement"><tt class="xref py py-class docutils literal"><span class="pre">DDLElement</span></tt></a> already have the
299
“autocommit” flag turned on.</p>
293
301
<div class="section" id="changing-the-default-compilation-of-existing-constructs">
294
302
<h2>Changing the default compilation of existing constructs<a class="headerlink" href="#changing-the-default-compilation-of-existing-constructs" title="Permalink to this headline">¶</a></h2>
295
<p>The compiler extension applies just as well to the existing constructs. When overriding
296
the compilation of a built in SQL construct, the @compiles decorator is invoked upon
297
the appropriate class (be sure to use the class, i.e. <tt class="docutils literal"><span class="pre">Insert</span></tt> or <tt class="docutils literal"><span class="pre">Select</span></tt>, instead of the creation function such as <tt class="docutils literal"><span class="pre">insert()</span></tt> or <tt class="docutils literal"><span class="pre">select()</span></tt>).</p>
298
<p>Within the new compilation function, to get at the “original” compilation routine,
299
use the appropriate visit_XXX method - this because compiler.process() will call upon the
300
overriding routine and cause an endless loop. Such as, to add “prefix” to all insert statements:</p>
303
<p>The compiler extension applies just as well to the existing constructs. When
304
overriding the compilation of a built in SQL construct, the @compiles
305
decorator is invoked upon the appropriate class (be sure to use the class,
306
i.e. <tt class="docutils literal"><span class="pre">Insert</span></tt> or <tt class="docutils literal"><span class="pre">Select</span></tt>, instead of the creation function such
307
as <tt class="docutils literal"><span class="pre">insert()</span></tt> or <tt class="docutils literal"><span class="pre">select()</span></tt>).</p>
308
<p>Within the new compilation function, to get at the “original” compilation
309
routine, use the appropriate visit_XXX method - this
310
because compiler.process() will call upon the overriding routine and cause
311
an endless loop. Such as, to add “prefix” to all insert statements:</p>
301
312
<div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">sqlalchemy.sql.expression</span> <span class="kn">import</span> <span class="n">Insert</span>
303
314
<span class="nd">@compiles</span><span class="p">(</span><span class="n">Insert</span><span class="p">)</span>
304
315
<span class="k">def</span> <span class="nf">prefix_inserts</span><span class="p">(</span><span class="n">insert</span><span class="p">,</span> <span class="n">compiler</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">):</span>
305
316
<span class="k">return</span> <span class="n">compiler</span><span class="o">.</span><span class="n">visit_insert</span><span class="p">(</span><span class="n">insert</span><span class="o">.</span><span class="n">prefix_with</span><span class="p">(</span><span class="s">"some prefix"</span><span class="p">),</span> <span class="o">**</span><span class="n">kw</span><span class="p">)</span></pre></div>
307
<p>The above compiler will prefix all INSERT statements with “some prefix” when compiled.</p>
318
<p>The above compiler will prefix all INSERT statements with “some prefix” when
309
321
<div class="section" id="changing-compilation-of-types">
310
322
<span id="type-compilation-extension"></span><h2>Changing Compilation of Types<a class="headerlink" href="#changing-compilation-of-types" title="Permalink to this headline">¶</a></h2>
311
<p><tt class="docutils literal"><span class="pre">compiler</span></tt> works for types, too, such as below where we implement the MS-SQL specific ‘max’ keyword for <tt class="docutils literal"><span class="pre">String</span></tt>/<tt class="docutils literal"><span class="pre">VARCHAR</span></tt>:</p>
323
<p><tt class="docutils literal"><span class="pre">compiler</span></tt> works for types, too, such as below where we implement the
324
MS-SQL specific ‘max’ keyword for <tt class="docutils literal"><span class="pre">String</span></tt>/<tt class="docutils literal"><span class="pre">VARCHAR</span></tt>:</p>
312
325
<div class="highlight-python"><div class="highlight"><pre><span class="nd">@compiles</span><span class="p">(</span><span class="n">String</span><span class="p">,</span> <span class="s">'mssql'</span><span class="p">)</span>
313
326
<span class="nd">@compiles</span><span class="p">(</span><span class="n">VARCHAR</span><span class="p">,</span> <span class="s">'mssql'</span><span class="p">)</span>
314
327
<span class="k">def</span> <span class="nf">compile_varchar</span><span class="p">(</span><span class="n">element</span><span class="p">,</span> <span class="n">compiler</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">):</span>
375
388
<tt class="docutils literal"><span class="pre">execute_at()</span></tt> method, allowing the construct to be invoked during CREATE
376
389
TABLE and DROP TABLE sequences.</p>
378
<li><p class="first"><a class="reference internal" href="expression_api.html#sqlalchemy.sql.expression.Executable" title="sqlalchemy.sql.expression.Executable"><tt class="xref py py-class docutils literal"><span class="pre">Executable</span></tt></a> - This is a mixin which should be
379
used with any expression class that represents a “standalone” SQL statement that
380
can be passed directly to an <tt class="docutils literal"><span class="pre">execute()</span></tt> method. It is already implicit
381
within <tt class="docutils literal"><span class="pre">DDLElement</span></tt> and <tt class="docutils literal"><span class="pre">FunctionElement</span></tt>.</p>
391
<li><p class="first"><a class="reference internal" href="expression_api.html#sqlalchemy.sql.expression.Executable" title="sqlalchemy.sql.expression.Executable"><tt class="xref py py-class docutils literal"><span class="pre">Executable</span></tt></a> - This is a mixin which
392
should be used with any expression class that represents a “standalone”
393
SQL statement that can be passed directly to an <tt class="docutils literal"><span class="pre">execute()</span></tt> method. It
394
is already implicit within <tt class="docutils literal"><span class="pre">DDLElement</span></tt> and <tt class="docutils literal"><span class="pre">FunctionElement</span></tt>.</p>
386
399
<h2>Further Examples<a class="headerlink" href="#further-examples" title="Permalink to this headline">¶</a></h2>
387
400
<div class="section" id="utc-timestamp-function">
388
401
<h3>“UTC timestamp” function<a class="headerlink" href="#utc-timestamp-function" title="Permalink to this headline">¶</a></h3>
389
<p>A function that works like “CURRENT_TIMESTAMP” except applies the appropriate conversions
390
so that the time is in UTC time. Timestamps are best stored in relational databases
391
as UTC, without time zones. UTC so that your database doesn’t think time has gone
392
backwards in the hour when daylight savings ends, without timezones because timezones
393
are like character encodings - they’re best applied only at the endpoints of an
394
application (i.e. convert to UTC upon user input, re-apply desired timezone upon display).</p>
402
<p>A function that works like “CURRENT_TIMESTAMP” except applies the
403
appropriate conversions so that the time is in UTC time. Timestamps are best
404
stored in relational databases as UTC, without time zones. UTC so that your
405
database doesn’t think time has gone backwards in the hour when daylight
406
savings ends, without timezones because timezones are like character
407
encodings - they’re best applied only at the endpoints of an application
408
(i.e. convert to UTC upon user input, re-apply desired timezone upon display).</p>
395
409
<p>For Postgresql and Microsoft SQL Server:</p>
396
410
<div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">sqlalchemy.sql</span> <span class="kn">import</span> <span class="n">expression</span>
397
411
<span class="kn">from</span> <span class="nn">sqlalchemy.ext.compiler</span> <span class="kn">import</span> <span class="n">compiles</span>
423
437
<div class="section" id="greatest-function">
424
438
<h3>“GREATEST” function<a class="headerlink" href="#greatest-function" title="Permalink to this headline">¶</a></h3>
425
<p>The “GREATEST” function is given any number of arguments and returns the one that is
426
of the highest value - it’s equivalent to Python’s <tt class="docutils literal"><span class="pre">max</span></tt> function. A SQL
427
standard version versus a CASE based version which only accommodates two
439
<p>The “GREATEST” function is given any number of arguments and returns the one
440
that is of the highest value - it’s equivalent to Python’s <tt class="docutils literal"><span class="pre">max</span></tt>
441
function. A SQL standard version versus a CASE based version which only
442
accommodates two arguments:</p>
429
443
<div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">sqlalchemy.sql</span> <span class="kn">import</span> <span class="n">expression</span>
430
444
<span class="kn">from</span> <span class="nn">sqlalchemy.ext.compiler</span> <span class="kn">import</span> <span class="n">compiles</span>
431
445
<span class="kn">from</span> <span class="nn">sqlalchemy.types</span> <span class="kn">import</span> <span class="n">Numeric</span>
486
501
<span class="n">select</span><span class="p">([</span><span class="n">customers</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">customers</span><span class="o">.</span><span class="n">c</span><span class="o">.</span><span class="n">enrolled</span><span class="p">])</span>
487
502
<span class="p">)</span></pre></div>
504
<dl class="function">
505
<dt id="sqlalchemy.ext.compiler.compiles">
506
<tt class="descclassname">sqlalchemy.ext.compiler.</tt><tt class="descname">compiles</tt><big>(</big><em>class_</em>, <em>*specs</em><big>)</big><a class="headerlink" href="#sqlalchemy.ext.compiler.compiles" title="Permalink to this definition">¶</a></dt>
507
<dd><p>Register a function as a compiler for a
508
given <a class="reference internal" href="expression_api.html#sqlalchemy.sql.expression.ClauseElement" title="sqlalchemy.sql.expression.ClauseElement"><tt class="xref py py-class docutils literal"><span class="pre">ClauseElement</span></tt></a> type.</p>
511
<dl class="function">
512
<dt id="sqlalchemy.ext.compiler.deregister">
513
<tt class="descclassname">sqlalchemy.ext.compiler.</tt><tt class="descname">deregister</tt><big>(</big><em>class_</em><big>)</big><a class="headerlink" href="#sqlalchemy.ext.compiler.deregister" title="Permalink to this definition">¶</a></dt>
514
<dd><p>Remove all custom compilers associated with a given
515
<a class="reference internal" href="expression_api.html#sqlalchemy.sql.expression.ClauseElement" title="sqlalchemy.sql.expression.ClauseElement"><tt class="xref py py-class docutils literal"><span class="pre">ClauseElement</span></tt></a> type.</p>