1
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
3
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
4
<title>SQLite TH3</title>
5
<style type="text/css">
8
font-family: Verdana, sans-serif;
13
a:visited { color: #734559 }
15
.logo { position:absolute; margin:3px; }
31
.toolbar a { color: white; text-decoration: none; padding: 6px 12px; }
32
.toolbar a:visited { color: white; }
33
.toolbar a:hover { color: #044a64; background: white; }
35
.content { margin: 5%; }
36
.content dt { font-weight:bold; }
37
.content dd { margin-bottom: 25px; margin-left:20%; }
38
.content ul { padding:0px; padding-left: 15px; margin:0px; }
41
.se { background: url(images/se.gif) 100% 100% no-repeat #044a64}
42
.sw { background: url(images/sw.gif) 0% 100% no-repeat }
43
.ne { background: url(images/ne.gif) 100% 0% no-repeat }
44
.nw { background: url(images/nw.gif) 0% 0% no-repeat }
46
/* Things for "fancyformat" documents start here. */
47
.fancy img+p {font-style:italic}
48
.fancy .codeblock i { color: darkblue; }
49
.fancy h1,.fancy h2,.fancy h3,.fancy h4 {font-weight:normal;color:#044a64}
50
.fancy h2 { margin-left: 10px }
51
.fancy h3 { margin-left: 20px }
52
.fancy h4 { margin-left: 30px }
53
.fancy th {white-space:nowrap;text-align:left;border-bottom:solid 1px #444}
54
.fancy th, .fancy td {padding: 0.2em 1ex; vertical-align:top}
55
.fancy #toc a { color: darkblue ; text-decoration: none }
56
.fancy .todo { color: #AA3333 ; font-style : italic }
57
.fancy .todo:before { content: 'TODO:' }
58
.fancy p.todo { border: solid #AA3333 1px; padding: 1ex }
59
.fancy img { display:block; }
60
.fancy :link:hover, .fancy :visited:hover { background: wheat }
61
.fancy p,.fancy ul,.fancy ol { margin: 1em 5ex }
62
.fancy li p { margin: 1em 0 }
63
/* End of "fancyformat" specific rules. */
69
<div><!-- container div to satisfy validator -->
72
<img class="logo" src="images/sqlite370_banner.gif" alt="SQLite Logo"
74
<div><!-- IE hack to prevent disappearing logo--></div>
75
<div class="tagline">Small. Fast. Reliable.<br>Choose any three.</div>
77
<table width=100% style="clear:both"><tr><td>
78
<div class="se"><div class="sw"><div class="ne"><div class="nw">
79
<table width=100% style="padding:0;margin:0;cell-spacing:0"><tr>
82
<a href="about.html">About</a>
83
<a href="sitemap.html">Sitemap</a>
84
<a href="docs.html">Documentation</a>
85
<a href="download.html">Download</a>
86
<a href="copyright.html">License</a>
87
<a href="news.html">News</a>
88
<a href="support.html">Support</a>
91
gMsg = "Search SQLite Docs..."
92
function entersearch() {
93
var q = document.getElementById("q");
94
if( q.value == gMsg ) { q.value = "" }
95
q.style.color = "black"
96
q.style.fontStyle = "normal"
98
function leavesearch() {
99
var q = document.getElementById("q");
100
if( q.value == "" ) {
102
q.style.color = "#044a64"
103
q.style.fontStyle = "italic"
108
<div style="padding:0 1em 0px 0;white-space:nowrap">
109
<form name=f method="GET" action="http://www.sqlite.org/search">
110
<input id=q name=q type=text
111
onfocus="entersearch()" onblur="leavesearch()" style="width:24ex;padding:1px 1ex; border:solid white 1px; font-size:0.9em ; font-style:italic;color:#044a64;" value="Search SQLite Docs...">
112
<input type=submit value="Go" style="border:solid white 1px;background-color:#044a64;color:white;font-size:0.9em;padding:0 1ex">
116
</div></div></div></div>
118
<div class=startsearch></div>
123
<h1 align="center">TH3: SQLite Test Harness #3</h1>
125
<h2>1.0 Overview</h2>
127
<p>SQLite Test Harness #3 (hereafter "TH3") is one of
128
<a href="testing.html#harnesses">three test harnesses</a> used for testing SQLite.
129
TH3 is designed to meet the following objectives:</p>
132
<li><p> TH3 runs on embedded platforms that lack the support
133
infrastructure of workstations.</p></li>
135
<li><p> TH3 tests SQLite in an as-deployed configuration using only
136
published and documented interfaces.
137
In other words, TH3 tests the compiled object code, not
138
the source code, thus verifying that no problems were introduced
139
by compiler bugs.</p></li>
141
<li><p> TH3 checks SQLite's response to out-of-memory errors, disk I/O
142
errors, and power loss during transaction commit. </p></li>
144
<li><p> TH3 exercises SQLite in a variety of run-time configurations
145
(UTF8 vs UTF16, different pages sizes, varying journal modes, etc.)
148
<li><p> TH3 achieves 100% branch test coverage over SQLite core.
149
(Test coverage of the operating-system specific <a href="vfs.html">VFSes</a> and extensions
150
such as FTS and RTREE is less than 100%). </p></li>
153
<p>TH3 was originally written for validation testing only, but has
154
subsequently been used for development testing and debugging
155
as well, and has proven very helpful in those roles. A full-coverage
156
test run for TH3 takes less than 10 minutes on a workstation and hence
157
serves as a fast but effect regression test during day-to-day maintenance
158
of the SQLite code base.</p>
160
<h2>2.0 Operation</h2>
162
<p>TH3 is a test program generator. The output of TH3 is a program
163
written in ANSI-C and intended to be
164
linked against the SQLite library under test. The generated test
165
program is compiled and run on the target platform in order to verify
166
correct operation of SQLite on that platform.</p>
168
<p>The inputs to TH3 are test modules written in C or SQL and
170
files that determine how to initialize SQLite. The
171
TH3 package includes hundreds of test
172
modules and dozens of configuration files. New modules and configurations
173
can be added to customize TH3 for specialized applications.
174
Each time TH3 is run, it reads
175
a subset of the available test modules and configuration files to generate
176
a custom C program that performs all of the specified tests under all
177
configurations. A complete test of SQLite normally involves running
178
TH3 multiple times to generate multiple test programs covering different
179
aspects of SQLite's operation, then linking all test programs against
180
a common SQLite library and running them separately on the target platform.
181
SQLite will be found to work if all test programs pass.</p>
183
<p>There are no arbitrary limits in TH3. One could generate a
184
single test program that contained all test modules and configuration files.
185
However, such a test program might be too large to deploy on embedded
186
platforms. Hence, TH3 provides the ability to break the library of test
187
modules up into smaller, more easily digested pieces.</p>
189
<p>Each individual test module might contain dozens, hundreds, or thousands
190
of separate tests. The test modules can be written in C or as scripts of
192
Test modules done in SQL are very easy to write and
193
the test modules written in C are not as nearly as cumbersome
194
to write as one might suppose. The TH3
195
system provides high-level interfaces that simplify test writing.
196
A typical C-language test case in TH3 might contains slightly more syntax that
197
the corresponding TCL-script test, but the difference is not that great.
198
The test modules written as SQL contain special comments that define
199
the boundaries and operation of each test case and the expected results.</p>
201
<p>Each test module file contains a header which describes the circumstances
202
under which the test is valid. For a particular configuration, only those
203
modules that are compatible with the configuration are run. </p>
205
<h2>3.0 Generating A Test Program</h2>
207
<p>The TH3 program generator is a TCL script named "<tt>mkth3.tcl</tt>".
208
To generate a test program, one has merely to run this script and supply
209
the names of files containing test modules and configurations on the
210
command line. TH3 does not force any specific names for test modules
211
or configuration files, but it is customary to use suffixes "<tt>.test</tt>"
212
and "<tt>.cfg</tt>". With these conventions, one might generate a
213
test program as follows:</p>
216
tclsh mkth3.tcl *.test *.cfg >testprog1.c
219
<p>The output from the mkth3.tcl script is a C program that contains
220
everything needed to run the tests - everything that is except for
221
the SQLite library itself. The generated test program contains
222
implementations for all of the support interfaces used by the test
223
modules and it contains the <tt>main()</tt> routine that drives the
224
tests. To convert the test program into a working executable, simply
225
compile it against SQLite:</p>
228
cc -o testprog1 testprog1.c sqlite3.c
231
<p>The compilation step shown immediately above is merely representative.
232
In a working installation, one would normally want
233
to specify optimization parameters and compile-time switches on the
234
compiler command line.</p>
236
<p>Once the test program is generated, it is run with no arguments to
237
perform the tests. Progress information as well as error diagnostics
238
appear on standard output. The program returns zero if there are no
239
errors and non-zero if any problems were detected.</p>
241
<h2>4.0 Test Coverage</h2>
243
<p>Using one particular subset of the available TH3 test modules (the "cov1"
244
tests) SQLite obtained
245
<a href="testing.html#coverage">100% branch test coverage</a> and 100% <a href="testing.html#mcdc">MC/DC</a> as measured
246
by <a href="http://gcc.gnu.org/onlinedocs/gcc/Gcov.html">gcov</a>
247
on SuSE Linux 10.1 on x86 hardware on 2009-07-25. The SQLite developers
248
are committed to maintaining 100% branch coverage and MC/DC for all
249
future releases of SQLite.</p>
251
<p>The cov1 test set used to obtain 100% branch test coverage are only a
252
subset of the tests currently implemented using TH3. New test modules are
253
added on a regular basis.</p>
255
<h2>5.0 TH3 License</h2>
257
<p>SQLite itself is in the <a href="copyright.html">public domain</a> and
258
can be used for any purpose. But TH3 is proprietary and requires a license.
259
Members of the <a href="consortium.html">SQLite Consortium</a> get free and unlimited access to TH3.
260
Others can contact the SQLite developers for information on how to obtain
261
a license to access and use TH3.</p>
263
<p>Licensees of TH3 are given read access to the software configuration
264
management system used to manage TH3 and so can download the latest version
265
of TH3 (or any historical version) whenever they like.</p>
267
<p>Even though open-source users do not have direct access to TH3, all
268
users of SQLite benefit from TH3 indirectly since each version of SQLite is
269
validated by TH3 prior to release. So anyone using an official release
270
of SQLite can deploy their application with the confidence of knowing that
271
it has been tested using TH3. They simply cannot rerun those tests
272
themselves without purchasing a TH3 license.</p>
added
removed
th3.html
