25
font-variant: small-caps;
26
25
text-align: center;
27
26
line-height: 1.6em;
31
30
.toolbar a { color: white; text-decoration: none; padding: 6px 12px; }
32
31
.toolbar a:visited { color: white; }
33
.toolbar a:hover { color: #80a796; background: white; }
32
.toolbar a:hover { color: #044a64; background: white; }
35
34
.content { margin: 5%; }
36
35
.content dt { font-weight:bold; }
38
37
.content ul { padding:0px; padding-left: 15px; margin:0px; }
40
39
/* rounded corners */
41
.se { background: url(images/se.png) 100% 100% no-repeat #80a796}
42
.sw { background: url(images/sw.png) 0% 100% no-repeat }
43
.ne { background: url(images/ne.png) 100% 0% no-repeat }
44
.nw { background: url(images/nw.png) 0% 0% no-repeat }
40
.se { background: url(images/se.gif) 100% 100% no-repeat #044a64}
41
.sw { background: url(images/sw.gif) 0% 100% no-repeat }
42
.ne { background: url(images/ne.gif) 100% 0% no-repeat }
43
.nw { background: url(images/nw.gif) 0% 0% no-repeat }
46
45
/* Things for "fancyformat" documents start here. */
46
.fancy img+p {font-style:italic}
47
47
.fancy .codeblock i { color: darkblue; }
48
.fancy h1,.fancy h2,.fancy h3,.fancy h4 {font-weight:normal;color:#80a796}
48
.fancy h1,.fancy h2,.fancy h3,.fancy h4 {font-weight:normal;color:#044a64}
49
49
.fancy h2 { margin-left: 10px }
50
50
.fancy h3 { margin-left: 20px }
51
51
.fancy h4 { margin-left: 30px }
69
69
<div><!-- container div to satisfy validator -->
71
71
<a href="index.html">
72
<img class="logo" src="images/SQLite.gif" alt="SQLite Logo"
72
<img class="logo" src="images/sqlite370_banner.gif" alt="SQLite Logo"
74
74
<div><!-- IE hack to prevent disappearing logo--></div>
75
75
<div class="tagline">Small. Fast. Reliable.<br>Choose any three.</div>
77
77
<table width=100% style="clear:both"><tr><td>
78
78
<div class="se"><div class="sw"><div class="ne"><div class="nw">
79
<table width=100% style="padding:0;margin:0;cell-spacing:0"><tr>
79
81
<div class="toolbar">
80
82
<a href="about.html">About</a>
81
83
<a href="sitemap.html">Sitemap</a>
83
85
<a href="download.html">Download</a>
84
86
<a href="copyright.html">License</a>
85
87
<a href="news.html">News</a>
86
<!-- <a href="dev.html">Developers</a> -->
87
88
<a href="support.html">Support</a>
88
</div></div></div></div></div>
91
gMsg = "Search SQLite Docs..."
92
function entersearch() {
93
var q = document.getElementById("q");
94
if( q.value == gMsg ) { q.value = "" }
95
q.style.color = "black"
96
q.style.fontStyle = "normal"
98
function leavesearch() {
99
var q = document.getElementById("q");
100
if( q.value == "" ) {
102
q.style.color = "#044a64"
103
q.style.fontStyle = "italic"
108
<div style="padding:0 1em 0px 0;white-space:nowrap">
109
<form name=f method="GET" action="http://www.sqlite.org/search">
110
<input id=q name=q type=text
111
onfocus="entersearch()" onblur="leavesearch()" style="width:24ex;padding:1px 1ex; border:solid white 1px; font-size:0.9em ; font-style:italic;color:#044a64;" value="Search SQLite Docs...">
112
<input type=submit value="Go" style="border:solid white 1px;background-color:#044a64;color:white;font-size:0.9em;padding:0 1ex">
116
</div></div></div></div>
89
117
</td></tr></table>
118
<div class=startsearch></div>
93
122
<h1>Dynamic Memory Allocation In SQLite</h1>
96
<p>SQLite uses of dynamic memory allocation to obtain
125
<p>SQLite uses dynamic memory allocation to obtain
97
126
memory for storing various objects
98
127
(ex: <a href="c3ref/sqlite3.html">database connections</a> and <a href="c3ref/stmt.html">prepared statements</a>) and to build
99
128
a memory cache of the database file and to hold the results of queries.
144
173
<b>Memory usage limits.</b>
145
The <a href="c3ref/soft_heap_limit.html">sqlite3_soft_heap_limit()</a> mechanism allows the application to
174
The <a href="c3ref/soft_heap_limit64.html">sqlite3_soft_heap_limit64()</a> mechanism allows the application to
146
175
set a memory usage limit that SQLite strives to stay below. SQLite
147
176
will attempt to reuse memory from its caches rather than allocation new
148
177
memory as it approaches the soft limit.
202
231
75% of the code in the SQLite source tree is devoted purely to
203
232
<a href="testing.html">testing and verification</a>. Reliability is important to SQLite.
204
Among the tasks of the test infrastructure is to insure that
233
Among the tasks of the test infrastructure is to ensure that
205
234
SQLite does not misuse dynamically allocated memory, that SQLite
206
235
does not leak memory, and that SQLite responds
207
236
correctly to a dynamic memory allocation failure.</p>
224
253
<li><p><b>Use of memory after freeing.</b>
225
254
When each block of memory is freed, every byte is overwritten with a
226
nonsense bit pattern. This helps to insure that no memory is ever
255
nonsense bit pattern. This helps to ensure that no memory is ever
227
256
used after having been freed.</p></li>
229
258
<li><p><b>Freeing memory not obtained from malloc.</b>
234
263
<li><p><b>Uninitialized memory.</b>
235
264
The instrumented memory allocator initializes each memory allocation
236
to a nonsense bit pattern to help insure that the user makes no
265
to a nonsense bit pattern to help ensure that the user makes no
237
266
assumptions about the content of allocation memory.</p></li>
241
270
used, SQLite keeps track of how much memory is currently checked out.
242
271
There are hundreds of test scripts used for testing SQLite. At the
243
272
end of each script, all objects are destroyed and a test is made to
244
insure that all memory has been freed. This is how memory
273
ensure that all memory has been freed. This is how memory
245
274
leaks are detected. Notice that memory leak detection is in force at
246
275
all times, during test builds and during production builds. Whenever
247
276
one of the developers runs any individual test script, memory leak
340
369
makes sure each new allocation is filled with nonsense data prior to
341
370
returning the allocation to the caller. And as soon as an allocation
342
371
is free, it is again filled with nonsense data. These two actions help
343
to insure that the SQLite core does not make assumptions about the state
372
to ensure that the SQLite core does not make assumptions about the state
344
373
of newly allocated memory and that memory allocations are not used after
345
374
they have been freed.</p>
362
391
interface at start-time:</p>
364
393
<blockquote><pre>
365
<a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_HEAP</a>, pBuf, szBuf, mnReq);
394
<a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html#sqliteconfigheap">SQLITE_CONFIG_HEAP</a>, pBuf, szBuf, mnReq);
366
395
</pre></blockquote>
368
397
<p>In the call above, pBuf is a pointer to a large, contiguous chunk
402
431
<p>If SQLite is compiled with <a href="compile.html#enable_memsys3">SQLITE_ENABLE_MEMSYS3</a> than another
403
432
zero-malloc memory allocator, similar to memsys5, is included in the
404
433
source tree. The memsys3 allocator, like memsys5, must be activated
405
by a call to <a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_HEAP</a>,...). Memsys3
434
by a call to <a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html#sqliteconfigheap">SQLITE_CONFIG_HEAP</a>,...). Memsys3
406
435
uses the memory buffer supplied as its source for all memory allocations.
407
436
The difference between memsys3 and memsys5 is that memsys3 uses a
408
437
different memory allocation algorithm that seems to work well in
459
488
simply calls:</p>
461
490
<blockquote><pre>
462
<a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_MALLOC</a>, pMem);
491
<a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html#sqliteconfigmalloc">SQLITE_CONFIG_MALLOC</a>, pMem);
463
492
</pre></blockquote>
465
494
<p>In the call above, pMem is a pointer to an <a href="c3ref/mem_methods.html">sqlite3_mem_methods</a> object
471
500
<p>In a multi-threaded application, access to the <a href="c3ref/mem_methods.html">sqlite3_mem_methods</a>
472
is serialized if and only if <a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_MEMSTATUS</a> is enabled.
473
If <a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_MEMSTATUS</a> is disabled then the methods in
501
is serialized if and only if <a href="c3ref/c_config_getmalloc.html#sqliteconfigmemstatus">SQLITE_CONFIG_MEMSTATUS</a> is enabled.
502
If <a href="c3ref/c_config_getmalloc.html#sqliteconfigmemstatus">SQLITE_CONFIG_MEMSTATUS</a> is disabled then the methods in
474
503
<a href="c3ref/mem_methods.html">sqlite3_mem_methods</a> must take care of their own serialization needs.</p>
476
505
<a name="overlayalloc"></a>
485
514
<p>An overlay can be created by using the</p>
487
516
<blockquote><pre>
488
<a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_GETMALLOC</a>, pOldMem);
517
<a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html#sqliteconfiggetmalloc">SQLITE_CONFIG_GETMALLOC</a>, pOldMem);
489
518
</pre></blockquote>
491
520
<p>interface to obtain pointers to the existing memory allocator.
492
521
The existing allocator is saved by the overlay and is used as
493
522
a fallback to do real memory allocation. Then the overlay is
494
523
inserted in place of the existing memory allocator using
495
the <a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_MALLOC</a>,...) as described
524
the <a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html#sqliteconfigmalloc">SQLITE_CONFIG_MALLOC</a>,...) as described
496
525
<a href="#appalloc">above</a>.
498
527
<a name="stuballoc"></a>
508
537
on systems that may not have malloc(), free(), or realloc() in their
509
538
standard library.
510
539
An application that is compiled with <a href="compile.html#zero_malloc">SQLITE_ZERO_MALLOC</a> will need to
511
use <a href="c3ref/config.html">sqlite3_config()</a> together with <a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_MALLOC</a> or
512
<a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_HEAP</a> to specify a new alternative memory allocator
540
use <a href="c3ref/config.html">sqlite3_config()</a> together with <a href="c3ref/c_config_getmalloc.html#sqliteconfigmalloc">SQLITE_CONFIG_MALLOC</a> or
541
<a href="c3ref/c_config_getmalloc.html#sqliteconfigheap">SQLITE_CONFIG_HEAP</a> to specify a new alternative memory allocator
513
542
before beginning to use SQLite.</p>
515
544
<a name="scratch"></a>
538
567
<p>The scratch memory allocator is set up as follows:</p>
540
569
<blockquote><pre>
541
<a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_SCRATCH</a>, pBuf, sz, N);
570
<a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html#sqliteconfigscratch">SQLITE_CONFIG_SCRATCH</a>, pBuf, sz, N);
542
571
</pre></blockquote>
544
573
<p>The pBuf parameter is a pointer to a contiguous range of bytes that
547
576
is the maximum size of each scratch allocation. N is the maximum
548
577
number of simultaneous scratch allocations. The "sz" parameter should
549
578
be approximately 6 times the maximum database page size. N should
550
be the number of threads running in the system. No single thread will
551
ever request more than one scratch allocation at a time so if there
579
be twice the number of threads running in the system. No single thread will
580
ever request more than two scratch allocation at a time so if there
552
581
are never more than N threads, then there will always be enough scratch
553
582
memory available.</p>
595
624
An application can enabled it at start-time as follows:</p>
597
626
<blockquote><pre>
598
<a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_PAGECACHE</a>, pBuf, sz, N);
627
<a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html#sqliteconfigpagecache">SQLITE_CONFIG_PAGECACHE</a>, pBuf, sz, N);
599
628
</pre></blockquote>
601
630
<p>The pBuf parameter is a pointer to a contiguous range of bytes that
663
692
following interface at start-time:</p>
665
694
<blockquote><pre>
666
<a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_LOOKASIDE</a>, sz, cnt);
695
<a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html#sqliteconfiglookaside">SQLITE_CONFIG_LOOKASIDE</a>, sz, cnt);
667
696
</pre></blockquote>
669
698
<p>The "sz" parameter is the size in bytes of each lookaside slot.
681
710
<a href="c3ref/sqlite3.html">database connection</a> "db" using this call:</p>
683
712
<blockquote><pre>
684
<a href="c3ref/db_config.html">sqlite3_db_config</a>(db, <a href="c3ref/c_dbconfig_lookaside.html">SQLITE_DBCONFIG_LOOKASIDE</a>, pBuf, sz, cnt);
713
<a href="c3ref/db_config.html">sqlite3_db_config</a>(db, <a href="c3ref/c_dbconfig_enable_fkey.html">SQLITE_DBCONFIG_LOOKASIDE</a>, pBuf, sz, cnt);
685
714
</pre></blockquote>
687
716
<p>The "pBuf" parameter is a pointer to memory space that will be
719
748
following interface is used at start-time:</p>
721
750
<blockquote><pre>
722
<a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_MEMSTATUS</a>, onoff);
751
<a href="c3ref/config.html">sqlite3_config</a>(<a href="c3ref/c_config_getmalloc.html#sqliteconfigmemstatus">SQLITE_CONFIG_MEMSTATUS</a>, onoff);
723
752
</pre></blockquote>
725
754
<p>The "onoff" parameter is true to enable the tracking of memory
729
758
to access them:</p>
731
760
<blockquote><pre>
732
<a href="c3ref/status.html">sqlite3_status</a>(<a href="c3ref/c_status_malloc_size.html">verb</a>, ¤t, &highwater, resetflag);
761
<a href="c3ref/status.html">sqlite3_status</a>(<a href="c3ref/c_status_malloc_count.html#sqlitestatusmemoryused">verb</a>, ¤t, &highwater, resetflag);
733
762
</pre></blockquote>
735
764
<p>The "verb" argument determines what statistic is accessed.
736
There are <a href="c3ref/c_status_malloc_size.html">various verbs</a> defined. The
765
There are <a href="c3ref/c_status_malloc_count.html#sqlitestatusmemoryused">various verbs</a> defined. The
737
766
list is expected to grow as the <a href="c3ref/status.html">sqlite3_status()</a> interface matures.
738
767
The current value the selected parameter is written into integer
739
768
"current" and the highest historical value
745
774
single <a href="c3ref/sqlite3.html">database connection</a>:</p>
747
776
<blockquote><pre>
748
<a href="c3ref/db_status.html">sqlite3_db_status</a>(db, <a href="c3ref/c_dbstatus_lookaside_used.html">verb</a>, ¤t, &highwater, resetflag);
777
<a href="c3ref/db_status.html">sqlite3_db_status</a>(db, <a href="c3ref/c_dbstatus options.html#sqlitedbstatuslookasideused">verb</a>, ¤t, &highwater, resetflag);
749
778
</pre></blockquote>
751
780
<p>This interface is similar except that it takes a pointer to
752
781
a <a href="c3ref/sqlite3.html">database connection</a> as its first argument and returns statistics about
753
782
that one object rather than about the entire SQLite library.
754
783
The <a href="c3ref/db_status.html">sqlite3_db_status()</a> interface currently only recognizes a
755
single verb <a href="c3ref/c_dbstatus_lookaside_used.html">SQLITE_DBSTATUS_LOOKASIDE_USED</a>, though additional verbs
784
single verb <a href="c3ref/c_dbstatus options.html#sqlitedbstatuslookasideused">SQLITE_DBSTATUS_LOOKASIDE_USED</a>, though additional verbs
756
785
may be added in the future.</p>
758
787
<p>The per-connection statistics do not use global variables and hence
759
788
do not require mutexes to update or access. Consequently the
760
789
per-connection statistics continue to function even if
761
<a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_MEMSTATUS</a> is turned off.</p>
790
<a href="c3ref/c_config_getmalloc.html#sqliteconfigmemstatus">SQLITE_CONFIG_MEMSTATUS</a> is turned off.</p>
763
792
<a name="heaplimit"></a>
764
793
<h3>3.6 Setting memory usage limits</h3>
766
<p>The <a href="c3ref/soft_heap_limit.html">sqlite3_soft_heap_limit()</a> interface can be used to set an
795
<p>The <a href="c3ref/soft_heap_limit64.html">sqlite3_soft_heap_limit64()</a> interface can be used to set an
767
796
upper bound on the total amount of outstanding memory that the
768
797
general-purpose memory allocator for SQLite will allow to be outstanding
769
798
at one time. If attempts are made to allocate more memory that specified
770
799
by the soft heap limit, then SQLite will first attempt to free cache
771
800
memory before continuing with the allocation request. The soft heap
772
801
limit mechanism only works if <a href="malloc.html#memstatus">memory statistics</a> are enabled and
773
803
if the SQLite library is compiled with the <a href="compile.html#enable_memory_management">SQLITE_ENABLE_MEMORY_MANAGEMENT</a>
774
804
compile-time option.</p>
856
886
<b>n</b> are known, or at least have known upper bounds, and if the
858
888
the <a href="malloc.html#memsys5">memsys5</a> memory allocator and is provided with <b>N</b> bytes of
859
available memory space using <a href="c3ref/c_config_getmalloc.html">SQLITE_CONFIG_HEAP</a>
889
available memory space using <a href="c3ref/c_config_getmalloc.html#sqliteconfigheap">SQLITE_CONFIG_HEAP</a>
860
890
then Robson proves that no memory allocation request will ever fail
861
891
within the application.
862
892
To put this another way, the application developer can select a value
884
914
and therefore <b>N</b>=<b>M</b>. In other words, the memory pool need
885
915
be no larger than the largest amount of memory in use at any given moment.</p>
887
<p>SQLite guarantees that no thread will ever use more than a single
888
scratch memory slot at one time. So if an application allocates as many
917
<p>SQLite guarantees that no thread will ever use more than two
918
scratch memory slots at one time. So if an application allocates twice as many
889
919
scratch memory slots as there are threads, and assuming the size of
890
920
each slot is large enough, there is never a chance of overflowing the
891
921
scratch memory allocator. An upper bound on the size of scratch memory
1004
1034
<a href="c3ref/sqlite3.html">database connection</a> and <a href="c3ref/stmt.html">prepared statement</a> objects the application
1005
1035
uses, and on the complexity of the <a href="c3ref/stmt.html">prepared statements</a>. For any
1006
1036
given application, these factors are normally fixed and can be
1007
determined experimentally using <a href="c3ref/c_status_malloc_size.html">SQLITE_STATUS_MEMORY_USED</a>.
1037
determined experimentally using <a href="c3ref/c_status_malloc_count.html#sqlitestatusmemoryused">SQLITE_STATUS_MEMORY_USED</a>.
1008
1038
A typical application might only use about 40KB of general-purpose
1009
1039
memory. This gives a value of <b>N</b> of around 100KB.</p>
1054
1084
these interfaces need to be prepared to modify their applications
1055
1085
to accommodate changes in the SQLite interface.</p>
1087
<p><b>Update:</b> As of SQLite version 3.7.0, all of these interfaces
1088
are considered stable</p>
1057
1090
<a name="summary"></a>
1058
1091
<h2>6.0 Summary Of Memory Allocator Interfaces</h2>
added
removed
www/malloc.html
