2
<!--Copyright 1997-2002 by Sleepycat Software, Inc.-->
3
<!--All rights reserved.-->
4
<!--See the file LICENSE for redistribution information.-->
7
<title>Berkeley DB Reference Guide: Mapping the terrain: theory and practice</title>
8
<meta name="description" content="Berkeley DB: An embedded database programmatic toolkit.">
9
<meta name="keywords" content="embedded,database,programmatic,toolkit,b+tree,btree,hash,hashing,transaction,transactions,locking,logging,access method,access methods,java,C,C++">
12
<table width="100%"><tr valign=top>
13
<td><h3><dl><dt>Berkeley DB Reference Guide:<dd>Introduction</dl></h3></td>
14
<td align=right><a href="../../ref/intro/data.html"><img src="../../images/prev.gif" alt="Prev"></a><a href="../../reftoc.html"><img src="../../images/ref.gif" alt="Ref"></a><a href="../../ref/intro/dbis.html"><img src="../../images/next.gif" alt="Next"></a>
17
<h1 align=center>Mapping the terrain: theory and practice</h1>
18
<p>The first step in selecting a database system is figuring out what the
19
choices are. Decades of research and real-world deployment have produced
20
countless systems. We need to organize them somehow to reduce the number
22
<p>One obvious way to group systems is to use the common labels that
23
vendors apply to them. The buzzwords here include "network,"
24
"relational," "object-oriented," and "embedded," with some
25
cross-fertilization like "object-relational" and "embedded network".
26
Understanding the buzzwords is important. Each has some grounding in
27
theory, but has also evolved into a practical label for categorizing
28
systems that work in a certain way.
29
<p>All database systems, regardless of the buzzwords that apply to them,
30
provide a few common services. All of them store data, for example.
31
We'll begin by exploring the common services that all systems provide,
32
and then examine the differences among the different kinds of systems.
33
<h3>Data access and data management</h3>
34
<p>Fundamentally, database systems provide two services.
35
<p>The first service is <i>data access</i>. Data access means adding
36
new data to the database (inserting), finding data of interest
37
(searching), changing data already stored (updating), and removing data
38
from the database (deleting). All databases provide these services. How
39
they work varies from category to category, and depends on the record
40
structure that the database supports.
41
<p>Each record in a database is a collection of values. For example, the
42
record for a Web site customer might include a name, email address,
43
shipping address, and payment information. Records are usually stored
44
in tables. Each table holds records of the same kind. For example, the
45
<b>customer</b> table at an e-commerce Web site might store the
46
customer records for every person who shopped at the site. Often,
47
database records have a different structure from the structures or
48
instances supported by the programming language in which an application
49
is written. As a result, working with records can mean:
51
<li>using database operations like searches and updates on records; and
52
<li>converting between programming language structures and database record
53
types in the application.
55
<p>The second service is <i>data management</i>. Data management is
56
more complicated than data access. Providing good data management
57
services is the hard part of building a database system. When you
58
choose a database system to use in an application you build, making sure
59
it supports the data management services you need is critical.
60
<p>Data management services include allowing multiple users to work on the
61
database simultaneously (concurrency), allowing multiple records to be
62
changed instantaneously (transactions), and surviving application and
63
system crashes (recovery). Different database systems offer different
64
data management services. Data management services are entirely
65
independent of the data access services listed above. For example,
66
nothing about relational database theory requires that the system
67
support transactions, but most commercial relational systems do.
68
<p>Concurrency means that multiple users can operate on the database at
69
the same time. Support for concurrency ranges from none (single-user
70
access only) to complete (many readers and writers working
72
<p>Transactions permit users to make multiple changes appear at once. For
73
example, a transfer of funds between bank accounts needs to be a
74
transaction because the balance in one account is reduced and the
75
balance in the other increases. If the reduction happened before the
76
increase, than a poorly-timed system crash could leave the customer
77
poorer; if the bank used the opposite order, then the same system crash
78
could make the customer richer. Obviously, both the customer and the
79
bank are best served if both operations happen at the same instant.
80
<p>Transactions have well-defined properties in database systems. They are
81
<i>atomic</i>, so that the changes happen all at once or not at all.
82
They are <i>consistent</i>, so that the database is in a legal state
83
when the transaction begins and when it ends. They are typically
84
<i>isolated</i>, which means that any other users in the database
85
cannot interfere with them while they are in progress. And they are
86
<i>durable</i>, so that if the system or application crashes after
87
a transaction finishes, the changes are not lost. Together, the
88
properties of <i>atomicity</i>, <i>consistency</i>,
89
<i>isolation</i>, and <i>durability</i> are known as the ACID
91
<p>As is the case for concurrency, support for transactions varies among
92
databases. Some offer atomicity without making guarantees about
93
durability. Some ignore isolatability, especially in single-user
94
systems; there's no need to isolate other users from the effects of
95
changes when there are no other users.
96
<p>Another important data management service is recovery. Strictly
97
speaking, recovery is a procedure that the system carries out when it
98
starts up. The purpose of recovery is to guarantee that the database is
99
complete and usable. This is most important after a system or
100
application crash, when the database may have been damaged. The recovery
101
process guarantees that the internal structure of the database is good.
102
Recovery usually means that any completed transactions are checked, and
103
any lost changes are reapplied to the database. At the end of the
104
recovery process, applications can use the database as if there had been
105
no interruption in service.
106
<p>Finally, there are a number of data management services that permit
107
copying of data. For example, most database systems are able to import
108
data from other sources, and to export it for use elsewhere. Also, most
109
systems provide some way to back up databases and to restore in the
110
event of a system failure that damages the database. Many commercial
111
systems allow <i>hot backups</i>, so that users can back up
112
databases while they are in use. Many applications must run without
113
interruption, and cannot be shut down for backups.
114
<p>A particular database system may provide other data management services.
115
Some provide browsers that show database structure and contents. Some
116
include tools that enforce data integrity rules, such as the rule that
117
no employee can have a negative salary. These data management services
118
are not common to all systems, however. Concurrency, recovery, and
119
transactions are the data management services that most database vendors
121
<p>Deciding what kind of database to use means understanding the data
122
access and data management services that your application needs. Berkeley DB
123
is an embedded database that supports fairly simple data access with a
124
rich set of data management services. To highlight its strengths and
125
weaknesses, we can compare it to other database system categories.
126
<h3>Relational databases</h3>
127
<p>Relational databases are probably the best-known database variant,
128
because of the success of companies like Oracle. Relational databases
129
are based on the mathematical field of set theory. The term "relation"
130
is really just a synonym for "set" -- a relation is just a set of
131
records or, in our terminology, a table. One of the main innovations in
132
early relational systems was to insulate the programmer from the
133
physical organization of the database. Rather than walking through
134
arrays of records or traversing pointers, programmers make statements
135
about tables in a high-level language, and the system executes those
137
<p>Relational databases operate on <i>tuples</i>, or records, composed
138
of values of several different data types, including integers, character
139
strings, and others. Operations include searching for records whose
140
values satisfy some criteria, updating records, and so on.
141
<p>Virtually all relational databases use the Structured Query Language,
142
or SQL. This language permits people and computer programs to work with
143
the database by writing simple statements. The database engine reads
144
those statements and determines how to satisfy them on the tables in
146
<p>SQL is the main practical advantage of relational database systems.
147
Rather than writing a computer program to find records of interest, the
148
relational system user can just type a query in a simple syntax, and
149
let the engine do the work. This gives users enormous flexibility; they
150
do not need to decide in advance what kind of searches they want to do,
151
and they do not need expensive programmers to find the data they need.
152
Learning SQL requires some effort, but it's much simpler than a
153
full-blown high-level programming language for most purposes. And there
154
are a lot of programmers who have already learned SQL.
155
<h3>Object-oriented databases</h3>
156
<p>Object-oriented databases are less common than relational systems, but
157
are still fairly widespread. Most object-oriented databases were
158
originally conceived as persistent storage systems closely wedded to
159
particular high-level programming languages like C++. With the spread
160
of Java, most now support more than one programming language, but
161
object-oriented database systems fundamentally provide the same class
162
and method abstractions as do object-oriented programming languages.
163
<p>Many object-oriented systems allow applications to operate on objects
164
uniformly, whether they are in memory or on disk. These systems create
165
the illusion that all objects are in memory all the time. The advantage
166
to object-oriented programmers who simply want object storage and
167
retrieval is clear. They need never be aware of whether an object is in
168
memory or not. The application simply uses objects, and the database
169
system moves them between disk and memory transparently. All of the
170
operations on an object, and all its behavior, are determined by the
171
programming language.
172
<p>Object-oriented databases aren't nearly as widely deployed as relational
173
systems. In order to attract developers who understand relational
174
systems, many of the object-oriented systems have added support for
175
query languages very much like SQL. In practice, though, object-oriented
176
databases are mostly used for persistent storage of objects in C++ and
178
<h3>Network databases</h3>
179
<p>The "network model" is a fairly old technique for managing and
180
navigating application data. Network databases are designed to make
181
pointer traversal very fast. Every record stored in a network database
182
is allowed to contain pointers to other records. These pointers are
183
generally physical addresses, so fetching the record to which it refers
184
just means reading it from disk by its disk address.
185
<p>Network database systems generally permit records to contain integers,
186
floating point numbers, and character strings, as well as references to
187
other records. An application can search for records of interest. After
188
retrieving a record, the application can fetch any record to which it
190
<p>Pointer traversal is fast because most network systems use physical disk
191
addresses as pointers. When the application wants to fetch a record,
192
the database system uses the address to fetch exactly the right string
193
of bytes from the disk. This requires only a single disk access in all
194
cases. Other systems, by contrast, often must do more than one disk read
195
to find a particular record.
196
<p>The key advantage of the network model is also its main drawback. The
197
fact that pointer traversal is so fast means that applications that do
198
it will run well. On the other hand, storing pointers all over the
199
database makes it very hard to reorganize the database. In effect, once
200
you store a pointer to a record, it is difficult to move that record
201
elsewhere. Some network databases handle this by leaving forwarding
202
pointers behind, but this defeats the speed advantage of doing a single
203
disk access in the first place. Other network databases find, and fix,
204
all the pointers to a record when it moves, but this makes
205
reorganization very expensive. Reorganization is often necessary in
206
databases, since adding and deleting records over time will consume
207
space that cannot be reclaimed without reorganizing. Without periodic
208
reorganization to compact network databases, they can end up with a
209
considerable amount of wasted space.
210
<h3>Clients and servers</h3>
211
<p>Database vendors have two choices for system architecture. They can
212
build a server to which remote clients connect, and do all the database
213
management inside the server. Alternatively, they can provide a module
214
that links directly into the application, and does all database
215
management locally. In either case, the application developer needs
216
some way of communicating with the database (generally, an Application
217
Programming Interface (API) that does work in the process or that
218
communicates with a server to get work done).
219
<p>Almost all commercial database products are implemented as servers, and
220
applications connect to them as clients. Servers have several features
221
that make them attractive.
222
<p>First, because all of the data is managed by a separate process, and
223
possibly on a separate machine, it's easy to isolate the database server
224
from bugs and crashes in the application.
225
<p>Second, because some database products (particularly relational engines)
226
are quite large, splitting them off as separate server processes keeps
227
applications small, which uses less disk space and memory. Relational
228
engines include code to parse SQL statements, to analyze them and
229
produce plans for execution, to optimize the plans, and to execute
231
<p>Finally, by storing all the data in one place and managing it with a
232
single server, it's easier for organizations to back up, protect, and
233
set policies on their databases. The enterprise databases for large
234
companies often have several full-time administrators caring for them,
235
making certain that applications run quickly, granting and denying
236
access to users, and making backups.
237
<p>However, centralized administration can be a disadvantage in some cases.
238
In particular, if a programmer wants to build an application that uses
239
a database for storage of important information, then shipping and
240
supporting the application is much harder. The end user needs to install
241
and administer a separate database server, and the programmer must
242
support not just one product, but two. Adding a server process to the
243
application creates new opportunity for installation mistakes and
245
<table width="100%"><tr><td><br></td><td align=right><a href="../../ref/intro/data.html"><img src="../../images/prev.gif" alt="Prev"></a><a href="../../reftoc.html"><img src="../../images/ref.gif" alt="Ref"></a><a href="../../ref/intro/dbis.html"><img src="../../images/next.gif" alt="Next"></a>
247
<p><font size=1><a href="http://www.sleepycat.com">Copyright Sleepycat Software</a></font>
added
removed
libdb/docs/ref/intro/terrain.html
