« back to all changes in this revision
Viewing changes to developers/index.html
- browse files at revision 1
- compare with another revision
- download diff
- download tarball
- view history from revision 1
- Committer: Bazaar Package Importer
- Author(s): John Goerzen
- Date: 2006-08-15 09:44:08 UTC
- Revision ID: james.westby@ubuntu.com-20060815094408-1kvvfls2hs3d9uw8
Tags: upstream-1.38.11.1
Import upstream version 1.38.11.1
- files added:
- autoconf
- bacula-web
- bacula-web/bacula-web
- developers
- developers/developers
- home-page
- home-page/images
- home-page/images/manual
- home-page/inc
- home-page/pages
- images
- images/hires
- manual
- manual-de
- manual-fr
- manual/bacula
- techlogs
- techlogs/2001
- techlogs/2002
- techlogs/2003
- techlogs/2004
- techlogs/2005
added
removed
developers/index.html
1
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
2
3
<!--Converted with LaTeX2HTML 2002-2-1 (1.70)
4
original version by: Nikos Drakos, CBLU, University of Leeds
5
* revised and updated by: Marcus Hennecke, Ross Moore, Herb Swan
6
* with significant contributions from:
7
Jens Lippmann, Marek Rouchal, Martin Wilck and others -->
8
<HTML>
9
<HEAD>
10
<TITLE>It comes in the night and sucks the essence from your computers. </TITLE>
11
<META NAME="description" CONTENT="It comes in the night and sucks the essence from your computers. ">
12
<META NAME="keywords" CONTENT="developers">
13
<META NAME="resource-type" CONTENT="document">
14
<META NAME="distribution" CONTENT="global">
15
16
<META NAME="Generator" CONTENT="LaTeX2HTML v2002-2-1">
17
<META HTTP-EQUIV="Content-Style-Type" CONTENT="text/css">
18
19
<LINK REL="STYLESHEET" HREF="developers.css">
20
21
</HEAD>
22
23
<BODY >
24
<!--Navigation Panel-->
25
<IMG WIDTH="81" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next_inactive"
26
SRC="file:/usr/share/latex2html/icons/nx_grp_g.png">
27
<IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up"
28
SRC="file:/usr/share/latex2html/icons/up_g.png">
29
<IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous"
30
SRC="file:/usr/share/latex2html/icons/prev_g.png">
31
<BR>
32
<BR>
33
<BR>
34
<!--End of Navigation Panel-->
35
36
<P>
37
38
<P>
39
40
<P>
41
<H1 ALIGN="CENTER"><IMG
42
WIDTH="458" HEIGHT="99" ALIGN="BOTTOM" BORDER="0"
43
SRC="bacula-logo.png"
44
ALT="\includegraphics{./bacula-logo.eps}">
45
<BR><P><P>
46
<BR>
47
<DIV ALIGN="CENTER">
48
<FONT SIZE="+1">It comes in the night and sucks
49
the essence from your computers.
50
51
</FONT></DIV></H1>
52
<DIV>
53
54
<P ALIGN="CENTER"><STRONG>Kern Sibbald</STRONG></P>
55
<P ALIGN="CENTER"><STRONG>
56
<BR>
57
<BR>
58
<BR>
59
<BR>
60
<BR>
61
<BR>
62
<BR>
63
1 July 2006
64
<BR>
65
This manual documents Bacula version 1.38.11 (01 July 2006)
66
67
<BR>
68
<BR>
69
<BR>
70
<BR>
71
Copyright ©1999-2006, Kern Sibbald
72
<BR> <BR>
73
<BR>
74
<BR>
75
Permission is granted to copy, distribute and/or modify this document under the terms of the
76
<BR>
77
GNU Free Documentation License, Version 1.2 published by the Free Software Foundation;
78
<BR>
79
with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
80
<BR>
81
A copy of the license is included in the section entitled "GNU Free Documentation License".
82
</STRONG></P>
83
</DIV>
84
85
<P>
86
87
<BR>
88
89
<H2><A NAME="SECTION00010000000000000000">
90
Contents</A>
91
</H2>
92
<!--Table of Contents-->
93
94
<UL>
95
<LI><A NAME="tex2html190"
96
HREF="developers.html#SECTION00020000000000000000">List of Figures</A>
97
<LI><A NAME="tex2html191"
98
HREF="developers.html#SECTION00030000000000000000">List of Tables</A>
99
<LI><A NAME="tex2html192"
100
HREF="developers.html#SECTION00040000000000000000">Bacula Developer Notes</A>
101
<UL>
102
<LI><A NAME="tex2html193"
103
HREF="developers.html#SECTION00041000000000000000">General</A>
104
<LI><A NAME="tex2html194"
105
HREF="developers.html#SECTION00042000000000000000">Basic CVS Usage</A>
106
<LI><A NAME="tex2html195"
107
HREF="developers.html#SECTION00043000000000000000">The Development Cycle</A>
108
<LI><A NAME="tex2html196"
109
HREF="developers.html#SECTION00044000000000000000">Developing Bacula</A>
110
</UL>
111
<BR>
112
<LI><A NAME="tex2html197"
113
HREF="developers.html#SECTION00050000000000000000">Platform Support</A>
114
<UL>
115
<LI><A NAME="tex2html198"
116
HREF="developers.html#SECTION00051000000000000000">General</A>
117
<LI><A NAME="tex2html199"
118
HREF="developers.html#SECTION00052000000000000000">Requirements to become a Supported Platform</A>
119
</UL>
120
<BR>
121
<LI><A NAME="tex2html200"
122
HREF="developers.html#SECTION00060000000000000000">Daemon Protocol</A>
123
<UL>
124
<LI><A NAME="tex2html201"
125
HREF="developers.html#SECTION00061000000000000000">General</A>
126
<LI><A NAME="tex2html202"
127
HREF="developers.html#SECTION00062000000000000000">Low Level Network Protocol</A>
128
<LI><A NAME="tex2html203"
129
HREF="developers.html#SECTION00063000000000000000">General Daemon Protocol</A>
130
<LI><A NAME="tex2html204"
131
HREF="developers.html#SECTION00064000000000000000">The Protocol Used Between the Director and the Storage Daemon</A>
132
<LI><A NAME="tex2html205"
133
HREF="developers.html#SECTION00065000000000000000">The Protocol Used Between the Director and the File Daemon</A>
134
<LI><A NAME="tex2html206"
135
HREF="developers.html#SECTION00066000000000000000">The Save Protocol Between the File Daemon and the Storage Daemon</A>
136
</UL>
137
<BR>
138
<LI><A NAME="tex2html207"
139
HREF="developers.html#SECTION00070000000000000000">Director Services Daemon</A>
140
<LI><A NAME="tex2html208"
141
HREF="developers.html#SECTION00080000000000000000">File Services Daemon</A>
142
<UL>
143
<LI><A NAME="tex2html209"
144
HREF="developers.html#SECTION00081000000000000000">Commands Received from the Director for a Backup</A>
145
<LI><A NAME="tex2html210"
146
HREF="developers.html#SECTION00082000000000000000">Commands Received from the Director for a Restore</A>
147
</UL>
148
<BR>
149
<LI><A NAME="tex2html211"
150
HREF="developers.html#SECTION00090000000000000000">Storage Daemon Design</A>
151
<UL>
152
<LI><A NAME="tex2html212"
153
HREF="developers.html#SECTION00091000000000000000">SD Design Introduction</A>
154
<LI><A NAME="tex2html213"
155
HREF="developers.html#SECTION00092000000000000000">SD Development Outline</A>
156
<LI><A NAME="tex2html214"
157
HREF="developers.html#SECTION00093000000000000000">SD Connections and Sessions</A>
158
<LI><A NAME="tex2html215"
159
HREF="developers.html#SECTION00094000000000000000">SD Data Structures</A>
160
</UL>
161
<BR>
162
<LI><A NAME="tex2html216"
163
HREF="developers.html#SECTION000100000000000000000">Catalog Services</A>
164
<UL>
165
<LI><A NAME="tex2html217"
166
HREF="developers.html#SECTION000101000000000000000">General</A>
167
<LI><A NAME="tex2html218"
168
HREF="developers.html#SECTION000102000000000000000">Sequence of Creation of Records for a Save Job</A>
169
<LI><A NAME="tex2html219"
170
HREF="developers.html#SECTION000103000000000000000">Database Tables</A>
171
</UL>
172
<BR>
173
<LI><A NAME="tex2html220"
174
HREF="developers.html#SECTION000110000000000000000">Storage Media Output Format</A>
175
<UL>
176
<LI><A NAME="tex2html221"
177
HREF="developers.html#SECTION000111000000000000000">General</A>
178
<LI><A NAME="tex2html222"
179
HREF="developers.html#SECTION000112000000000000000">Definitions</A>
180
<LI><A NAME="tex2html223"
181
HREF="developers.html#SECTION000113000000000000000">Storage Daemon File Output Format</A>
182
<LI><A NAME="tex2html224"
183
HREF="developers.html#SECTION000114000000000000000">Overall Format</A>
184
<LI><A NAME="tex2html225"
185
HREF="developers.html#SECTION000115000000000000000">Serialization</A>
186
<LI><A NAME="tex2html226"
187
HREF="developers.html#SECTION000116000000000000000">Block Header</A>
188
<LI><A NAME="tex2html227"
189
HREF="developers.html#SECTION000117000000000000000">Record Header</A>
190
<LI><A NAME="tex2html228"
191
HREF="developers.html#SECTION000118000000000000000">Version BB02 Block Header</A>
192
<LI><A NAME="tex2html229"
193
HREF="developers.html#SECTION000119000000000000000">Version 2 Record Header</A>
194
<LI><A NAME="tex2html230"
195
HREF="developers.html#SECTION0001110000000000000000">Volume Label Format</A>
196
<LI><A NAME="tex2html231"
197
HREF="developers.html#SECTION0001111000000000000000">Session Label</A>
198
<LI><A NAME="tex2html232"
199
HREF="developers.html#SECTION0001112000000000000000">Overall Storage Format</A>
200
<LI><A NAME="tex2html233"
201
HREF="developers.html#SECTION0001113000000000000000">Unix File Attributes</A>
202
<LI><A NAME="tex2html234"
203
HREF="developers.html#SECTION0001114000000000000000">Old Depreciated Tape Format</A>
204
</UL>
205
<BR>
206
<LI><A NAME="tex2html235"
207
HREF="developers.html#SECTION000120000000000000000">Bacula Porting Notes</A>
208
<UL>
209
<LI><A NAME="tex2html236"
210
HREF="developers.html#SECTION000121000000000000000">Porting Requirements</A>
211
<LI><A NAME="tex2html237"
212
HREF="developers.html#SECTION000122000000000000000">Steps to Take for Porting</A>
213
</UL>
214
<BR>
215
<LI><A NAME="tex2html238"
216
HREF="developers.html#SECTION000130000000000000000">Implementing a Bacula GUI Interface</A>
217
<UL>
218
<LI><A NAME="tex2html239"
219
HREF="developers.html#SECTION000131000000000000000">General</A>
220
</UL>
221
<BR>
222
<LI><A NAME="tex2html240"
223
HREF="developers.html#SECTION000140000000000000000">TLS</A>
224
<UL>
225
<LI><A NAME="tex2html241"
226
HREF="developers.html#SECTION000141000000000000000">Introduction to TLS</A>
227
<LI><A NAME="tex2html242"
228
HREF="developers.html#SECTION000142000000000000000">New Configuration Directives</A>
229
<LI><A NAME="tex2html243"
230
HREF="developers.html#SECTION000143000000000000000">TLS API Implementation</A>
231
<LI><A NAME="tex2html244"
232
HREF="developers.html#SECTION000144000000000000000">Bnet API Changes</A>
233
<LI><A NAME="tex2html245"
234
HREF="developers.html#SECTION000145000000000000000">Authentication Negotiation</A>
235
</UL>
236
<BR>
237
<LI><A NAME="tex2html246"
238
HREF="developers.html#SECTION000150000000000000000">Bacula Regression Testing</A>
239
<UL>
240
<LI><A NAME="tex2html247"
241
HREF="developers.html#SECTION000151000000000000000">General</A>
242
<LI><A NAME="tex2html248"
243
HREF="developers.html#SECTION000152000000000000000">Running the Regression Script</A>
244
<LI><A NAME="tex2html249"
245
HREF="developers.html#SECTION000153000000000000000">Writing a Regression Test</A>
246
</UL>
247
<BR>
248
<LI><A NAME="tex2html250"
249
HREF="developers.html#SECTION000160000000000000000">Bacula MD5 Algorithm</A>
250
<UL>
251
<LI><A NAME="tex2html251"
252
HREF="developers.html#SECTION000161000000000000000">Command Line Message Digest Utility </A>
253
<LI><A NAME="tex2html252"
254
HREF="developers.html#SECTION000162000000000000000">Download md5.zip
255
(Zipped
256
archive)</A>
257
</UL>
258
<BR>
259
<LI><A NAME="tex2html253"
260
HREF="developers.html#SECTION000170000000000000000">Bacula Memory Management</A>
261
<UL>
262
<LI><A NAME="tex2html254"
263
HREF="developers.html#SECTION000171000000000000000">General</A>
264
</UL>
265
<BR>
266
<LI><A NAME="tex2html255"
267
HREF="developers.html#SECTION000180000000000000000">TCP/IP Network Protocol</A>
268
<UL>
269
<LI><A NAME="tex2html256"
270
HREF="developers.html#SECTION000181000000000000000">General</A>
271
<LI><A NAME="tex2html257"
272
HREF="developers.html#SECTION000182000000000000000">bnet and Threads</A>
273
<LI><A NAME="tex2html258"
274
HREF="developers.html#SECTION000183000000000000000">bnet_open</A>
275
<LI><A NAME="tex2html259"
276
HREF="developers.html#SECTION000184000000000000000">bnet_send</A>
277
<LI><A NAME="tex2html260"
278
HREF="developers.html#SECTION000185000000000000000">bnet_fsend</A>
279
<LI><A NAME="tex2html261"
280
HREF="developers.html#SECTION000186000000000000000">Additional Error information</A>
281
<LI><A NAME="tex2html262"
282
HREF="developers.html#SECTION000187000000000000000">bnet_recv</A>
283
<LI><A NAME="tex2html263"
284
HREF="developers.html#SECTION000188000000000000000">bnet_sig</A>
285
<LI><A NAME="tex2html264"
286
HREF="developers.html#SECTION000189000000000000000">bnet_strerror</A>
287
<LI><A NAME="tex2html265"
288
HREF="developers.html#SECTION0001810000000000000000">bnet_close</A>
289
<LI><A NAME="tex2html266"
290
HREF="developers.html#SECTION0001811000000000000000">Becoming a Server</A>
291
<LI><A NAME="tex2html267"
292
HREF="developers.html#SECTION0001812000000000000000">Higher Level Conventions</A>
293
</UL>
294
<BR>
295
<LI><A NAME="tex2html268"
296
HREF="developers.html#SECTION000190000000000000000">Smart Memory Allocation With Orphaned Buffer Detection </A>
297
<UL>
298
<LI><A NAME="tex2html269"
299
HREF="developers.html#SECTION000191000000000000000">Download smartall.zip
300
(Zipped archive)</A>
301
</UL>
302
<BR>
303
<LI><A NAME="tex2html270"
304
HREF="developers.html#SECTION000200000000000000000">GNU Free Documentation License</A>
305
<LI><A NAME="tex2html271"
306
HREF="developers.html#SECTION000210000000000000000">General Index</A>
307
<LI><A NAME="tex2html272"
308
HREF="developers.html#SECTION000220000000000000000">About this document ...</A>
309
</UL>
310
<!--End of Table of Contents-->
311
<BR>
312
313
<H2><A NAME="SECTION00020000000000000000">
314
List of Figures</A>
315
</H2><OL>
316
<LI><A HREF="developers.html#4963">Smart Memory Allocation with Orphaned BufferDetection</A>
317
</OL>
318
<BR>
319
320
<H2><A NAME="SECTION00030000000000000000">
321
List of Tables</A>
322
</H2><OL>
323
<LI><A HREF="developers.html#361">Message Error Code Classes</A>
324
<LI><A HREF="developers.html#1676">Filename Table Layout</A>
325
<LI><A HREF="developers.html#1701">Path Table Layout</A>
326
<LI><A HREF="developers.html#1726">File Table Layout</A>
327
<LI><A HREF="developers.html#1770">Job Table Layout</A>
328
<LI><A HREF="developers.html#1855">Job Types</A>
329
<LI><A HREF="developers.html#1879">Job Statuses</A>
330
<LI><A HREF="developers.html#1929">File Sets Table Layout</A>
331
<LI><A HREF="developers.html#1960">JobMedia Table Layout</A>
332
<LI><A HREF="developers.html#2009">Media Table Layout</A>
333
<LI><A HREF="developers.html#2097">Pool Table Layout</A>
334
<LI><A HREF="developers.html#2164">Client Table Layout</A>
335
<LI><A HREF="developers.html#2201">Unsaved Files Table Layout</A>
336
<LI><A HREF="developers.html#2232">Counter Table Layout</A>
337
<LI><A HREF="developers.html#2266">Version Table Layout</A>
338
<LI><A HREF="developers.html#2288">Base Files Table Layout</A>
339
<LI><A HREF="developers.html#2843">File Attributes</A>
340
</OL>
341
342
<P>
343
344
<P>
345
346
<H1><A NAME="SECTION00040000000000000000"></A>
347
<A NAME="_ChapterStart10"></A><A NAME="60"></A>
348
<A NAME="61"></A>
349
<BR>
350
Bacula Developer Notes
351
</H1>
352
<A NAME="64"></A>
353
354
<P>
355
356
<H2><A NAME="SECTION00041000000000000000"></A>
357
<A NAME="66"></A>
358
<BR>
359
General
360
</H2>
361
<A NAME="69"></A>
362
363
<P>
364
This document is intended mostly for developers and describes the the general
365
framework of making Bacula source changes.
366
367
<P>
368
369
<H3><A NAME="SECTION00041100000000000000"></A>
370
<A NAME="71"></A>
371
<BR>
372
Contributions
373
</H3>
374
<A NAME="74"></A>
375
376
<P>
377
Contributions from programmers are broken into two groups. The first are
378
contributions that are aids and not essential to Bacula. In general, these
379
will be scripts or will go into and examples or contributions directory.
380
For these kinds of non-essential contributions there is no obligation to do
381
a copyright assignment as described below. However, a copyright assignment
382
would still be appreciated.
383
384
<P>
385
The second class of contributions are those which will be integrated with
386
Bacula and become an essential part. Within this class of contributions, there
387
are two hurdles to surmount. One is getting your patch accepted, and two is
388
dealing with copyright issues. The following text describes some of the
389
requirements for such code.
390
391
<P>
392
393
<H3><A NAME="SECTION00041200000000000000"></A>
394
<A NAME="76"></A>
395
<BR>
396
Patches
397
</H3>
398
<A NAME="79"></A>
399
400
<P>
401
Subject to the copyright assignment described below, your patches should be
402
sent in <B>diff -u</B> format relative to the current contents of the Source
403
Forge CVS, which is the easiest to understand and integrate.
404
If you have checked out the source with CVS, you can get a diff using:
405
406
<P>
407
<PRE>
408
cvs diff -u > change.patch
409
</PRE>
410
411
<P>
412
If you plan on doing significant development work over a period of time,
413
after having your first patch reviewed and approved, you will be eligible
414
for having developer CVS access so that you can commit your changes
415
directly to the CVS repository. To do so, you will need a userid on Source
416
Forge.
417
418
<P>
419
420
<H3><A NAME="SECTION00041300000000000000"></A>
421
<A NAME="84"></A>
422
<BR>
423
Copyrights
424
</H3>
425
<A NAME="87"></A>
426
427
<P>
428
To avoid future problems concerning changing licensing or copyrights, all
429
code contributions more than a hand full of lines must be in the Public
430
Domain or have the copyright assigned to Kern Sibbald as in the current
431
code. Note, prior to November 2004, the code was copyrighted by Kern
432
Sibbald and John Walker.
433
434
<P>
435
Your name should be clearly indicated as the author of the code, and you
436
must be extremely careful not to violate any copyrights or use other
437
people's code without acknowledging it. The purpose of this requirement is
438
to avoid future copyright, patent, or intellectual property problems. To
439
understand on possible source of future problems, please examine the
440
difficulties Mozilla is (was?) having finding previous contributors at
441
<A NAME="tex2html1"
442
HREF="http://www.mozilla.org/MPL/missing.html">http://www.mozilla.org/MPL/missing.html</A>. The other important issue is to
443
avoid copyright, patent, or intellectual property violations as are currently
444
(May 2003) being claimed by SCO against IBM.
445
446
<P>
447
Although the copyright will be held by Kern, each developer is expected to
448
indicate that he wrote and/or modified a particular module (or file) and
449
any other sources. The copyright assignment may seem a bit unusual, but in
450
reality, it is not. Most large projects require this. In fact, the
451
paperwork associated with making contributions to the Free Software
452
Foundation, was for me unsurmountable, so hopefully the rather
453
simplified procedure we have will not create any difficulties for
454
you.
455
456
<P>
457
If you have any doubts about this, please don't hesitate to ask. The
458
objective is to assure the long term servival of the Bacula project. There
459
is no consideration of personal gain in this request. Our (John and my)
460
track records with Autodesk are easily available; early
461
programmers/founders/contributors and later employees had substantial
462
shares of the company, and no one founder had a controlling part of the
463
company. Even though Microsoft created many millionaires among early
464
employees, the politics of Autodesk (during our time at the helm) is in
465
stark contrast to Microsoft where the majority of the company is still
466
tightly held among a few.
467
468
<P>
469
Items not needing a copyright assignment are: most small changes,
470
enhancements, or bug fixes of 5-10 lines of code, and documentation.
471
472
<P>
473
474
<H3><A NAME="SECTION00041400000000000000"></A>
475
<A NAME="91"></A>
476
<A NAME="92"></A>
477
<BR>
478
Copyright Assignment
479
</H3>
480
<A NAME="95"></A>
481
482
<P>
483
Since this is not a commercial enterprise, and I prefer to believe in
484
everyone's good faith, developers can assign the copyright by explicitly
485
acknowledging that they do so in their first submission. This is
486
sufficient if the developer is independent, or an employee of a
487
not-for-profit organization or a university. Any developer who wants to
488
contribute and is employed by a company must get a copyright assignment
489
from his employer. This is to avoid misunderstandings between the
490
employee, the company, and the Bacula project. A good number of
491
companies have already followed this procedure.
492
493
<P>
494
495
<H3><A NAME="SECTION00041500000000000000"></A>
496
<A NAME="97"></A>
497
<A NAME="98"></A>
498
<BR>
499
Corporate Assignment Statement
500
</H3>
501
<A NAME="101"></A>
502
503
<P>
504
The following statement must be filled out by the employer, signed, and mailed
505
to my address (please ask me for my address and I will email it -- I'd prefer
506
not to include it here).
507
508
<P>
509
<PRE>
510
Copyright release and transfer statement.
511
<On company letter head>
512
513
To: Kern Sibbald
514
Concerning: Copyright release and transfer
515
516
<Company, Inc> is hereby agrees that <names-of-developers> and
517
other employees of <Company, Inc> are authorized to develop
518
code for and contribute code to the Bacula project for an
519
undetermined period of time. The copyright as well as all
520
other rights to and interests in such contributed code are
521
hereby transferred to Kern Sibbald.
522
523
Signed in <City, Country> on <Date>:
524
525
<Name of Person>, <Position in Company, Inc>
526
</PRE>
527
<P>
528
This release/transfer statement must be sent to:
529
Kern Sibbald
530
Address-to-be-given
531
532
<P>
533
If you wish to retain the full rights to use the software you
534
have contributed in different projects, this is not a problem. Just
535
request a perpetual non-exclusive license before sending in your
536
copyright assignment.
537
538
<P>
539
540
<H2><A NAME="SECTION00042000000000000000"></A>
541
<A NAME="105"></A>
542
<A NAME="106"></A>
543
<BR>
544
Basic CVS Usage
545
</H2>
546
<A NAME="109"></A>
547
548
<P>
549
The Bacula CVS is kept at Source Forge. If you are not a developer,
550
you only be able to access the public CVS, which runs about 6 hours behind
551
the developer's CVS. If you are a developer, then you will have immediate
552
access to "the" CVS and any changes ("commit") that you make will be
553
immediately seen by other developers.
554
555
<P>
556
For developer access to the CVS, go to the Bacula page on Source Forge
557
and click on the CVS menu item. Then follow the instructions for
558
installing your SSH public key. It may take a few hours until the
559
key is actually installed, and then you will have access to the CVS.
560
561
<P>
562
The Bacula CVS is divided into the following CVS "projects" or "modules".
563
564
<P>
565
<PRE>
566
bacula (Bacula source code)
567
docs (Bacula documentation)
568
gui (Some of the GUI programs that do not use
569
the Bacula core code)
570
rescue (Bacula CDROM rescue code)
571
regress (Bacula regression scripts)
572
ryol (Roll your own Linux -- incomplete project)
573
</PRE>
574
575
<P>
576
Most of you will want either the Bacula source (bacula) and/or the
577
documents (docs).
578
579
<P>
580
To get the source for a project, you must check it out ("checkout"), which
581
you do usually once.
582
583
<P>
584
585
<H3><A NAME="SECTION00042100000000000000"></A>
586
<A NAME="113"></A>
587
<BR>
588
Public CVS Access
589
</H3>
590
<A NAME="116"></A>
591
592
<P>
593
The first time you checkout the code for each project, you will need to
594
tell the cvs program where the CVS repository is. The procedure for
595
checking out code from the public CVS is:
596
597
<P>
598
<PRE>
599
cvs -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/bacula login
600
</PRE>
601
Then when it prompts for the password for <B>anonymous</B>, simply
602
press the Enter key. The above command is necessary only once
603
the very first time you login. Then enter the following command:
604
605
<P>
606
<PRE>
607
cvs -z3 -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/bacula co -P bacula
608
</PRE>
609
610
<P>
611
The above will place the contents of the bacula module in a directory
612
named <B>bacula</B> in the current directory. This data will come
613
from the public CVS, which typically runs 6 hours to a day behind the
614
developer's CVS. Once you have created a copy of the CVS, you can use
615
the commands listed below under the title CVS Usage.
616
617
<P>
618
619
<H3><A NAME="SECTION00042200000000000000"></A>
620
<A NAME="124"></A>
621
<BR>
622
Developer CVS Access
623
</H3>
624
<A NAME="127"></A>
625
626
<P>
627
If you are registered as a Bacula developer (contact Kern about this),
628
you may access the developer's CVS using:
629
630
<P>
631
<PRE>
632
export CVS_RSH=ssh
633
export CVSROOT=:ext:<nnnn>@cvs.bacula.sourceforge.net:/cvsroot/bacula
634
</PRE>
635
636
<P>
637
where you replace <nnnn> by your Source Forge user name.
638
639
<P>
640
Then do:
641
642
<P>
643
<PRE>
644
cvs -z3 checkout -d <directory> bacula
645
</PRE>
646
647
<P>
648
where you replace <directory> by the name of the directory you want
649
to contain the Bacula source code. If you want the docs, replace the
650
word "bacula" on the above line by "docs" and be sure to put it in
651
a different directory. The -z3 just tells CVS to use compression during
652
the transmission, which makes things go faster. There is no need to
653
do the anonymous login as is the case for non-developers.
654
655
<P>
656
The above command should generate output that looks a bit like the
657
following:
658
659
<P>
660
<PRE>
661
cvs checkout: Updating bacula
662
U bacula/Makefile
663
U bacula/bar.c
664
U bacula/foo.c
665
U bacula/main.c
666
...
667
</PRE>
668
669
<P>
670
671
<H3><A NAME="SECTION00042300000000000000"></A>
672
<A NAME="139"></A>
673
<BR>
674
CVS Usage
675
</H3>
676
<A NAME="142"></A>
677
678
<P>
679
The commands that follow with the exception of the <B>commit</B>
680
work the same whether you are accessing the public CVS or
681
the developer's CVS.
682
683
<P>
684
Let's assume you used the name "bacula" for the directory, so your
685
command was:
686
687
<P>
688
<PRE>
689
cvs -z3 checkout -d bacula bacula
690
</PRE>
691
692
<P>
693
When the command is done, simply do:
694
695
<P>
696
<PRE>
697
cd bacula
698
</PRE>
699
700
<P>
701
and then build Bacula. You will notice a lot of extra CVS directories
702
in the source code. Don't ever change or delete them, or you will mess up
703
your copy of the project. Also, do not rename or delete any of the files
704
in your copy of the repository or you may have problems. Any files that
705
you change will remain only on your system until you do a "commit", which
706
is explained later.
707
708
<P>
709
Let's say you are working in the directory scripts. You would then do:
710
711
<P>
712
<PRE>
713
cd scripts
714
(edit some files)
715
</PRE>
716
717
<P>
718
when you are happy with your changes, you can do the following:
719
720
<P>
721
<PRE>
722
cd bacula (to your top level directory)
723
cvs diff -u >my-changes.patch
724
</PRE>
725
726
<P>
727
When the command is done, you can look in the file my-changes.patch
728
and you will see all the changes you have made to your copy of the
729
repository. Make sure that you understand all the changes that
730
it reports before proceeding. If you modified files that you do
731
do not want to commit to the main repository, you can simply delete
732
them from your local directory, and they will be restored from the
733
repository with the "cvs update" that is shown below. Normally, you
734
should not find changes to files that you do not want to commit, and
735
if you find yourself in that position a lot, you are probably doing
736
something wrong.
737
738
<P>
739
Let's assume that now you want to commit your changes to the main CVS
740
repository.
741
742
<P>
743
First do:
744
745
<P>
746
<PRE>
747
cvs bacula
748
cvs update
749
</PRE>
750
751
<P>
752
When you do this, it will pull any changes made by other developers into
753
your local copy of the repository, and it will check for conflicts. If there
754
are any, it will tell you, and you will need to resolve them. The problems
755
of resolving conflicts are a bit more than this document can cover, but
756
you can examine the files it claims have conflicts and look for <<<<
757
or look in the .rej files that it creates. If you have problems, just ask
758
on the developer's list.
759
760
<P>
761
Note, doing the above "cvs update" is not absolutely necessary. There are
762
times when you may be working on code and you want to commit it, but you
763
explicitly do not want to move up to the latest version of the code in
764
the CVS. If that is the case, you can simply skip the "cvs update" and
765
do the commit shown below. If the commit fails because of a conflict, it
766
will tell you, and you must resolve the conflict before it will permit
767
you to do the commit.
768
769
<P>
770
Once your local copy of the repository has been updated, you can now
771
commit your changes:
772
773
<P>
774
<PRE>
775
cvs commit -m "Some comment about what you changed"
776
</PRE>
777
778
<P>
779
or if you really only want to commit a single file, you can
780
do:
781
782
<P>
783
<PRE>
784
cvs commit -m "comment" scripts/file-I-edited
785
</PRE>
786
787
<P>
788
Note, if you have done a build in your directory, or you have added
789
other new files, the commit will update only the files that are
790
actually in the repository. For example, none of the object files
791
are stored in the repository, so when you do a commit, those object
792
files will simply be ignored.
793
794
<P>
795
If you want to add new files or remove files from the main CVS
796
repository, and you are not experienced with CVS, please ask Kern
797
to do it. If you follow the simple steps above, it is unlikely that
798
you will do any damage to the repository, and if you do, it is always
799
possible for us to recover, but it can be painful.
800
801
<P>
802
If you are only working in one subdirectory of say the bacula project,
803
for example, the scripts directory, you can do your commit from
804
that subdirectory, and only the changes in that directory and all its
805
subdirectories will be committed. This can be helpful for translators.
806
If you are doing a French translation, you will be working in
807
docs/manual-fr, and if you are always cd'ed into that directory when
808
doing your commits, your commit will effect only that directory. As
809
long as you are careful only to change files that you want changed,
810
you have little to worry about.
811
812
<P>
813
814
<H2><A NAME="SECTION00043000000000000000"></A>
815
<A NAME="163"></A>
816
<A NAME="164"></A>
817
<BR>
818
The Development Cycle
819
</H2>
820
<A NAME="167"></A>
821
822
<P>
823
As I noted in the 1.38 ReleaseNotes, version 1.38 was different from prior
824
versions because it had a lot more contributions. I expect that this trend
825
will continue. As a consequence, I am going to modify how I normally do
826
development, and instead of making a list of all the features that I will
827
implement in the next version, I will personally sign up for one (maybe
828
two) projects at a time, and when they are complete, I will release a new
829
version.
830
831
<P>
832
The difference is that I will have more time to review the new code that is
833
being contributed, and will be able to devote more time to a smaller number
834
of projects (1.38 had too many new features for me to handle correctly).
835
836
<P>
837
I expect that future release schedules will be much the same, and the
838
number of new features will also be much the same providing that the
839
contributions continue to come -- and they show no signs of let up :-)
840
841
<P>
842
<A NAME="168"></A>
843
<B>Feature Requests:</B>
844
<BR>
845
In addition, I would like to "formalize" the feature requests a bit.
846
847
<P>
848
Instead of me maintaining an informal list of everything I run into
849
(kernstodo), I would like to maintain a "formal" list of projects. This
850
means that all new feature requests, including those recently discussed on
851
the email lists, must be formally submitted and approved.
852
853
<P>
854
Formal submission of feature requests will take two forms:
855
<BR>
856
1. non-mandatory, but highly recommended is to discuss proposed new features
857
on the mailing list.
858
<BR>
859
2. Formal submission of an Feature Request in a special format.
860
I'll give an example of this below, but you can also find it on the web
861
site under "Support -> Feature Requests". Since it takes a bit of time to
862
properly fill out a Feature Request form, you probably should check on the email list
863
first.
864
865
<P>
866
Once I receive the Feature Request, I will either accept it, send it back
867
asking for clarification, send it to the email list asking for opinions, or
868
reject it.
869
870
<P>
871
If it is accepted, it will go in the "projects" file (a simple ASCII file)
872
maintained in the main Bacula source directory.
873
874
<P>
875
<B>Implementation of Feature Requests:</B>
876
<BR>
877
Any qualified developer can sign up for a project. The project must have
878
an entry in the projects file, and the developer's name will appear in the
879
Status field.
880
881
<P>
882
<B>How Feature Requests are accepted:</B>
883
<BR>
884
Acceptance of Feature Requests depends on several things:
885
<BR>
886
1. feedback from users. If it is negative, the Feature Request will probably not be
887
accepted.
888
<BR>
889
2. the difficulty of the project. A project that is so
890
difficult that I cannot imagine finding someone to implement probably won't
891
be accepted.
892
<BR>
893
3. whether or not the Feature Request fits within the
894
current stategy of Bacula (for example an Feature Request that requests changing the
895
tape to tar format would not be accepted, ...)
896
897
<P>
898
<B>How Feature Requests are prioritized:</B>
899
<BR>
900
Once an Feature Request is accepted, it needs to be implemented. If you
901
can find a developer for it, or one signs up for implementing it, then the
902
Feature Request becomes top priority (at least for that developer).
903
904
<P>
905
Between releases of Bacula, I will generally solicit Feature Request input
906
for the next version, and by way of this email, I suggest that you send
907
discuss and send in your Feature Requests for the next release. Please
908
verify that the Feature Request is not in the current list (attached to this email).
909
910
<P>
911
Once users have had several weeks to submit Feature Requests, I will
912
organize them, and request users to vote on them. This will allow fixing
913
prioritizing the Feature Requests. Having a priority is one thing, but
914
getting it implement is another thing -- I am hoping that the Bacula
915
community will take more responsibility for assuring the implementation of
916
accepted Feature Requests.
917
918
<P>
919
Feature Request format:
920
<PRE>
921
============= Empty Feature Request form ===========
922
Item n: One line summary ...
923
Date: Date submitted
924
Origin: Name and email of originator.
925
Status:
926
927
What: More detailed explanation ...
928
929
Why: Why it is important ...
930
931
Notes: Additional notes or features (omit if not used)
932
============== End Feature Request form ==============
933
</PRE>
934
935
<P>
936
<PRE>
937
============= Example Completed Feature Request form ===========
938
Item 1: Implement a Migration job type that will move the job
939
data from one device to another.
940
Origin: Sponsored by Riege Sofware International GmbH. Contact:
941
Daniel Holtkamp <holtkamp at riege dot com>
942
Date: 28 October 2005
943
Status: Partially coded in 1.37 -- much more to do. Assigned to
944
Kern.
945
946
What: The ability to copy, move, or archive data that is on a
947
device to another device is very important.
948
949
Why: An ISP might want to backup to disk, but after 30 days
950
migrate the data to tape backup and delete it from
951
disk. Bacula should be able to handle this
952
automatically. It needs to know what was put where,
953
and when, and what to migrate -- it is a bit like
954
retention periods. Doing so would allow space to be
955
freed up for current backups while maintaining older
956
data on tape drives.
957
958
Notes: Migration could be triggered by:
959
Number of Jobs
960
Number of Volumes
961
Age of Jobs
962
Highwater size (keep total size)
963
Lowwater mark
964
=================================================
965
</PRE>
966
967
<P>
968
969
<H2><A NAME="SECTION00044000000000000000"></A>
970
<A NAME="179"></A>
971
<A NAME="180"></A>
972
<BR>
973
Developing Bacula
974
</H2>
975
<A NAME="183"></A>
976
977
<P>
978
Typically the simplest way to develop Bacula is to open one xterm window
979
pointing to the source directory you wish to update; a second xterm window at
980
the top source directory level, and a third xterm window at the bacula
981
directory <top>/src/bacula. After making source changes in one of the
982
directories, in the top source directory xterm, build the source, and start
983
the daemons by entering:
984
985
<P>
986
make and
987
988
<P>
989
./startit then in the enter:
990
991
<P>
992
./console or
993
994
<P>
995
./gnome-console to start the Console program. Enter any commands for testing.
996
For example: run kernsverify full.
997
998
<P>
999
Note, the instructions here to use <B>./startit</B> are different from using a
1000
production system where the administrator starts Bacula by entering <B>./bacula start</B>. This difference allows a development version of <B>Bacula</B>
1001
to be run on a computer at the same time that a production system is running.
1002
The <B>./startit</B> strip starts <B>Bacula</B> using a different set of
1003
configuration files, and thus permits avoiding conflicts with any production
1004
system.
1005
1006
<P>
1007
To make additional source changes, exit from the Console program, and in the
1008
top source directory, stop the daemons by entering:
1009
1010
<P>
1011
./stopit then repeat the process.
1012
1013
<P>
1014
1015
<H3><A NAME="SECTION00044100000000000000"></A>
1016
<A NAME="192"></A>
1017
<BR>
1018
Debugging
1019
</H3>
1020
<A NAME="195"></A>
1021
1022
<P>
1023
Probably the first thing to do is to turn on debug output.
1024
1025
<P>
1026
A good place to start is with a debug level of 20 as in <B>./startit -d20</B>.
1027
The startit command starts all the daemons with the same debug level.
1028
Alternatively, you can start the appropriate daemon with the debug level you
1029
want. If you really need more info, a debug level of 60 is not bad, and for
1030
just about everything a level of 200.
1031
1032
<P>
1033
1034
<H3><A NAME="SECTION00044200000000000000"></A>
1035
<A NAME="198"></A>
1036
<A NAME="199"></A>
1037
<BR>
1038
Using a Debugger
1039
</H3>
1040
<A NAME="202"></A>
1041
1042
<P>
1043
If you have a serious problem such as a segmentation fault, it can usually be
1044
found quickly using a good multiple thread debugger such as <B>gdb</B>. For
1045
example, suppose you get a segmentation violation in <B>bacula-dir</B>. You
1046
might use the following to find the problem:
1047
1048
<P>
1049
<start the Storage and File daemons>
1050
cd dird
1051
gdb ./bacula-dir
1052
run -f -s -c ./dird.conf
1053
<it dies with a segmentation fault>
1054
where
1055
The <B>-f</B> option is specified on the <B>run</B> command to inhibit <B>dird</B> from going into the background. You may also want to add the <B>-s</B>
1056
option to the run command to disable signals which can potentially interfere
1057
with the debugging.
1058
1059
<P>
1060
As an alternative to using the debugger, each <B>Bacula</B> daemon has a built
1061
in back trace feature when a serious error is encountered. It calls the
1062
debugger on itself, produces a back trace, and emails the report to the
1063
developer. For more details on this, please see the chapter in the main Bacula
1064
manual entitled ``What To Do When Bacula Crashes (Kaboom)''.
1065
1066
<P>
1067
1068
<H3><A NAME="SECTION00044300000000000000"></A>
1069
<A NAME="215"></A>
1070
<A NAME="216"></A>
1071
<BR>
1072
Memory Leaks
1073
</H3>
1074
<A NAME="219"></A>
1075
1076
<P>
1077
Because Bacula runs routinely and unattended on client and server machines, it
1078
may run for a long time. As a consequence, from the very beginning, Bacula
1079
uses SmartAlloc to ensure that there are no memory leaks. To make detection of
1080
memory leaks effective, all Bacula code that dynamically allocates memory MUST
1081
have a way to release it. In general when the memory is no longer needed, it
1082
should be immediately released, but in some cases, the memory will be held
1083
during the entire time that Bacula is executing. In that case, there MUST be a
1084
routine that can be called at termination time that releases the memory. In
1085
this way, we will be able to detect memory leaks. Be sure to immediately
1086
correct any and all memory leaks that are printed at the termination of the
1087
daemons.
1088
1089
<P>
1090
1091
<H3><A NAME="SECTION00044400000000000000"></A>
1092
<A NAME="221"></A>
1093
<A NAME="222"></A>
1094
<BR>
1095
Special Files
1096
</H3>
1097
<A NAME="225"></A>
1098
1099
<P>
1100
Kern uses files named 1, 2, ... 9 with any extension as scratch files. Thus
1101
any files with these names are subject to being rudely deleted at any time.
1102
1103
<P>
1104
1105
<H3><A NAME="SECTION00044500000000000000"></A>
1106
<A NAME="227"></A>
1107
<A NAME="228"></A>
1108
<BR>
1109
When Implementing Incomplete Code
1110
</H3>
1111
<A NAME="231"></A>
1112
1113
<P>
1114
Please identify all incomplete code with a comment that contains
1115
1116
<P>
1117
<PRE>
1118
***FIXME***
1119
</PRE>
1120
1121
<P>
1122
where there are three asterisks (*) before and after the word
1123
FIXME (in capitals) and no intervening spaces. This is important as it allows
1124
new programmers to easily recognize where things are partially implemented.
1125
1126
<P>
1127
1128
<H3><A NAME="SECTION00044600000000000000"></A>
1129
<A NAME="235"></A>
1130
<A NAME="236"></A>
1131
<BR>
1132
Bacula Source File Structure
1133
</H3>
1134
<A NAME="239"></A>
1135
1136
<P>
1137
The distribution generally comes as a tar file of the form <B>bacula.x.y.z.tar.gz</B> where x, y, and z are the version, release, and update
1138
numbers respectively.
1139
1140
<P>
1141
Once you detar this file, you will have a directory structure as follows:
1142
1143
<P>
1144
<PRE>
1145
|
1146
Tar file:
1147
|- depkgs
1148
|- mtx (autochanger control program + tape drive info)
1149
|- sqlite (SQLite database program)
1150
1151
Tar file:
1152
|- depkgs-win32
1153
|- pthreads (Native win32 pthreads library -- dll)
1154
|- zlib (Native win32 zlib library)
1155
|- wx (wxWidgets source code)
1156
1157
Project bacula:
1158
|- bacula (main source directory containing configuration
1159
| and installation files)
1160
|- autoconf (automatic configuration files, not normally used
1161
| by users)
1162
|- intl (programs used to translate)
1163
|- platforms (OS specific installation files)
1164
|- redhat (Red Hat installation)
1165
|- solaris (Sun installation)
1166
|- freebsd (FreeBSD installation)
1167
|- irix (Irix installation -- not tested)
1168
|- unknown (Default if system not identified)
1169
|- po (translations of source strings)
1170
|- src (source directory; contains global header files)
1171
|- cats (SQL catalog database interface directory)
1172
|- console (bacula user agent directory)
1173
|- dird (Director daemon)
1174
|- filed (Unix File daemon)
1175
|- win32 (Win32 files to make bacula-fd be a service)
1176
|- findlib (Unix file find library for File daemon)
1177
|- gnome-console (GNOME version of console program)
1178
|- lib (General Bacula library)
1179
|- stored (Storage daemon)
1180
|- tconsole (Tcl/tk console program -- not yet working)
1181
|- testprogs (test programs -- normally only in Kern's tree)
1182
|- tools (Various tool programs)
1183
|- win32 (Native Win32 File daemon)
1184
|- baculafd (Visual Studio project file)
1185
|- compat (compatibility interface library)
1186
|- filed (links to src/filed)
1187
|- findlib (links to src/findlib)
1188
|- lib (links to src/lib)
1189
|- console (beginning of native console program)
1190
|- wx-console (wxWidget console Win32 specific parts)
1191
|- wx-console (wxWidgets console main source program)
1192
1193
Project regress:
1194
|- regress (Regression scripts)
1195
|- bin (temporary directory to hold Bacula installed binaries)
1196
|- build (temporary directory to hold Bacula source)
1197
|- scripts (scripts and .conf files)
1198
|- tests (test scripts)
1199
|- tmp (temporary directory for temp files)
1200
1201
Project docs:
1202
|- docs (documentation directory)
1203
|- developers (Developer's guide)
1204
|- home-page (Bacula's home page source)
1205
|- manual (html document directory)
1206
|- manual-fr (French translation)
1207
|- manual-de (German translation)
1208
|- techlogs (Technical development notes);
1209
1210
Project rescue:
1211
|- rescue (Bacula rescue CDROM)
1212
|- linux (Linux rescue CDROM)
1213
|- cdrom (Linux rescue CDROM code)
1214
...
1215
|- solaris (Solaris rescue -- incomplete)
1216
|- freebsd (FreeBSD rescue -- incomplete)
1217
1218
Project gui:
1219
|- gui (Bacula GUI projects)
1220
|- bacula-web (Bacula web php management code)
1221
|- bimagemgr (Web application for burning CDROMs)
1222
</PRE>
1223
<P>
1224
1225
<H3><A NAME="SECTION00044700000000000000"></A>
1226
<A NAME="244"></A>
1227
<A NAME="245"></A>
1228
<BR>
1229
Header Files
1230
</H3>
1231
<A NAME="248"></A>
1232
1233
<P>
1234
Please carefully follow the scheme defined below as it permits in general only
1235
two header file includes per C file, and thus vastly simplifies programming.
1236
With a large complex project like Bacula, it isn't always easy to ensure that
1237
the right headers are invoked in the right order (there are a few kludges to
1238
make this happen -- i.e. in a few include files because of the chicken and egg
1239
problem, certain references to typedefs had to be replaced with <B>void</B> ).
1240
1241
<P>
1242
Every file should include <B>bacula.h</B>. It pulls in just about everything,
1243
with very few exceptions. If you have system dependent ifdefing, please do it
1244
in <B>baconfig.h</B>. The version number and date are kept in <B>version.h</B>.
1245
1246
<P>
1247
Each of the subdirectories (console, cats, dird, filed, findlib, lib, stored,
1248
...) contains a single directory dependent include file generally the name of
1249
the directory, which should be included just after the include of <B>bacula.h</B>. This file (for example, for the dird directory, it is <B>dird.h</B>)
1250
contains either definitions of things generally needed in this directory, or
1251
it includes the appropriate header files. It always includes <B>protos.h</B>.
1252
See below.
1253
1254
<P>
1255
Each subdirectory contains a header file named <B>protos.h</B>, which contains
1256
the prototypes for subroutines exported by files in that directory. <B>protos.h</B> is always included by the main directory dependent include file.
1257
1258
<P>
1259
1260
<H3><A NAME="SECTION00044800000000000000"></A>
1261
<A NAME="259"></A>
1262
<A NAME="260"></A>
1263
<BR>
1264
Programming Standards
1265
</H3>
1266
<A NAME="263"></A>
1267
1268
<P>
1269
For the most part, all code should be written in C unless there is a burning
1270
reason to use C++, and then only the simplest C++ constructs will be used.
1271
Note, Bacula is slowly evolving to use more and more C++.
1272
1273
<P>
1274
Code should have some documentation -- not a lot, but enough so that I can
1275
understand it. Look at the current code, and you will see that I document more
1276
than most, but am definitely not a fanatic.
1277
1278
<P>
1279
I prefer simple linear code where possible. Gotos are strongly discouraged
1280
except for handling an error to either bail out or to retry some code, and
1281
such use of gotos can vastly simplify the program.
1282
1283
<P>
1284
Remember this is a C program that is migrating to a <B>tiny</B> subset of C++,
1285
so be conservative in your use of C++ features.
1286
1287
<P>
1288
1289
<H3><A NAME="SECTION00044900000000000000"></A>
1290
<A NAME="266"></A>
1291
<A NAME="267"></A>
1292
<BR>
1293
Do Not Use
1294
</H3>
1295
<A NAME="270"></A>
1296
1297
<P>
1298
1299
<UL>
1300
<LI>STL -- it is totally incomprehensible.
1301
1302
</LI>
1303
</UL>
1304
1305
<P>
1306
1307
<H3><A NAME="SECTION000441000000000000000"></A>
1308
<A NAME="274"></A>
1309
<A NAME="275"></A>
1310
<BR>
1311
Avoid if Possible
1312
</H3>
1313
<A NAME="278"></A>
1314
1315
<P>
1316
1317
<UL>
1318
<LI>Using <B>void *</B> because this generally means that one must
1319
using casting, and in C++ casting is rather ugly. It is OK to use
1320
void * to pass structure address where the structure is not known
1321
to the routines accepting the packet (typically callback routines).
1322
However, declaring "void *buf" is a bad idea. Please use the
1323
correct types whenever possible.
1324
</LI>
1325
<LI>Using undefined storage specifications such as (short, int, long,
1326
long long, size_t ...). The problem with all these is that the number of bytes
1327
they allocate depends on the compiler and the system. Instead use
1328
Bacula's types (int8_t, uint8_t, int32_t, uint32_t, int64_t, and
1329
uint64_t). This guarantees that the variables are given exactly the
1330
size you want. Please try at all possible to avoid using size_t ssize_t
1331
and the such. They are very system dependent. However, some system
1332
routines may need them, so their use is often unavoidable.
1333
</LI>
1334
<LI>Returning a malloc'ed buffer from a subroutine -- someone will forget
1335
to release it.
1336
</LI>
1337
<LI>Using reference variables -- it allows subroutines to create side
1338
effects. Reference variables are OK, if you are sure the variable
1339
*always* exists, and you are sure you can handle the side effects of
1340
a subroutine changing the "pointer".
1341
</LI>
1342
<LI>Heap allocation (malloc) unless needed -- it is expensive.
1343
</LI>
1344
<LI>Templates -- they can create portability problems.
1345
</LI>
1346
<LI>Fancy or tricky C or C++ code, unless you give a good explanation of
1347
why you used it.
1348
</LI>
1349
<LI>Too much inheritance -- it can complicate the code, and make reading it
1350
difficult (unless you are in love with colons)
1351
1352
</LI>
1353
</UL>
1354
1355
<P>
1356
1357
<H3><A NAME="SECTION000441100000000000000"></A>
1358
<A NAME="283"></A>
1359
<A NAME="284"></A>
1360
<BR>
1361
Do Use Whenever Possible
1362
</H3>
1363
<A NAME="287"></A>
1364
1365
<P>
1366
1367
<UL>
1368
<LI>Locking and unlocking within a single subroutine.
1369
</LI>
1370
<LI>Malloc and free within a single subroutine.
1371
</LI>
1372
<LI>Comments and global explanations on what your code or algorithm does.
1373
1374
</LI>
1375
</UL>
1376
1377
<P>
1378
1379
<H3><A NAME="SECTION000441200000000000000"></A>
1380
<A NAME="291"></A>
1381
<A NAME="292"></A>
1382
<BR>
1383
Indenting Standards
1384
</H3>
1385
<A NAME="295"></A>
1386
1387
<P>
1388
I cannot stand code indented 8 columns at a time. This makes the code
1389
unreadable. Even 4 at a time uses a lot of space, so I have adopted indenting
1390
3 spaces at every level. Note, indention is the visual appearance of the
1391
source on the page, while tabbing is replacing a series of up to 8 spaces from
1392
a tab character.
1393
1394
<P>
1395
The closest set of parameters for the Linux <B>indent</B> program that will
1396
produce reasonably indented code are:
1397
1398
<P>
1399
<PRE>
1400
-nbad -bap -bbo -nbc -br -brs -c36 -cd36 -ncdb -ce -ci3 -cli0
1401
-cp36 -d0 -di1 -ndj -nfc1 -nfca -hnl -i3 -ip0 -l85 -lp -npcs
1402
-nprs -npsl -saf -sai -saw -nsob -nss -nbc -ncs -nbfda
1403
</PRE>
1404
<P>
1405
You can put the above in your .indent.pro file, and then just invoke indent on
1406
your file. However, be warned. This does not produce perfect indenting, and it
1407
will mess up C++ class statements pretty badly.
1408
1409
<P>
1410
Braces are required in all if statements (missing in some very old code). To
1411
avoid generating too many lines, the first brace appears on the first line
1412
(e.g. of an if), and the closing brace is on a line by itself. E.g.
1413
1414
<P>
1415
<PRE>
1416
if (abc) {
1417
some_code;
1418
}
1419
</PRE>
1420
<P>
1421
Just follow the convention in the code. Originally I indented case clauses
1422
under a switch(), but now I prefer non-indented cases.
1423
1424
<P>
1425
<PRE>
1426
switch (code) {
1427
case 'A':
1428
do something
1429
break;
1430
case 'B':
1431
again();
1432
break;
1433
default:
1434
break;
1435
}
1436
</PRE>
1437
<P>
1438
Avoid using // style comments except for temporary code or turning off debug
1439
code. Standard C comments are preferred (this also keeps the code closer to
1440
C).
1441
1442
<P>
1443
Attempt to keep all lines less than 85 characters long so that the whole line
1444
of code is readable at one time. This is not a rigid requirement.
1445
1446
<P>
1447
Always put a brief description at the top of any new file created describing
1448
what it does and including your name and the date it was first written. Please
1449
don't forget any Copyrights and acknowledgments if it isn't 100% your code.
1450
Also, include the Bacula copyright notice that is in <B>src/c</B>.
1451
1452
<P>
1453
In general you should have two includes at the top of the an include for the
1454
particular directory the code is in, for includes are needed, but this should
1455
be rare.
1456
1457
<P>
1458
In general (except for self-contained packages), prototypes should all be put
1459
in <B>protos.h</B> in each directory.
1460
1461
<P>
1462
Always put space around assignment and comparison operators.
1463
1464
<P>
1465
<PRE>
1466
a = 1;
1467
if (b >= 2) {
1468
cleanup();
1469
}
1470
</PRE>
1471
<P>
1472
but your can compress things in a <B>for</B> statement:
1473
1474
<P>
1475
<PRE>
1476
for (i=0; i < del.num_ids; i++) {
1477
...
1478
</PRE>
1479
<P>
1480
Don't overuse the inline if (?:). A full <B>if</B> is preferred, except in a
1481
print statement, e.g.:
1482
1483
<P>
1484
<PRE>
1485
if (ua->verbose \&& del.num_del != 0) {
1486
bsendmsg(ua, _("Pruned %d %s on Volume %s from catalog.\n"), del.num_del,
1487
del.num_del == 1 ? "Job" : "Jobs", mr->VolumeName);
1488
}
1489
</PRE>
1490
<P>
1491
Leave a certain amount of debug code (Dmsg) in code you submit, so that future
1492
problems can be identified. This is particularly true for complicated code
1493
likely to break. However, try to keep the debug code to a minimum to avoid
1494
bloating the program and above all to keep the code readable.
1495
1496
<P>
1497
Please keep the same style in all new code you develop. If you include code
1498
previously written, you have the option of leaving it with the old indenting
1499
or re-indenting it. If the old code is indented with 8 spaces, then please
1500
re-indent it to Bacula standards.
1501
1502
<P>
1503
If you are using <B>vim</B>, simply set your tabstop to 8 and your shiftwidth
1504
to 3.
1505
1506
<P>
1507
1508
<H3><A NAME="SECTION000441300000000000000"></A>
1509
<A NAME="315"></A>
1510
<BR>
1511
Tabbing
1512
</H3>
1513
<A NAME="318"></A>
1514
1515
<P>
1516
Tabbing (inserting the tab character in place of spaces) is as normal on all
1517
Unix systems -- a tab is converted space up to the next column multiple of 8.
1518
My editor converts strings of spaces to tabs automatically -- this results in
1519
significant compression of the files. Thus, you can remove tabs by replacing
1520
them with spaces if you wish. Please don't confuse tabbing (use of tab
1521
characters) with indenting (visual alignment of the code).
1522
1523
<P>
1524
1525
<H3><A NAME="SECTION000441400000000000000"></A>
1526
<A NAME="320"></A>
1527
<BR>
1528
Don'ts
1529
</H3>
1530
<A NAME="323"></A>
1531
1532
<P>
1533
Please don't use:
1534
1535
<P>
1536
<PRE>
1537
strcpy()
1538
strcat()
1539
strncpy()
1540
strncat();
1541
sprintf()
1542
snprintf()
1543
</PRE>
1544
<P>
1545
They are system dependent and un-safe. These should be replaced by the Bacula
1546
safe equivalents:
1547
1548
<P>
1549
<PRE>
1550
char *bstrncpy(char *dest, char *source, int dest_size);
1551
char *bstrncat(char *dest, char *source, int dest_size);
1552
int bsnprintf(char *buf, int32_t buf_len, const char *fmt, ...);
1553
int bvsnprintf(char *str, int32_t size, const char *format, va_list ap);
1554
</PRE>
1555
<P>
1556
See src/lib/bsys.c for more details on these routines.
1557
1558
<P>
1559
Don't use the <B>%lld</B> or the <B>%q</B> printf format editing types to edit
1560
64 bit integers -- they are not portable. Instead, use <B>%s</B> with <B>edit_uint64()</B>. For example:
1561
1562
<P>
1563
<PRE>
1564
char buf[100];
1565
uint64_t num = something;
1566
char ed1[50];
1567
bsnprintf(buf, sizeof(buf), "Num=%s\n", edit_uint64(num, ed1));
1568
</PRE>
1569
<P>
1570
The edit buffer <B>ed1</B> must be at least 27 bytes long to avoid overflow.
1571
See src/lib/edit.c for more details. If you look at the code, don't start
1572
screaming that I use <B>lld</B>. I actually use subtle trick taught to me by
1573
John Walker. The <B>lld</B> that appears in the editing routine is actually
1574
<B>#define</B> to a what is needed on your OS (usually ``lld'' or ``q'') and
1575
is defined in autoconf/configure.in for each OS. C string concatenation causes
1576
the appropriate string to be concatenated to the ``%''.
1577
1578
<P>
1579
Also please don't use the STL or Templates or any complicated C++ code.
1580
1581
<P>
1582
1583
<H3><A NAME="SECTION000441500000000000000"></A>
1584
<A NAME="339"></A>
1585
<A NAME="340"></A>
1586
<BR>
1587
Message Classes
1588
</H3>
1589
<A NAME="343"></A>
1590
1591
<P>
1592
Currently, there are five classes of messages: Debug, Error, Job, Memory,
1593
and Queued.
1594
1595
<P>
1596
1597
<H3><A NAME="SECTION000441600000000000000"></A>
1598
<A NAME="345"></A>
1599
<A NAME="346"></A>
1600
<BR>
1601
Debug Messages
1602
</H3>
1603
<A NAME="349"></A>
1604
1605
<P>
1606
Debug messages are designed to be turned on at a specified debug level and are
1607
always sent to STDOUT. There are designed to only be used in the development
1608
debug process. They are coded as:
1609
1610
<P>
1611
DmsgN(level, message, arg1, ...) where the N is a number indicating how many
1612
arguments are to be substituted into the message (i.e. it is a count of the
1613
number arguments you have in your message -- generally the number of percent
1614
signs (%)). <B>level</B> is the debug level at which you wish the message to
1615
be printed. message is the debug message to be printed, and arg1, ... are the
1616
arguments to be substituted. Since not all compilers support #defines with
1617
varargs, you must explicitly specify how many arguments you have.
1618
1619
<P>
1620
When the debug message is printed, it will automatically be prefixed by the
1621
name of the daemon which is running, the filename where the Dmsg is, and the
1622
line number within the file.
1623
1624
<P>
1625
Some actual examples are:
1626
1627
<P>
1628
Dmsg2(20, ``MD5len=%d MD5=%s\n'', strlen(buf), buf);
1629
1630
<P>
1631
Dmsg1(9, ``Created client %s record\n'', client->hdr.name);
1632
1633
<P>
1634
1635
<H3><A NAME="SECTION000441700000000000000"></A>
1636
<A NAME="354"></A>
1637
<A NAME="355"></A>
1638
<BR>
1639
Error Messages
1640
</H3>
1641
<A NAME="358"></A>
1642
1643
<P>
1644
Error messages are messages that are related to the daemon as a whole rather
1645
than a particular job. For example, an out of memory condition my generate an
1646
error message. They should be very rarely needed. In general, you should be
1647
using Job and Job Queued messages (Jmsg and Qmsg). They are coded as:
1648
1649
<P>
1650
EmsgN(error-code, level, message, arg1, ...) As with debug messages, you must
1651
explicitly code the of arguments to be substituted in the message. error-code
1652
indicates the severity or class of error, and it may be one of the following:
1653
1654
<P>
1655
<A NAME="361"></A>
1656
<TABLE CELLPADDING=3>
1657
<TR><TD ALIGN="LEFT"><B>M_ABORT</B></TD>
1658
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=216>Causes the daemon to immediately abort. This should be
1659
used only in extreme cases. It attempts to produce a traceback.</TD>
1660
</TR>
1661
<TR><TD ALIGN="LEFT"><B>M_ERROR_TERM</B></TD>
1662
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=216>Causes the daemon to immediately terminate. This
1663
should be used only in extreme cases. It does not produce a traceback.</TD>
1664
</TR>
1665
<TR><TD ALIGN="LEFT"><B>M_FATAL</B></TD>
1666
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=216>Causes the daemon to terminate the current job, but the
1667
daemon keeps running</TD>
1668
</TR>
1669
<TR><TD ALIGN="LEFT"><B>M_ERROR</B></TD>
1670
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=216>Reports the error. The daemon and the job continue
1671
running</TD>
1672
</TR>
1673
<TR><TD ALIGN="LEFT"><B>M_WARNING</B></TD>
1674
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=216>Reports an warning message. The daemon and the job
1675
continue running</TD>
1676
</TR>
1677
<TR><TD ALIGN="LEFT"><B>M_INFO</B></TD>
1678
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=216>Reports an informational message.
1679
1680
<P></TD>
1681
</TR>
1682
</TABLE>
1683
1684
<P>
1685
There are other error message classes, but they are in a state of being
1686
redesigned or deprecated, so please do not use them. Some actual examples are:
1687
1688
<P>
1689
Emsg1(M_ABORT, 0, ``Cannot create message thread: %s\n'',
1690
strerror(status));
1691
1692
<P>
1693
Emsg3(M_WARNING, 0, ``Connect to File daemon %s at %s:%d failed. Retrying
1694
...\n'', client->hdr.name, client->address,
1695
client->port);
1696
1697
<P>
1698
Emsg3(M_FATAL, 0, ``bdird<filed: bad response from Filed to %s command:
1699
%d %s\n'', cmd, n, strerror(errno));
1700
1701
<P>
1702
1703
<H3><A NAME="SECTION000441800000000000000"></A>
1704
<A NAME="385"></A>
1705
<A NAME="386"></A>
1706
<BR>
1707
Job Messages
1708
</H3>
1709
<A NAME="389"></A>
1710
1711
<P>
1712
Job messages are messages that pertain to a particular job such as a file that
1713
could not be saved, or the number of files and bytes that were saved. They
1714
Are coded as:
1715
<PRE>
1716
Jmsg(jcr, M\_FATAL, 0, "Text of message");
1717
</PRE>
1718
A Jmsg with M_FATAL will fail the job. The Jmsg() takes varargs so can
1719
have any number of arguments for substituted in a printf like format.
1720
Output from the Jmsg() will go to the Job report.
1721
<br>
1722
If the Jmsg is followed with a number such as Jmsg1(...), the number
1723
indicates the number of arguments to be substituted (varargs is not
1724
standard for #defines), and what is more important is that the file and
1725
line number will be prefixed to the message. This permits a sort of debug
1726
from user's output.
1727
1728
<P>
1729
1730
<H3><A NAME="SECTION000441900000000000000"></A>
1731
<A NAME="393"></A>
1732
<A NAME="394"></A>
1733
<BR>
1734
Queued Job Messages
1735
</H3>
1736
<A NAME="397"></A>
1737
Queued Job messages are similar to Jmsg()s except that the message is
1738
Queued rather than immediately dispatched. This is necessary within the
1739
network subroutines and in the message editing routines. This is to prevent
1740
recursive loops, and to ensure that messages can be delivered even in the
1741
event of a network error.
1742
1743
<P>
1744
1745
<H3><A NAME="SECTION000442000000000000000"></A>
1746
<A NAME="399"></A>
1747
<A NAME="400"></A>
1748
<BR>
1749
Memory Messages
1750
</H3>
1751
<A NAME="403"></A>
1752
1753
<P>
1754
Memory messages are messages that are edited into a memory buffer. Generally
1755
they are used in low level routines such as the low level device file dev.c in
1756
the Storage daemon or in the low level Catalog routines. These routines do not
1757
generally have access to the Job Control Record and so they return error
1758
messages reformatted in a memory buffer. Mmsg() is the way to do this.
1759
1760
<P>
1761
1762
<H1><A NAME="SECTION00050000000000000000"></A>
1763
<A NAME="_PlatformChapter"></A><A NAME="937"></A>
1764
<A NAME="938"></A>
1765
<BR>
1766
Platform Support
1767
</H1>
1768
<A NAME="941"></A>
1769
1770
<P>
1771
1772
<H2><A NAME="SECTION00051000000000000000"></A>
1773
<A NAME="943"></A>
1774
<BR>
1775
General
1776
</H2>
1777
<A NAME="946"></A>
1778
1779
<P>
1780
This chapter describes the requirements for having a
1781
supported platform (Operating System). In general, Bacula is
1782
quite portable. It supports 32 and 64 bit architectures as well
1783
as bigendian and littleendian machines. For full
1784
support, the platform (Operating System) must implement POSIX Unix
1785
system calls. However, for File daemon support only, a small
1786
compatibility library can be written to support almost any
1787
architecture.
1788
1789
<P>
1790
Currently Linux, FreeBSD, and Solaris are fully supported
1791
platforms, which means that the code has been tested on those
1792
machines and passes a full set of regression tests.
1793
1794
<P>
1795
In addition, the Windows File daemon is supported on most versions
1796
of Windows, and finally, there are a number of other platforms
1797
where the File daemon (client) is known to run: NetBSD, OpenBSD,
1798
Mac OSX, SGI, ...
1799
1800
<P>
1801
1802
<H2><A NAME="SECTION00052000000000000000"></A>
1803
<A NAME="948"></A>
1804
<A NAME="949"></A>
1805
<BR>
1806
Requirements to become a Supported Platform
1807
</H2>
1808
<A NAME="952"></A>
1809
1810
<P>
1811
As mentioned above, in order to become a fully supported platform, it
1812
must support POSIX Unix system calls. In addition, the following
1813
requirements must be met:
1814
1815
<P>
1816
1817
<UL>
1818
<LI>The principal developer (currently Kern) must have
1819
non-root ssh access to a test machine running the platform.
1820
</LI>
1821
<LI>The ideal requirements and minimum requirements
1822
for this machine are given below.
1823
</LI>
1824
<LI>There must be a defined platform champion who is normally
1825
a system administrator for the machine that is available. This
1826
person need not be a developer/programmer but must be familiar
1827
with system administration of the platform.
1828
</LI>
1829
<LI>There must be at least one person designated who will
1830
run regression tests prior to each release. Releases occur
1831
approximately once every 6 months, but can be more frequent.
1832
It takes at most a day's effort to setup the regression scripts
1833
in the beginning, and after that, they can either be run daily
1834
or on demand before a release. Running the regression scripts
1835
involves only one or two command line commands and is fully
1836
automated.
1837
</LI>
1838
<LI>Ideally there are one or more persons who will package
1839
each Bacula release.
1840
</LI>
1841
<LI>Ideally there are one or more developers who can respond to
1842
and fix platform specific bugs.
1843
</LI>
1844
</UL>
1845
1846
<P>
1847
Ideal requirements for a test machine:
1848
1849
<UL>
1850
<LI>The principal developer will have non-root ssh access to
1851
the test machine at all times.
1852
</LI>
1853
<LI>The pricipal developer will have a root password.
1854
</LI>
1855
<LI>The test machine will provide approximately 200 MB of
1856
disk space for continual use.
1857
</LI>
1858
<LI>The test machine will have approximately 500 MB of free
1859
disk space for temporary use.
1860
</LI>
1861
<LI>The test machine will run the most common version of the OS.
1862
</LI>
1863
<LI>The test machine will have an autochanger of DDS-4 technology
1864
or later having two or more tapes.
1865
</LI>
1866
<LI>The test machine will have MySQL and/or PostgreSQL database
1867
access for account "bacula" available.
1868
</LI>
1869
<LI>The test machine will have sftp access.
1870
</LI>
1871
<LI>The test machine will provide an smtp server.
1872
</LI>
1873
</UL>
1874
1875
<P>
1876
Minimum requirements for a test machine:
1877
1878
<UL>
1879
<LI>The principal developer will have non-root ssh access to
1880
the test machine when requested approximately once a month.
1881
</LI>
1882
<LI>The pricipal developer not have root access.
1883
</LI>
1884
<LI>The test machine will provide approximately 80 MB of
1885
disk space for continual use.
1886
</LI>
1887
<LI>The test machine will have approximately 300 MB of free
1888
disk space for temporary use.
1889
</LI>
1890
<LI>The test machine will run the the OS.
1891
</LI>
1892
<LI>The test machine will have a tape drive of DDS-4 technology
1893
or later that can be scheduled for access.
1894
</LI>
1895
<LI>The test machine will not have MySQL and/or PostgreSQL database
1896
access.
1897
</LI>
1898
<LI>The test machine will have no sftp access.
1899
</LI>
1900
<LI>The test machine will provide no email access.
1901
</LI>
1902
</UL>
1903
1904
<P>
1905
Bare bones test machine requirements:
1906
1907
<UL>
1908
<LI>The test machine is available only to a designated
1909
test person (your own machine).
1910
</LI>
1911
<LI>The designated test person runs the regession
1912
tests on demand.
1913
</LI>
1914
<LI>The test machine has a tape drive available.
1915
</LI>
1916
</UL>
1917
1918
<P>
1919
1920
<H1><A NAME="SECTION00060000000000000000"></A>
1921
<A NAME="_ChapterStart2"></A><A NAME="985"></A>
1922
<A NAME="986"></A>
1923
<BR>
1924
Daemon Protocol
1925
</H1>
1926
<A NAME="989"></A>
1927
1928
<P>
1929
1930
<H2><A NAME="SECTION00061000000000000000"></A>
1931
<A NAME="991"></A>
1932
<BR>
1933
General
1934
</H2>
1935
<A NAME="994"></A>
1936
1937
<P>
1938
This document describes the protocols used between the various daemons. As
1939
Bacula has developed, it has become quite out of date. The general idea still
1940
holds true, but the details of the fields for each command, and indeed the
1941
commands themselves have changed considerably.
1942
1943
<P>
1944
It is intended to be a technical discussion of the general daemon protocols
1945
and as such is not targeted at end users but rather at developers and system
1946
administrators that want or need to know more of the working details of <B>Bacula</B>.
1947
1948
<P>
1949
1950
<H2><A NAME="SECTION00062000000000000000"></A>
1951
<A NAME="997"></A>
1952
<A NAME="998"></A>
1953
<BR>
1954
Low Level Network Protocol
1955
</H2>
1956
<A NAME="1001"></A>
1957
1958
<P>
1959
At the lowest level, the network protocol is handled by <B>BSOCK</B> packets
1960
which contain a lot of information about the status of the network connection:
1961
who is at the other end, etc. Each basic <B>Bacula</B> network read or write
1962
actually consists of two low level network read/writes. The first write always
1963
sends four bytes of data in machine independent byte order. If data is to
1964
follow, the first four bytes are a positive non-zero integer indicating the
1965
length of the data that follow in the subsequent write. If the four byte
1966
integer is zero or negative, it indicates a special request, a sort of network
1967
signaling capability. In this case, no data packet will follow. The low level
1968
BSOCK routines expect that only a single thread is accessing the socket at a
1969
time. It is advised that multiple threads do not read/write the same socket.
1970
If you must do this, you must provide some sort of locking mechanism. It would
1971
not be appropriate for efficiency reasons to make every call to the BSOCK
1972
routines lock and unlock the packet.
1973
1974
<P>
1975
1976
<H2><A NAME="SECTION00063000000000000000"></A>
1977
<A NAME="1005"></A>
1978
<A NAME="1006"></A>
1979
<BR>
1980
General Daemon Protocol
1981
</H2>
1982
<A NAME="1009"></A>
1983
1984
<P>
1985
In general, all the daemons follow the following global rules. There may be
1986
exceptions depending on the specific case. Normally, one daemon will be
1987
sending commands to another daemon (specifically, the Director to the Storage
1988
daemon and the Director to the File daemon).
1989
1990
<P>
1991
1992
<UL>
1993
<LI>Commands are always ASCII commands that are upper/lower case dependent
1994
as well as space sensitive.
1995
</LI>
1996
<LI>All binary data is converted into ASCII (either with printf statements
1997
or using base64 encoding).
1998
</LI>
1999
<LI>All responses to commands sent are always prefixed with a return
2000
numeric code where codes in the 1000's are reserved for the Director, the
2001
2000's are reserved for the File daemon, and the 3000's are reserved for the
2002
Storage daemon.
2003
</LI>
2004
<LI>Any response that is not prefixed with a numeric code is a command (or
2005
subcommand if you like) coming from the other end. For example, while the
2006
Director is corresponding with the Storage daemon, the Storage daemon can
2007
request Catalog services from the Director. This convention permits each side
2008
to send commands to the other daemon while simultaneously responding to
2009
commands.
2010
</LI>
2011
<LI>Any response that is of zero length, depending on the context, either
2012
terminates the data stream being sent or terminates command mode prior to
2013
closing the connection.
2014
</LI>
2015
<LI>Any response that is of negative length is a special sign that normally
2016
requires a response. For example, during data transfer from the File daemon
2017
to the Storage daemon, normally the File daemon sends continuously without
2018
intervening reads. However, periodically, the File daemon will send a packet
2019
of length -1 indicating that the current data stream is complete and that the
2020
Storage daemon should respond to the packet with an OK, ABORT JOB, PAUSE,
2021
etc. This permits the File daemon to efficiently send data while at the same
2022
time occasionally ``polling'' the Storage daemon for his status or any
2023
special requests.
2024
2025
<P>
2026
Currently, these negative lengths are specific to the daemon, but shortly,
2027
the range 0 to -999 will be standard daemon wide signals, while -1000 to
2028
-1999 will be for Director user, -2000 to -2999 for the File daemon, and
2029
-3000 to -3999 for the Storage daemon.
2030
</LI>
2031
</UL>
2032
2033
<P>
2034
2035
<H2><A NAME="SECTION00064000000000000000"></A>
2036
<A NAME="1013"></A>
2037
<A NAME="1014"></A>
2038
<BR>
2039
The Protocol Used Between the Director and the Storage Daemon
2040
</H2>
2041
<A NAME="1017"></A>
2042
2043
<P>
2044
Before sending commands to the File daemon, the Director opens a Message
2045
channel with the Storage daemon, identifies itself and presents its password.
2046
If the password check is OK, the Storage daemon accepts the Director. The
2047
Director then passes the Storage daemon, the JobId to be run as well as the
2048
File daemon authorization (append, read all, or read for a specific session).
2049
The Storage daemon will then pass back to the Director a enabling key for this
2050
JobId that must be presented by the File daemon when opening the job. Until
2051
this process is complete, the Storage daemon is not available for use by File
2052
daemons.
2053
2054
<P>
2055
<PRE>
2056
SD: listens
2057
DR: makes connection
2058
DR: Hello <Director-name> calling <password>
2059
SD: 3000 OK Hello
2060
DR: JobId=nnn Allow=(append, read) Session=(*, SessionId)
2061
(Session not implemented yet)
2062
SD: 3000 OK Job Authorization=<password>
2063
DR: use device=<device-name> media_type=<media-type>
2064
pool_name=<pool-name> pool_type=<pool_type>
2065
SD: 3000 OK use device
2066
</PRE>
2067
<P>
2068
For the Director to be authorized, the <Director-name> and the
2069
<password> must match the values in one of the Storage daemon's
2070
Director resources (there may be several Directors that can access a single
2071
Storage daemon).
2072
2073
<P>
2074
2075
<H2><A NAME="SECTION00065000000000000000"></A>
2076
<A NAME="1025"></A>
2077
<A NAME="1026"></A>
2078
<BR>
2079
The Protocol Used Between the Director and the File Daemon
2080
</H2>
2081
<A NAME="1029"></A>
2082
2083
<P>
2084
A typical conversation might look like the following:
2085
2086
<P>
2087
<PRE>
2088
FD: listens
2089
DR: makes connection
2090
DR: Hello <Director-name> calling <password>
2091
FD: 2000 OK Hello
2092
DR: JobId=nnn Authorization=<password>
2093
FD: 2000 OK Job
2094
DR: storage address = <Storage daemon address> port = <port-number>
2095
name = <DeviceName> mediatype = <MediaType>
2096
FD: 2000 OK storage
2097
DR: include
2098
DR: <directory1>
2099
DR: <directory2>
2100
...
2101
DR: Null packet
2102
FD: 2000 OK include
2103
DR: exclude
2104
DR: <directory1>
2105
DR: <directory2>
2106
...
2107
DR: Null packet
2108
FD: 2000 OK exclude
2109
DR: full
2110
FD: 2000 OK full
2111
DR: save
2112
FD: 2000 OK save
2113
FD: Attribute record for each file as sent to the
2114
Storage daemon (described above).
2115
FD: Null packet
2116
FD: <append close responses from Storage daemon>
2117
e.g.
2118
3000 OK Volumes = <number of volumes>
2119
3001 Volume = <volume-id> <start file> <start block>
2120
<end file> <end block> <volume session-id>
2121
3002 Volume data = <date/time of last write> <Number bytes written>
2122
<number errors>
2123
... additional Volume / Volume data pairs for volumes 2 .. n
2124
FD: Null packet
2125
FD: close socket
2126
</PRE>
2127
<P>
2128
2129
<H2><A NAME="SECTION00066000000000000000"></A>
2130
<A NAME="1033"></A>
2131
<A NAME="1034"></A>
2132
<BR>
2133
The Save Protocol Between the File Daemon and the Storage Daemon
2134
</H2>
2135
<A NAME="1037"></A>
2136
2137
<P>
2138
Once the Director has send a <B>save</B> command to the File daemon, the File
2139
daemon will contact the Storage daemon to begin the save.
2140
2141
<P>
2142
In what follows: FD: refers to information set via the network from the File
2143
daemon to the Storage daemon, and SD: refers to information set from the
2144
Storage daemon to the File daemon.
2145
2146
<P>
2147
2148
<H3><A NAME="SECTION00066100000000000000"></A>
2149
<A NAME="1040"></A>
2150
<A NAME="1041"></A>
2151
<BR>
2152
Command and Control Information
2153
</H3>
2154
<A NAME="1044"></A>
2155
2156
<P>
2157
Command and control information is exchanged in human readable ASCII commands.
2158
2159
<P>
2160
<PRE>
2161
FD: listens
2162
SD: makes connection
2163
FD: append open session = <JobId> [<password>]
2164
SD: 3000 OK ticket = <number>
2165
FD: append data <ticket-number>
2166
SD: 3000 OK data address = <IPaddress> port = <port>
2167
</PRE>
2168
<P>
2169
2170
<H3><A NAME="SECTION00066200000000000000"></A>
2171
<A NAME="1048"></A>
2172
<A NAME="1049"></A>
2173
<BR>
2174
Data Information
2175
</H3>
2176
<A NAME="1052"></A>
2177
2178
<P>
2179
The Data information consists of the file attributes and data to the Storage
2180
daemon. For the most part, the data information is sent one way: from the File
2181
daemon to the Storage daemon. This allows the File daemon to transfer
2182
information as fast as possible without a lot of handshaking and network
2183
overhead.
2184
2185
<P>
2186
However, from time to time, the File daemon needs to do a sort of checkpoint
2187
of the situation to ensure that everything is going well with the Storage
2188
daemon. To do so, the File daemon sends a packet with a negative length
2189
indicating that he wishes the Storage daemon to respond by sending a packet of
2190
information to the File daemon. The File daemon then waits to receive a packet
2191
from the Storage daemon before continuing.
2192
2193
<P>
2194
All data sent are in binary format except for the header packet, which is in
2195
ASCII. There are two packet types used data transfer mode: a header packet,
2196
the contents of which are known to the Storage daemon, and a data packet, the
2197
contents of which are never examined by the Storage daemon.
2198
2199
<P>
2200
The first data packet to the Storage daemon will be an ASCII header packet
2201
consisting of the following data.
2202
2203
<P>
2204
<File-Index> <Stream-Id> <Info> where <B><File-Index></B> is a sequential number beginning from one that
2205
increments with each file (or directory) sent.
2206
2207
<P>
2208
where <B><Stream-Id></B> will be 1 for the Attributes record and 2 for
2209
uncompressed File data. 3 is reserved for the MD5 signature for the file.
2210
2211
<P>
2212
where <B><Info></B> transmit information about the Stream to the
2213
Storage Daemon. It is a character string field where each character has a
2214
meaning. The only character currently defined is 0 (zero), which is simply a
2215
place holder (a no op). In the future, there may be codes indicating
2216
compressed data, encrypted data, etc.
2217
2218
<P>
2219
Immediately following the header packet, the Storage daemon will expect any
2220
number of data packets. The series of data packets is terminated by a zero
2221
length packet, which indicates to the Storage daemon that the next packet will
2222
be another header packet. As previously mentioned, a negative length packet is
2223
a request for the Storage daemon to temporarily enter command mode and send a
2224
reply to the File daemon. Thus an actual conversation might contain the
2225
following exchanges:
2226
2227
<P>
2228
<PRE>
2229
FD: <1 1 0> (header packet)
2230
FD: <data packet containing file-attributes>
2231
FD: Null packet
2232
FD: <1 2 0>
2233
FD: <multiple data packets containing the file data>
2234
FD: Packet length = -1
2235
SD: 3000 OK
2236
FD: <2 1 0>
2237
FD: <data packet containing file-attributes>
2238
FD: Null packet
2239
FD: <2 2 0>
2240
FD: <multiple data packets containing the file data>
2241
FD: Null packet
2242
FD: Null packet
2243
FD: append end session <ticket-number>
2244
SD: 3000 OK end
2245
FD: append close session <ticket-number>
2246
SD: 3000 OK Volumes = <number of volumes>
2247
SD: 3001 Volume = <volumeid> <start file> <start block>
2248
<end file> <end block> <volume session-id>
2249
SD: 3002 Volume data = <date/time of last write> <Number bytes written>
2250
<number errors>
2251
SD: ... additional Volume / Volume data pairs for
2252
volumes 2 .. n
2253
FD: close socket
2254
</PRE>
2255
<P>
2256
The information returned to the File daemon by the Storage daemon in response
2257
to the <B>append close session</B> is transmit in turn to the Director.
2258
2259
<P>
2260
2261
<H1><A NAME="SECTION00070000000000000000"></A>
2262
<A NAME="_ChapterStart6"></A><A NAME="1173"></A>
2263
<A NAME="1174"></A>
2264
<BR>
2265
Director Services Daemon
2266
</H1>
2267
<A NAME="1177"></A>
2268
2269
<P>
2270
This chapter is intended to be a technical discussion of the Director services
2271
and as such is not targeted at end users but rather at developers and system
2272
administrators that want or need to know more of the working details of <B>Bacula</B>.
2273
2274
<P>
2275
The <B>Bacula Director</B> services consist of the program that supervises all
2276
the backup and restore operations.
2277
2278
<P>
2279
To be written ...
2280
2281
<P>
2282
2283
<H1><A NAME="SECTION00080000000000000000"></A>
2284
<A NAME="_ChapterStart11"></A><A NAME="1194"></A>
2285
<A NAME="1195"></A>
2286
<BR>
2287
File Services Daemon
2288
</H1>
2289
<A NAME="1198"></A>
2290
2291
<P>
2292
Please note, this section is somewhat out of date as the code has evolved
2293
significantly. The basic idea has not changed though.
2294
2295
<P>
2296
This chapter is intended to be a technical discussion of the File daemon
2297
services and as such is not targeted at end users but rather at developers and
2298
system administrators that want or need to know more of the working details of
2299
<B>Bacula</B>.
2300
2301
<P>
2302
The <B>Bacula File Services</B> consist of the programs that run on the system
2303
to be backed up and provide the interface between the Host File system and
2304
Bacula -- in particular, the Director and the Storage services.
2305
2306
<P>
2307
When time comes for a backup, the Director gets in touch with the File daemon
2308
on the client machine and hands it a set of ``marching orders'' which, if
2309
written in English, might be something like the following:
2310
2311
<P>
2312
OK, <B>File daemon</B>, it's time for your daily incremental backup. I want you
2313
to get in touch with the Storage daemon on host archive.mysite.com and perform
2314
the following save operations with the designated options. You'll note that
2315
I've attached include and exclude lists and patterns you should apply when
2316
backing up the file system. As this is an incremental backup, you should save
2317
only files modified since the time you started your last backup which, as you
2318
may recall, was 2000-11-19-06:43:38. Please let me know when you're done and
2319
how it went. Thank you.
2320
2321
<P>
2322
So, having been handed everything it needs to decide what to dump and where to
2323
store it, the File daemon doesn't need to have any further contact with the
2324
Director until the backup is complete providing there are no errors. If there
2325
are errors, the error messages will be delivered immediately to the Director.
2326
While the backup is proceeding, the File daemon will send the file coordinates
2327
and data for each file being backed up to the Storage daemon, which will in
2328
turn pass the file coordinates to the Director to put in the catalog.
2329
2330
<P>
2331
During a <B>Verify</B> of the catalog, the situation is different, since the
2332
File daemon will have an exchange with the Director for each file, and will
2333
not contact the Storage daemon.
2334
2335
<P>
2336
A <B>Restore</B> operation will be very similar to the <B>Backup</B> except that
2337
during the <B>Restore</B> the Storage daemon will not send storage coordinates
2338
to the Director since the Director presumably already has them. On the other
2339
hand, any error messages from either the Storage daemon or File daemon will
2340
normally be sent directly to the Directory (this, of course, depends on how
2341
the Message resource is defined).
2342
2343
<P>
2344
2345
<H2><A NAME="SECTION00081000000000000000"></A>
2346
<A NAME="1207"></A>
2347
<A NAME="1208"></A>
2348
<BR>
2349
Commands Received from the Director for a Backup
2350
</H2>
2351
<A NAME="1211"></A>
2352
2353
<P>
2354
To be written ...
2355
2356
<P>
2357
2358
<H2><A NAME="SECTION00082000000000000000"></A>
2359
<A NAME="1213"></A>
2360
<A NAME="1214"></A>
2361
<BR>
2362
Commands Received from the Director for a Restore
2363
</H2>
2364
<A NAME="1217"></A>
2365
2366
<P>
2367
To be written ...
2368
2369
<P>
2370
2371
<H1><A NAME="SECTION00090000000000000000"></A>
2372
<A NAME="_ChapterStart3"></A><A NAME="1260"></A>
2373
<A NAME="1261"></A>
2374
<BR>
2375
Storage Daemon Design
2376
</H1>
2377
<A NAME="1264"></A>
2378
2379
<P>
2380
This chapter is intended to be a technical discussion of the Storage daemon
2381
services and as such is not targeted at end users but rather at developers and
2382
system administrators that want or need to know more of the working details of
2383
<B>Bacula</B>.
2384
2385
<P>
2386
This document is somewhat out of date.
2387
2388
<P>
2389
2390
<H2><A NAME="SECTION00091000000000000000"></A>
2391
<A NAME="1267"></A>
2392
<A NAME="1268"></A>
2393
<BR>
2394
SD Design Introduction
2395
</H2>
2396
<A NAME="1271"></A>
2397
2398
<P>
2399
The Bacula Storage daemon provides storage resources to a Bacula installation.
2400
An individual Storage daemon is associated with a physical permanent storage
2401
device (for example, a tape drive, CD writer, tape changer or jukebox, etc.),
2402
and may employ auxiliary storage resources (such as space on a hard disk file
2403
system) to increase performance and/or optimize use of the permanent storage
2404
medium.
2405
2406
<P>
2407
Any number of storage daemons may be run on a given machine; each associated
2408
with an individual storage device connected to it, and BACULA operations may
2409
employ storage daemons on any number of hosts connected by a network, local or
2410
remote. The ability to employ remote storage daemons (with appropriate
2411
security measures) permits automatic off-site backup, possibly to publicly
2412
available backup repositories.
2413
2414
<P>
2415
2416
<H2><A NAME="SECTION00092000000000000000"></A>
2417
<A NAME="1273"></A>
2418
<A NAME="1274"></A>
2419
<BR>
2420
SD Development Outline
2421
</H2>
2422
<A NAME="1277"></A>
2423
2424
<P>
2425
In order to provide a high performance backup and restore solution that scales
2426
to very large capacity devices and networks, the storage daemon must be able
2427
to extract as much performance from the storage device and network with which
2428
it interacts. In order to accomplish this, storage daemons will eventually
2429
have to sacrifice simplicity and painless portability in favor of techniques
2430
which improve performance. My goal in designing the storage daemon protocol
2431
and developing the initial prototype storage daemon is to provide for these
2432
additions in the future, while implementing an initial storage daemon which is
2433
very simple and portable to almost any POSIX-like environment. This original
2434
storage daemon (and its evolved descendants) can serve as a portable solution
2435
for non-demanding backup requirements (such as single servers of modest size,
2436
individual machines, or small local networks), while serving as the starting
2437
point for development of higher performance configurable derivatives which use
2438
techniques such as POSIX threads, shared memory, asynchronous I/O, buffering
2439
to high-speed intermediate media, and support for tape changers and jukeboxes.
2440
2441
<P>
2442
2443
<H2><A NAME="SECTION00093000000000000000"></A>
2444
<A NAME="1279"></A>
2445
<A NAME="1280"></A>
2446
<BR>
2447
SD Connections and Sessions
2448
</H2>
2449
<A NAME="1283"></A>
2450
2451
<P>
2452
A client connects to a storage server by initiating a conventional TCP
2453
connection. The storage server accepts the connection unless its maximum
2454
number of connections has been reached or the specified host is not granted
2455
access to the storage server. Once a connection has been opened, the client
2456
may make any number of Query requests, and/or initiate (if permitted), one or
2457
more Append sessions (which transmit data to be stored by the storage daemon)
2458
and/or Read sessions (which retrieve data from the storage daemon).
2459
2460
<P>
2461
Most requests and replies sent across the connection are simple ASCII strings,
2462
with status replies prefixed by a four digit status code for easier parsing.
2463
Binary data appear in blocks stored and retrieved from the storage. Any
2464
request may result in a single-line status reply of ``<TT>3201 Notification pending</TT>'', which indicates the client must send a ``Query notification''
2465
request to retrieve one or more notifications posted to it. Once the
2466
notifications have been returned, the client may then resubmit the request
2467
which resulted in the 3201 status.
2468
2469
<P>
2470
The following descriptions omit common error codes, yet to be defined, which
2471
can occur from most or many requests due to events like media errors,
2472
restarting of the storage daemon, etc. These details will be filled in, along
2473
with a comprehensive list of status codes along with which requests can
2474
produce them in an update to this document.
2475
2476
<P>
2477
2478
<H3><A NAME="SECTION00093100000000000000"></A>
2479
<A NAME="1286"></A>
2480
<A NAME="1287"></A>
2481
<BR>
2482
SD Append Requests
2483
</H3>
2484
<A NAME="1290"></A>
2485
2486
<P>
2487
<DL>
2488
<DT><STRONG>append open session = <JobId> [ <Password> ] </STRONG></DT>
2489
<DD><A NAME="1296"></A>
2490
A data append session is opened with the Job ID given by <I>JobId</I> with
2491
client password (if required) given by <I>Password</I>. If the session is
2492
successfully opened, a status of <TT>3000 OK</TT> is returned with a ``<TT>ticket = </TT><I>number</I>'' reply used to identify subsequent messages in the
2493
session. If too many sessions are open, or a conflicting session (for
2494
example, a read in progress when simultaneous read and append sessions are
2495
not permitted), a status of ``<TT>3502 Volume busy</TT>'' is returned. If no
2496
volume is mounted, or the volume mounted cannot be appended to, a status of
2497
``<TT>3503 Volume not mounted</TT>'' is returned.
2498
2499
<P>
2500
</DD>
2501
<DT><STRONG>append data = <ticket-number> </STRONG></DT>
2502
<DD><A NAME="1306"></A>
2503
If the append data is accepted, a status of <TT>3000 OK data address =
2504
<IPaddress> port = <port></TT> is returned, where the <TT>IPaddress</TT> and <TT>port</TT> specify the IP address and port number of the data
2505
channel. Error status codes are <TT>3504 Invalid ticket number</TT> and <TT>3505 Session aborted</TT>, the latter of which indicates the entire append
2506
session has failed due to a daemon or media error.
2507
2508
<P>
2509
Once the File daemon has established the connection to the data channel
2510
opened by the Storage daemon, it will transfer a header packet followed by
2511
any number of data packets. The header packet is of the form:
2512
2513
<P>
2514
<TT><file-index> <stream-id> <info></TT>
2515
2516
<P>
2517
The details are specified in the
2518
<A HREF="#_ChapterStart2">Daemon Protocol</A> section of this
2519
document.
2520
2521
<P>
2522
</DD>
2523
<DT><STRONG>*append abort session = <ticket-number> </STRONG></DT>
2524
<DD><A NAME="1325"></A>
2525
The open append session with ticket <I>ticket-number</I> is aborted; any blocks
2526
not yet written to permanent media are discarded. Subsequent attempts to
2527
append data to the session will receive an error status of <TT>3505 Session aborted</TT>.
2528
2529
<P>
2530
</DD>
2531
<DT><STRONG>append end session = <ticket-number> </STRONG></DT>
2532
<DD><A NAME="1330"></A>
2533
The open append session with ticket <I>ticket-number</I> is marked complete; no
2534
further blocks may be appended. The storage daemon will give priority to
2535
saving any buffered blocks from this session to permanent media as soon as
2536
possible.
2537
2538
<P>
2539
</DD>
2540
<DT><STRONG>append close session = <ticket-number> </STRONG></DT>
2541
<DD><A NAME="1334"></A>
2542
The append session with ticket <I>ticket</I> is closed. This message does not
2543
receive an <TT>3000 OK</TT> reply until all of the content of the session are
2544
stored on permanent media, at which time said reply is given, followed by a
2545
list of volumes, from first to last, which contain blocks from the session,
2546
along with the first and last file and block on each containing session data
2547
and the volume session key identifying data from that session in lines with
2548
the following format:
2549
2550
<P>
2551
<TT><TT>Volume = </TT><Volume-id> <start-file>
2552
<start-block> <end-file> <end-block>
2553
<volume-session-id></TT>where <I>Volume-id</I> is the volume label, <I>start-file</I> and <I>start-block</I> are the file and block containing the first
2554
data from that session on the volume, <I>end-file</I> and <I>end-block</I> are
2555
the file and block with the last data from the session on the volume and <I>volume-session-id</I> is the volume session ID for blocks from the session
2556
stored on that volume.
2557
</DD>
2558
</DL>
2559
2560
<P>
2561
2562
<H3><A NAME="SECTION00093200000000000000"></A>
2563
<A NAME="1358"></A>
2564
<A NAME="1359"></A>
2565
<BR>
2566
SD Read Requests
2567
</H3>
2568
<A NAME="1362"></A>
2569
2570
<P>
2571
<DL>
2572
<DT><STRONG>Read open session = <JobId> <Volume-id>
2573
<start-file> <start-block> <end-file>
2574
<end-block> <volume-session-id> <password> </STRONG></DT>
2575
<DD><A NAME="1380"></A>
2576
where <I>Volume-id</I> is the volume label, <I>start-file</I> and <I>start-block</I> are the file and block containing the first data from that
2577
session on the volume, <I>end-file</I> and <I>end-block</I> are the file and
2578
block with the last data from the session on the volume and <I>volume-session-id</I> is the volume session ID for blocks from the session
2579
stored on that volume.
2580
2581
<P>
2582
If the session is successfully opened, a status of
2583
2584
<P>
2585
<TT><TT>3100 OK Ticket = </TT><I>number</I>``</TT>
2586
2587
<P>
2588
is returned with a reply used to identify subsequent messages in the session.
2589
If too many sessions are open, or a conflicting session (for example, an
2590
append in progress when simultaneous read and append sessions are not
2591
permitted), a status of ''<TT>3502 Volume busy</TT>`` is returned. If no
2592
volume is mounted, or the volume mounted cannot be appended to, a status of
2593
''<TT>3503 Volume not mounted</TT>`` is returned. If no block with the given
2594
volume session ID and the correct client ID number appears in the given first
2595
file and block for the volume, a status of ''<TT>3505 Session not found</TT>`` is returned.
2596
2597
<P>
2598
</DD>
2599
<DT><STRONG>Read data = <Ticket> > <Block> </STRONG></DT>
2600
<DD><A NAME="1397"></A>
2601
The specified Block of data from open read session with the specified Ticket
2602
number is returned, with a status of <TT>3000 OK</TT> followed by a ''<TT>Length = </TT><I>size</I>`` line giving the length in bytes of the block data
2603
which immediately follows. Blocks must be retrieved in ascending order, but
2604
blocks may be skipped. If a block number greater than the largest stored on
2605
the volume is requested, a status of ''<TT>3201 End of volume</TT>`` is
2606
returned. If a block number greater than the largest in the file is
2607
requested, a status of ''<TT>3401 End of file</TT>`` is returned.
2608
2609
<P>
2610
</DD>
2611
<DT><STRONG>Read close session = <Ticket> </STRONG></DT>
2612
<DD><A NAME="1405"></A>
2613
The read session with Ticket number is closed. A read session may be closed
2614
at any time; you needn't read all its blocks before closing it.
2615
</DD>
2616
</DL>
2617
2618
<P>
2619
<I>by
2620
<A NAME="tex2html2"
2621
HREF="http://www.fourmilab.ch/">John Walker</A>
2622
January 30th, MM </I>
2623
2624
<P>
2625
2626
<H2><A NAME="SECTION00094000000000000000"></A>
2627
<A NAME="1410"></A>
2628
<BR>
2629
SD Data Structures
2630
</H2>
2631
<A NAME="1413"></A>
2632
2633
<P>
2634
In the Storage daemon, there is a Device resource (i.e. from conf file)
2635
that describes each physical device. When the physical device is used it
2636
is controled by the DEVICE structure (defined in dev.h), and typically
2637
refered to as dev in the C++ code. Anyone writing or reading a physical
2638
device must ultimately get a lock on the DEVICE structure -- this controls
2639
the device. However, multiple Jobs (defined by a JCR structure src/jcr.h)
2640
can be writing a physical DEVICE at the same time (of course they are
2641
sequenced by locking the DEVICE structure). There are a lot of job
2642
dependent "device" variables that may be different for each Job such as
2643
spooling (one job may spool and another may not, and when a job is
2644
spooling, it must have an i/o packet open, each job has its own record and
2645
block structures, ...), so there is a device control record or DCR that is
2646
the primary way of interfacing to the physical device. The DCR contains
2647
all the job specific data as well as a pointer to the Device resource
2648
(DEVRES structure) and the physical DEVICE structure.
2649
2650
<P>
2651
Now if a job is writing to two devices (it could be writing two separate
2652
streams to the same device), it must have two DCRs. Today, the code only
2653
permits one. This won't be hard to change, but it is new code.
2654
2655
<P>
2656
Today three jobs (threads), two physical devices each job
2657
writes to only one device:
2658
2659
<P>
2660
<PRE>
2661
Job1 -> DCR1 -> DEVICE1
2662
Job2 -> DCR2 -> DEVICE1
2663
Job3 -> DCR3 -> DEVICE2
2664
</PRE>
2665
2666
<P>
2667
To be implemented three jobs, three physical devices, but
2668
job1 is writing simultaneously to three devices:
2669
2670
<P>
2671
<PRE>
2672
Job1 -> DCR1 -> DEVICE1
2673
-> DCR4 -> DEVICE2
2674
-> DCR5 -> DEVICE3
2675
Job2 -> DCR2 -> DEVICE1
2676
Job3 -> DCR3 -> DEVICE2
2677
2678
Job = job control record
2679
DCR = Job contorl data for a specific device
2680
DEVICE = Device only control data
2681
</PRE>
2682
2683
<P>
2684
2685
<P>
2686
2687
<H1><A NAME="SECTION000100000000000000000"></A>
2688
<A NAME="_ChapterStart30"></A>
2689
<BR>
2690
Catalog Services
2691
</H1>
2692
<A NAME="1598"></A>
2693
<A NAME="1599"></A>
2694
<A NAME="1602"></A>
2695
2696
<P>
2697
2698
<H2><A NAME="SECTION000101000000000000000">
2699
General</A>
2700
</H2>
2701
<A NAME="1604"></A>
2702
<A NAME="1607"></A>
2703
2704
<P>
2705
This chapter is intended to be a technical discussion of the Catalog services
2706
and as such is not targeted at end users but rather at developers and system
2707
administrators that want or need to know more of the working details of <B>Bacula</B>.
2708
2709
<P>
2710
The <B>Bacula Catalog</B> services consist of the programs that provide the SQL
2711
database engine for storage and retrieval of all information concerning files
2712
that were backed up and their locations on the storage media.
2713
2714
<P>
2715
We have investigated the possibility of using the following SQL engines for
2716
Bacula: Beagle, mSQL, GNU SQL, PostgreSQL, SQLite, Oracle, and MySQL. Each
2717
presents certain problems with either licensing or maturity. At present, we
2718
have chosen for development purposes to use MySQL, PostgreSQL and SQLite.
2719
MySQL was chosen because it is fast, proven to be reliable, widely used, and
2720
actively being developed. MySQL is released under the GNU GPL license.
2721
PostgreSQL was chosen because it is a full-featured, very mature database, and
2722
because Dan Langille did the Bacula driver for it. PostgreSQL is distributed
2723
under the BSD license. SQLite was chosen because it is small, efficient, and
2724
can be directly embedded in <B>Bacula</B> thus requiring much less effort from
2725
the system administrator or person building <B>Bacula</B>. In our testing
2726
SQLite has performed very well, and for the functions that we use, it has
2727
never encountered any errors except that it does not appear to handle
2728
databases larger than 2GBytes. That said, we would not recommend it for
2729
serious production use.
2730
2731
<P>
2732
The Bacula SQL code has been written in a manner that will allow it to be
2733
easily modified to support any of the current SQL database systems on the
2734
market (for example: mSQL, iODBC, unixODBC, Solid, OpenLink ODBC, EasySoft
2735
ODBC, InterBase, Oracle8, Oracle7, and DB2).
2736
2737
<P>
2738
If you do not specify either <B><code>--</code>with-mysql</B> or <B><code>--</code>with-postgresql</B> or
2739
<B><code>--</code>with-sqlite</B> on the ./configure line, Bacula will use its minimalist
2740
internal database. This database is kept for build reasons but is no longer
2741
supported. Bacula <B>requires</B> one of the three databases (MySQL,
2742
PostgreSQL, or SQLite) to run.
2743
2744
<P>
2745
2746
<H3><A NAME="SECTION000101100000000000000">
2747
Filenames and Maximum Filename Length</A>
2748
</H3>
2749
<A NAME="1617"></A>
2750
<A NAME="1618"></A>
2751
<A NAME="1621"></A>
2752
2753
<P>
2754
In general, either MySQL, PostgreSQL or SQLite permit storing arbitrary long
2755
path names and file names in the catalog database. In practice, there still
2756
may be one or two places in the Catalog interface code that restrict the
2757
maximum path length to 512 characters and the maximum file name length to 512
2758
characters. These restrictions are believed to have been removed. Please note,
2759
these restrictions apply only to the Catalog database and thus to your ability
2760
to list online the files saved during any job. All information received and
2761
stored by the Storage daemon (normally on tape) allows and handles arbitrarily
2762
long path and filenames.
2763
2764
<P>
2765
2766
<H3><A NAME="SECTION000101200000000000000">
2767
Installing and Configuring MySQL</A>
2768
</H3>
2769
<A NAME="1623"></A>
2770
<A NAME="1624"></A>
2771
<A NAME="1627"></A>
2772
2773
<P>
2774
For the details of installing and configuring MySQL, please see the
2775
<A HREF="#_ChapterStart">Installing and Configuring MySQL</A> chapter of
2776
this manual.
2777
2778
<P>
2779
2780
<H3><A NAME="SECTION000101300000000000000">
2781
Installing and Configuring PostgreSQL</A>
2782
</H3>
2783
<A NAME="1631"></A>
2784
<A NAME="1632"></A>
2785
<A NAME="1635"></A>
2786
2787
<P>
2788
For the details of installing and configuring PostgreSQL, please see the
2789
<A HREF="#_ChapterStart10">Installing and Configuring PostgreSQL</A>
2790
chapter of this manual.
2791
2792
<P>
2793
2794
<H3><A NAME="SECTION000101400000000000000">
2795
Installing and Configuring SQLite</A>
2796
</H3>
2797
<A NAME="1639"></A>
2798
<A NAME="1640"></A>
2799
<A NAME="1643"></A>
2800
2801
<P>
2802
For the details of installing and configuring SQLite, please see the
2803
<A HREF="#_ChapterStart33">Installing and Configuring SQLite</A> chapter of
2804
this manual.
2805
2806
<P>
2807
2808
<H3><A NAME="SECTION000101500000000000000">
2809
Internal Bacula Catalog</A>
2810
</H3>
2811
<A NAME="1647"></A>
2812
<A NAME="1648"></A>
2813
<A NAME="1651"></A>
2814
2815
<P>
2816
Please see the
2817
<A HREF="#_ChapterStart42">Internal Bacula Database</A> chapter of this
2818
manual for more details.
2819
2820
<P>
2821
2822
<H3><A NAME="SECTION000101600000000000000">
2823
Database Table Design</A>
2824
</H3>
2825
<A NAME="1655"></A>
2826
<A NAME="1656"></A>
2827
<A NAME="1659"></A>
2828
2829
<P>
2830
All discussions that follow pertain to the MySQL database. The details for the
2831
PostgreSQL and SQLite databases are essentially identical except for that all
2832
fields in the SQLite database are stored as ASCII text and some of the
2833
database creation statements are a bit different. The details of the internal
2834
Bacula catalog are not discussed here.
2835
2836
<P>
2837
Because the Catalog database may contain very large amounts of data for large
2838
sites, we have made a modest attempt to normalize the data tables to reduce
2839
redundant information. While reducing the size of the database significantly,
2840
it does, unfortunately, add some complications to the structures.
2841
2842
<P>
2843
In simple terms, the Catalog database must contain a record of all Jobs run by
2844
Bacula, and for each Job, it must maintain a list of all files saved, with
2845
their File Attributes (permissions, create date, ...), and the location and
2846
Media on which the file is stored. This is seemingly a simple task, but it
2847
represents a huge amount interlinked data. Note: the list of files and their
2848
attributes is not maintained when using the internal Bacula database. The data
2849
stored in the File records, which allows the user or administrator to obtain a
2850
list of all files backed up during a job, is by far the largest volume of
2851
information put into the Catalog database.
2852
2853
<P>
2854
Although the Catalog database has been designed to handle backup data for
2855
multiple clients, some users may want to maintain multiple databases, one for
2856
each machine to be backed up. This reduces the risk of confusion of accidental
2857
restoring a file to the wrong machine as well as reducing the amount of data
2858
in a single database, thus increasing efficiency and reducing the impact of a
2859
lost or damaged database.
2860
2861
<P>
2862
2863
<H2><A NAME="SECTION000102000000000000000">
2864
Sequence of Creation of Records for a Save Job</A>
2865
</H2>
2866
<A NAME="1661"></A>
2867
<A NAME="1662"></A>
2868
<A NAME="1665"></A>
2869
2870
<P>
2871
Start with StartDate, ClientName, Filename, Path, Attributes, MediaName,
2872
MediaCoordinates. (PartNumber, NumParts). In the steps below, ``Create new''
2873
means to create a new record whether or not it is unique. ``Create unique''
2874
means each record in the database should be unique. Thus, one must first
2875
search to see if the record exists, and only if not should a new one be
2876
created, otherwise the existing RecordId should be used.
2877
2878
<P>
2879
2880
<OL>
2881
<LI>Create new Job record with StartDate; save JobId
2882
</LI>
2883
<LI>Create unique Media record; save MediaId
2884
</LI>
2885
<LI>Create unique Client record; save ClientId
2886
</LI>
2887
<LI>Create unique Filename record; save FilenameId
2888
</LI>
2889
<LI>Create unique Path record; save PathId
2890
</LI>
2891
<LI>Create unique Attribute record; save AttributeId
2892
store ClientId, FilenameId, PathId, and Attributes
2893
</LI>
2894
<LI>Create new File record
2895
store JobId, AttributeId, MediaCoordinates, etc
2896
</LI>
2897
<LI>Repeat steps 4 through 8 for each file
2898
</LI>
2899
<LI>Create a JobMedia record; save MediaId
2900
</LI>
2901
<LI>Update Job record filling in EndDate and other Job statistics
2902
2903
</LI>
2904
</OL>
2905
2906
<P>
2907
2908
<H2><A NAME="SECTION000103000000000000000">
2909
Database Tables</A>
2910
</H2>
2911
<A NAME="1669"></A>
2912
<A NAME="1670"></A>
2913
<A NAME="1673"></A>
2914
2915
<P>
2916
<A NAME="1676"></A>
2917
<TABLE CELLPADDING=3 BORDER="1">
2918
<TR><TD ALIGN="LEFT" COLSPAN=3><B>Filename </B></TD>
2919
</TR>
2920
<TR><TD ALIGN="CENTER" COLSPAN=1><B>Column Name </B></TD>
2921
<TD ALIGN="LEFT" COLSPAN=1><B>Data Type </B></TD>
2922
<TD ALIGN="LEFT" COLSPAN=1><B>Remark </B></TD>
2923
</TR>
2924
<TR><TD ALIGN="LEFT">FilenameId</TD>
2925
<TD ALIGN="LEFT">integer</TD>
2926
<TD ALIGN="LEFT">Primary Key</TD>
2927
</TR>
2928
<TR><TD ALIGN="LEFT">Name</TD>
2929
<TD ALIGN="LEFT">Blob</TD>
2930
<TD ALIGN="LEFT">Filename</TD>
2931
</TR>
2932
</TABLE>
2933
2934
<P>
2935
The <B>Filename</B> table shown above contains the name of each file backed up
2936
with the path removed. If different directories or machines contain the same
2937
filename, only one copy will be saved in this table.
2938
2939
<P>
2940
2941
<P>
2942
<A NAME="1701"></A>
2943
<TABLE CELLPADDING=3 BORDER="1">
2944
<TR><TD ALIGN="LEFT" COLSPAN=3><B>Path </B></TD>
2945
</TR>
2946
<TR><TD ALIGN="CENTER" COLSPAN=1><B>Column Name </B></TD>
2947
<TD ALIGN="CENTER" COLSPAN=1><B>Data Type
2948
</B></TD>
2949
<TD ALIGN="CENTER" COLSPAN=1><B>Remark </B></TD>
2950
</TR>
2951
<TR><TD ALIGN="LEFT">PathId</TD>
2952
<TD ALIGN="LEFT">integer</TD>
2953
<TD ALIGN="LEFT">Primary Key</TD>
2954
</TR>
2955
<TR><TD ALIGN="LEFT">Path</TD>
2956
<TD ALIGN="LEFT">Blob</TD>
2957
<TD ALIGN="LEFT">Full Path</TD>
2958
</TR>
2959
</TABLE>
2960
2961
<P>
2962
The <B>Path</B> table contains shown above the path or directory names of all
2963
directories on the system or systems. The filename and any MSDOS disk name are
2964
stripped off. As with the filename, only one copy of each directory name is
2965
kept regardless of how many machines or drives have the same directory. These
2966
path names should be stored in Unix path name format.
2967
2968
<P>
2969
Some simple testing on a Linux file system indicates that separating the
2970
filename and the path may be more complication than is warranted by the space
2971
savings. For example, this system has a total of 89,097 files, 60,467 of which
2972
have unique filenames, and there are 4,374 unique paths.
2973
2974
<P>
2975
Finding all those files and doing two stats() per file takes an average wall
2976
clock time of 1 min 35 seconds on a 400MHz machine running RedHat 6.1 Linux.
2977
2978
<P>
2979
Finding all those files and putting them directly into a MySQL database with
2980
the path and filename defined as TEXT, which is variable length up to 65,535
2981
characters takes 19 mins 31 seconds and creates a 27.6 MByte database.
2982
2983
<P>
2984
Doing the same thing, but inserting them into Blob fields with the filename
2985
indexed on the first 30 characters and the path name indexed on the 255 (max)
2986
characters takes 5 mins 18 seconds and creates a 5.24 MB database. Rerunning
2987
the job (with the database already created) takes about 2 mins 50 seconds.
2988
2989
<P>
2990
Running the same as the last one (Path and Filename Blob), but Filename
2991
indexed on the first 30 characters and the Path on the first 50 characters
2992
(linear search done there after) takes 5 mins on the average and creates a 3.4
2993
MB database. Rerunning with the data already in the DB takes 3 mins 35
2994
seconds.
2995
2996
<P>
2997
Finally, saving only the full path name rather than splitting the path and the
2998
file, and indexing it on the first 50 characters takes 6 mins 43 seconds and
2999
creates a 7.35 MB database.
3000
3001
<P>
3002
3003
<P>
3004
<A NAME="1726"></A>
3005
<TABLE CELLPADDING=3 BORDER="1">
3006
<TR><TD ALIGN="LEFT" COLSPAN=3><B>File </B></TD>
3007
</TR>
3008
<TR><TD ALIGN="CENTER" COLSPAN=1><B>Column Name </B></TD>
3009
<TD ALIGN="CENTER" COLSPAN=1><B>Data Type
3010
</B></TD>
3011
<TD ALIGN="CENTER" COLSPAN=1><B>Remark </B></TD>
3012
</TR>
3013
<TR><TD ALIGN="LEFT">FileId</TD>
3014
<TD ALIGN="LEFT">integer</TD>
3015
<TD ALIGN="LEFT">Primary Key</TD>
3016
</TR>
3017
<TR><TD ALIGN="LEFT">FileIndex</TD>
3018
<TD ALIGN="LEFT">integer</TD>
3019
<TD ALIGN="LEFT">The sequential file number in the Job</TD>
3020
</TR>
3021
<TR><TD ALIGN="LEFT">JobId</TD>
3022
<TD ALIGN="LEFT">integer</TD>
3023
<TD ALIGN="LEFT">Link to Job Record</TD>
3024
</TR>
3025
<TR><TD ALIGN="LEFT">PathId</TD>
3026
<TD ALIGN="LEFT">integer</TD>
3027
<TD ALIGN="LEFT">Link to Path Record</TD>
3028
</TR>
3029
<TR><TD ALIGN="LEFT">FilenameId</TD>
3030
<TD ALIGN="LEFT">integer</TD>
3031
<TD ALIGN="LEFT">Link to Filename Record</TD>
3032
</TR>
3033
<TR><TD ALIGN="LEFT">MarkId</TD>
3034
<TD ALIGN="LEFT">integer</TD>
3035
<TD ALIGN="LEFT">Used to mark files during Verify Jobs</TD>
3036
</TR>
3037
<TR><TD ALIGN="LEFT">LStat</TD>
3038
<TD ALIGN="LEFT">tinyblob</TD>
3039
<TD ALIGN="LEFT">File attributes in base64 encoding</TD>
3040
</TR>
3041
<TR><TD ALIGN="LEFT">MD5</TD>
3042
<TD ALIGN="LEFT">tinyblob</TD>
3043
<TD ALIGN="LEFT">MD5 signature in base64 encoding</TD>
3044
</TR>
3045
</TABLE>
3046
3047
<P>
3048
The <B>File</B> table shown above contains one entry for each file backed up by
3049
Bacula. Thus a file that is backed up multiple times (as is normal) will have
3050
multiple entries in the File table. This will probably be the table with the
3051
most number of records. Consequently, it is essential to keep the size of this
3052
record to an absolute minimum. At the same time, this table must contain all
3053
the information (or pointers to the information) about the file and where it
3054
is backed up. Since a file may be backed up many times without having changed,
3055
the path and filename are stored in separate tables.
3056
3057
<P>
3058
This table contains by far the largest amount of information in the Catalog
3059
database, both from the stand point of number of records, and the stand point
3060
of total database size. As a consequence, the user must take care to
3061
periodically reduce the number of File records using the <B>retention</B>
3062
command in the Console program.
3063
3064
<P>
3065
3066
<P>
3067
<A NAME="1770"></A>
3068
<TABLE CELLPADDING=3 BORDER="1">
3069
<TR><TD ALIGN="LEFT" COLSPAN=3><B>Job </B></TD>
3070
</TR>
3071
<TR><TD ALIGN="CENTER" COLSPAN=1><B>Column Name </B></TD>
3072
<TD ALIGN="CENTER" COLSPAN=1><B>Data Type
3073
</B></TD>
3074
<TD ALIGN="CENTER" COLSPAN=1><B>Remark </B></TD>
3075
</TR>
3076
<TR><TD ALIGN="LEFT">JobId</TD>
3077
<TD ALIGN="LEFT">integer</TD>
3078
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Primary Key</TD>
3079
</TR>
3080
<TR><TD ALIGN="LEFT">Job</TD>
3081
<TD ALIGN="LEFT">tinyblob</TD>
3082
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Unique Job Name</TD>
3083
</TR>
3084
<TR><TD ALIGN="LEFT">Name</TD>
3085
<TD ALIGN="LEFT">tinyblob</TD>
3086
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Job Name</TD>
3087
</TR>
3088
<TR><TD ALIGN="LEFT">PurgedFiles</TD>
3089
<TD ALIGN="LEFT">tinyint</TD>
3090
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Used by Bacula for purging/retention periods</TD>
3091
</TR>
3092
<TR><TD ALIGN="LEFT">Type</TD>
3093
<TD ALIGN="LEFT">binary(1)</TD>
3094
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Job Type: Backup, Copy, Clone, Archive, Migration</TD>
3095
</TR>
3096
<TR><TD ALIGN="LEFT">Level</TD>
3097
<TD ALIGN="LEFT">binary(1)</TD>
3098
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Job Level</TD>
3099
</TR>
3100
<TR><TD ALIGN="LEFT">ClientId</TD>
3101
<TD ALIGN="LEFT">integer</TD>
3102
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Client index</TD>
3103
</TR>
3104
<TR><TD ALIGN="LEFT">JobStatus</TD>
3105
<TD ALIGN="LEFT">binary(1)</TD>
3106
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Job Termination Status</TD>
3107
</TR>
3108
<TR><TD ALIGN="LEFT">SchedTime</TD>
3109
<TD ALIGN="LEFT">datetime</TD>
3110
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Time/date when Job scheduled</TD>
3111
</TR>
3112
<TR><TD ALIGN="LEFT">StartTime</TD>
3113
<TD ALIGN="LEFT">datetime</TD>
3114
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Time/date when Job started</TD>
3115
</TR>
3116
<TR><TD ALIGN="LEFT">EndTime</TD>
3117
<TD ALIGN="LEFT">datetime</TD>
3118
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Time/date when Job ended</TD>
3119
</TR>
3120
<TR><TD ALIGN="LEFT">JobTDate</TD>
3121
<TD ALIGN="LEFT">bigint</TD>
3122
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Start day in Unix format but 64 bits; used for
3123
Retention period.</TD>
3124
</TR>
3125
<TR><TD ALIGN="LEFT">VolSessionId</TD>
3126
<TD ALIGN="LEFT">integer</TD>
3127
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Unique Volume Session ID</TD>
3128
</TR>
3129
<TR><TD ALIGN="LEFT">VolSessionTime</TD>
3130
<TD ALIGN="LEFT">integer</TD>
3131
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Unique Volume Session Time</TD>
3132
</TR>
3133
<TR><TD ALIGN="LEFT">JobFiles</TD>
3134
<TD ALIGN="LEFT">integer</TD>
3135
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Number of files saved in Job</TD>
3136
</TR>
3137
<TR><TD ALIGN="LEFT">JobBytes</TD>
3138
<TD ALIGN="LEFT">bigint</TD>
3139
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Number of bytes saved in Job</TD>
3140
</TR>
3141
<TR><TD ALIGN="LEFT">JobErrors</TD>
3142
<TD ALIGN="LEFT">integer</TD>
3143
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Number of errors during Job</TD>
3144
</TR>
3145
<TR><TD ALIGN="LEFT">JobMissingFiles</TD>
3146
<TD ALIGN="LEFT">integer</TD>
3147
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Number of files not saved (not yet used)</TD>
3148
</TR>
3149
<TR><TD ALIGN="LEFT">PoolId</TD>
3150
<TD ALIGN="LEFT">integer</TD>
3151
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Link to Pool Record</TD>
3152
</TR>
3153
<TR><TD ALIGN="LEFT">FileSetId</TD>
3154
<TD ALIGN="LEFT">integer</TD>
3155
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Link to FileSet Record</TD>
3156
</TR>
3157
<TR><TD ALIGN="LEFT">PurgedFiles</TD>
3158
<TD ALIGN="LEFT">tiny integer</TD>
3159
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Set when all File records purged</TD>
3160
</TR>
3161
<TR><TD ALIGN="LEFT">HasBase</TD>
3162
<TD ALIGN="LEFT">tiny integer</TD>
3163
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Set when Base Job run</TD>
3164
</TR>
3165
</TABLE>
3166
3167
<P>
3168
The <B>Job</B> table contains one record for each Job run by Bacula. Thus
3169
normally, there will be one per day per machine added to the database. Note,
3170
the JobId is used to index Job records in the database, and it often is shown
3171
to the user in the Console program. However, care must be taken with its use
3172
as it is not unique from database to database. For example, the user may have
3173
a database for Client data saved on machine Rufus and another database for
3174
Client data saved on machine Roxie. In this case, the two database will each
3175
have JobIds that match those in another database. For a unique reference to a
3176
Job, see Job below.
3177
3178
<P>
3179
The Name field of the Job record corresponds to the Name resource record given
3180
in the Director's configuration file. Thus it is a generic name, and it will
3181
be normal to find many Jobs (or even all Jobs) with the same Name.
3182
3183
<P>
3184
The Job field contains a combination of the Name and the schedule time of the
3185
Job by the Director. Thus for a given Director, even with multiple Catalog
3186
databases, the Job will contain a unique name that represents the Job.
3187
3188
<P>
3189
For a given Storage daemon, the VolSessionId and VolSessionTime form a unique
3190
identification of the Job. This will be the case even if multiple Directors
3191
are using the same Storage daemon.
3192
3193
<P>
3194
The Job Type (or simply Type) can have one of the following values:
3195
3196
<P>
3197
<A NAME="1855"></A>
3198
<TABLE CELLPADDING=3 BORDER="1">
3199
<TR><TD ALIGN="CENTER" COLSPAN=1><B>Value </B></TD>
3200
<TD ALIGN="CENTER" COLSPAN=1><B>Meaning </B></TD>
3201
</TR>
3202
<TR><TD ALIGN="LEFT">B</TD>
3203
<TD ALIGN="LEFT">Backup Job</TD>
3204
</TR>
3205
<TR><TD ALIGN="LEFT">V</TD>
3206
<TD ALIGN="LEFT">Verify Job</TD>
3207
</TR>
3208
<TR><TD ALIGN="LEFT">R</TD>
3209
<TD ALIGN="LEFT">Restore Job</TD>
3210
</TR>
3211
<TR><TD ALIGN="LEFT">C</TD>
3212
<TD ALIGN="LEFT">Console program (not in database)</TD>
3213
</TR>
3214
<TR><TD ALIGN="LEFT">D</TD>
3215
<TD ALIGN="LEFT">Admin Job</TD>
3216
</TR>
3217
<TR><TD ALIGN="LEFT">A</TD>
3218
<TD ALIGN="LEFT">Archive Job (not implemented)</TD>
3219
</TR>
3220
</TABLE>
3221
3222
<P>
3223
The JobStatus field specifies how the job terminated, and can be one of the
3224
following:
3225
3226
<P>
3227
<A NAME="1879"></A>
3228
<TABLE CELLPADDING=3 BORDER="1">
3229
<TR><TD ALIGN="CENTER" COLSPAN=1><B>Value </B></TD>
3230
<TD ALIGN="CENTER" COLSPAN=1><B>Meaning </B></TD>
3231
</TR>
3232
<TR><TD ALIGN="LEFT">C</TD>
3233
<TD ALIGN="LEFT">Created but not yet running</TD>
3234
</TR>
3235
<TR><TD ALIGN="LEFT">R</TD>
3236
<TD ALIGN="LEFT">Running</TD>
3237
</TR>
3238
<TR><TD ALIGN="LEFT">B</TD>
3239
<TD ALIGN="LEFT">Blocked</TD>
3240
</TR>
3241
<TR><TD ALIGN="LEFT">T</TD>
3242
<TD ALIGN="LEFT">Terminated normally</TD>
3243
</TR>
3244
<TR><TD ALIGN="LEFT">E</TD>
3245
<TD ALIGN="LEFT">Terminated in Error</TD>
3246
</TR>
3247
<TR><TD ALIGN="LEFT">e</TD>
3248
<TD ALIGN="LEFT">Non-fatal error</TD>
3249
</TR>
3250
<TR><TD ALIGN="LEFT">f</TD>
3251
<TD ALIGN="LEFT">Fatal error</TD>
3252
</TR>
3253
<TR><TD ALIGN="LEFT">D</TD>
3254
<TD ALIGN="LEFT">Verify Differences</TD>
3255
</TR>
3256
<TR><TD ALIGN="LEFT">A</TD>
3257
<TD ALIGN="LEFT">Canceled by the user</TD>
3258
</TR>
3259
<TR><TD ALIGN="LEFT">F</TD>
3260
<TD ALIGN="LEFT">Waiting on the File daemon</TD>
3261
</TR>
3262
<TR><TD ALIGN="LEFT">S</TD>
3263
<TD ALIGN="LEFT">Waiting on the Storage daemon</TD>
3264
</TR>
3265
<TR><TD ALIGN="LEFT">m</TD>
3266
<TD ALIGN="LEFT">Waiting for a new Volume to be mounted</TD>
3267
</TR>
3268
<TR><TD ALIGN="LEFT">M</TD>
3269
<TD ALIGN="LEFT">Waiting for a Mount</TD>
3270
</TR>
3271
<TR><TD ALIGN="LEFT">s</TD>
3272
<TD ALIGN="LEFT">Waiting for Storage resource</TD>
3273
</TR>
3274
<TR><TD ALIGN="LEFT">j</TD>
3275
<TD ALIGN="LEFT">Waiting for Job resource</TD>
3276
</TR>
3277
<TR><TD ALIGN="LEFT">c</TD>
3278
<TD ALIGN="LEFT">Waiting for Client resource</TD>
3279
</TR>
3280
<TR><TD ALIGN="LEFT">d</TD>
3281
<TD ALIGN="LEFT">Wating for Maximum jobs</TD>
3282
</TR>
3283
<TR><TD ALIGN="LEFT">t</TD>
3284
<TD ALIGN="LEFT">Waiting for Start Time</TD>
3285
</TR>
3286
<TR><TD ALIGN="LEFT">p</TD>
3287
<TD ALIGN="LEFT">Waiting for higher priority job to finish</TD>
3288
</TR>
3289
</TABLE>
3290
3291
<P>
3292
3293
<P>
3294
<A NAME="1929"></A>
3295
<TABLE CELLPADDING=3 BORDER="1">
3296
<TR><TD ALIGN="LEFT" COLSPAN=3><B>FileSet </B></TD>
3297
</TR>
3298
<TR><TD ALIGN="CENTER" COLSPAN=1><B>Column Name </B></TD>
3299
<TD ALIGN="CENTER" COLSPAN=1><B>Data Type </B></TD>
3300
<TD ALIGN="CENTER" COLSPAN=1><B>Remark </B></TD>
3301
</TR>
3302
<TR><TD ALIGN="LEFT">FileSetId</TD>
3303
<TD ALIGN="LEFT">integer</TD>
3304
<TD ALIGN="LEFT">Primary Key</TD>
3305
</TR>
3306
<TR><TD ALIGN="LEFT">FileSet</TD>
3307
<TD ALIGN="LEFT">tinyblob</TD>
3308
<TD ALIGN="LEFT">FileSet name</TD>
3309
</TR>
3310
<TR><TD ALIGN="LEFT">MD5</TD>
3311
<TD ALIGN="LEFT">tinyblob</TD>
3312
<TD ALIGN="LEFT">MD5 checksum of FileSet</TD>
3313
</TR>
3314
<TR><TD ALIGN="LEFT">CreateTime</TD>
3315
<TD ALIGN="LEFT">datetime</TD>
3316
<TD ALIGN="LEFT">Time and date Fileset created</TD>
3317
</TR>
3318
</TABLE>
3319
3320
<P>
3321
The <B>FileSet</B> table contains one entry for each FileSet that is used. The
3322
MD5 signature is kept to ensure that if the user changes anything inside the
3323
FileSet, it will be detected and the new FileSet will be used. This is
3324
particularly important when doing an incremental update. If the user deletes a
3325
file or adds a file, we need to ensure that a Full backup is done prior to the
3326
next incremental.
3327
3328
<P>
3329
3330
<P>
3331
<A NAME="1960"></A>
3332
<TABLE CELLPADDING=3 BORDER="1">
3333
<TR><TD ALIGN="LEFT" COLSPAN=3><B>JobMedia </B></TD>
3334
</TR>
3335
<TR><TD ALIGN="CENTER" COLSPAN=1><B>Column Name </B></TD>
3336
<TD ALIGN="CENTER" COLSPAN=1><B>Data Type </B></TD>
3337
<TD ALIGN="CENTER" COLSPAN=1><B>Remark </B></TD>
3338
</TR>
3339
<TR><TD ALIGN="LEFT">JobMediaId</TD>
3340
<TD ALIGN="LEFT">integer</TD>
3341
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Primary Key</TD>
3342
</TR>
3343
<TR><TD ALIGN="LEFT">JobId</TD>
3344
<TD ALIGN="LEFT">integer</TD>
3345
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Link to Job Record</TD>
3346
</TR>
3347
<TR><TD ALIGN="LEFT">MediaId</TD>
3348
<TD ALIGN="LEFT">integer</TD>
3349
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>Link to Media Record</TD>
3350
</TR>
3351
<TR><TD ALIGN="LEFT">FirstIndex</TD>
3352
<TD ALIGN="LEFT">integer</TD>
3353
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>The index (sequence number) of the first file
3354
written for this Job to the Media</TD>
3355
</TR>
3356
<TR><TD ALIGN="LEFT">LastIndex</TD>
3357
<TD ALIGN="LEFT">integer</TD>
3358
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>The index of the last file written for this
3359
Job to the Media</TD>
3360
</TR>
3361
<TR><TD ALIGN="LEFT">StartFile</TD>
3362
<TD ALIGN="LEFT">integer</TD>
3363
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>The physical media (tape) file number of the
3364
first block written for this Job</TD>
3365
</TR>
3366
<TR><TD ALIGN="LEFT">EndFile</TD>
3367
<TD ALIGN="LEFT">integer</TD>
3368
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>The physical media (tape) file number of the
3369
last block written for this Job</TD>
3370
</TR>
3371
<TR><TD ALIGN="LEFT">StartBlock</TD>
3372
<TD ALIGN="LEFT">integer</TD>
3373
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>The number of the first block written for
3374
this Job</TD>
3375
</TR>
3376
<TR><TD ALIGN="LEFT">EndBlock</TD>
3377
<TD ALIGN="LEFT">integer</TD>
3378
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>The number of the last block written for this
3379
Job</TD>
3380
</TR>
3381
<TR><TD ALIGN="LEFT">VolIndex</TD>
3382
<TD ALIGN="LEFT">integer</TD>
3383
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=180>The Volume use sequence number within the Job</TD>
3384
</TR>
3385
</TABLE>
3386
3387
<P>
3388
The <B>JobMedia</B> table contains one entry at the following: start of
3389
the job, start of each new tape file, start of each new tape, end of the
3390
job. Since by default, a new tape file is written every 2GB, in general,
3391
you will have more than 2 JobMedia records per Job. The number can be
3392
varied by changing the "Maximum File Size" specified in the Device
3393
resource. This record allows Bacula to efficiently position close to
3394
(within 2GB) any given file in a backup. For restoring a full Job,
3395
these records are not very important, but if you want to retrieve
3396
a single file that was written near the end of a 100GB backup, the
3397
JobMedia records can speed it up by orders of magnitude by permitting
3398
forward spacing files and blocks rather than reading the whole 100GB
3399
backup.
3400
3401
<P>
3402
3403
<P>
3404
<A NAME="2009"></A>
3405
<TABLE CELLPADDING=3 BORDER="1">
3406
<TR><TD ALIGN="LEFT" COLSPAN=3><B>Media </B></TD>
3407
</TR>
3408
<TR><TD ALIGN="CENTER" COLSPAN=1><B>Column Name </B></TD>
3409
<TD ALIGN="CENTER" COLSPAN=1><B>Data Type </B></TD>
3410
<TD ALIGN="CENTER" COLSPAN=1><B>Remark </B></TD>
3411
</TR>
3412
<TR><TD ALIGN="LEFT">MediaId</TD>
3413
<TD ALIGN="LEFT">integer</TD>
3414
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Primary Key</TD>
3415
</TR>
3416
<TR><TD ALIGN="LEFT">VolumeName</TD>
3417
<TD ALIGN="LEFT">tinyblob</TD>
3418
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Volume name</TD>
3419
</TR>
3420
<TR><TD ALIGN="LEFT">Slot</TD>
3421
<TD ALIGN="LEFT">integer</TD>
3422
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Autochanger Slot number or zero</TD>
3423
</TR>
3424
<TR><TD ALIGN="LEFT">PoolId</TD>
3425
<TD ALIGN="LEFT">integer</TD>
3426
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Link to Pool Record</TD>
3427
</TR>
3428
<TR><TD ALIGN="LEFT">MediaType</TD>
3429
<TD ALIGN="LEFT">tinyblob</TD>
3430
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>The MediaType supplied by the user</TD>
3431
</TR>
3432
<TR><TD ALIGN="LEFT">FirstWritten</TD>
3433
<TD ALIGN="LEFT">datetime</TD>
3434
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Time/date when first written</TD>
3435
</TR>
3436
<TR><TD ALIGN="LEFT">LastWritten</TD>
3437
<TD ALIGN="LEFT">datetime</TD>
3438
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Time/date when last written</TD>
3439
</TR>
3440
<TR><TD ALIGN="LEFT">LabelDate</TD>
3441
<TD ALIGN="LEFT">datetime</TD>
3442
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Time/date when tape labeled</TD>
3443
</TR>
3444
<TR><TD ALIGN="LEFT">VolJobs</TD>
3445
<TD ALIGN="LEFT">integer</TD>
3446
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Number of jobs written to this media</TD>
3447
</TR>
3448
<TR><TD ALIGN="LEFT">VolFiles</TD>
3449
<TD ALIGN="LEFT">integer</TD>
3450
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Number of files written to this media</TD>
3451
</TR>
3452
<TR><TD ALIGN="LEFT">VolBlocks</TD>
3453
<TD ALIGN="LEFT">integer</TD>
3454
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Number of blocks written to this media</TD>
3455
</TR>
3456
<TR><TD ALIGN="LEFT">VolMounts</TD>
3457
<TD ALIGN="LEFT">integer</TD>
3458
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Number of time media mounted</TD>
3459
</TR>
3460
<TR><TD ALIGN="LEFT">VolBytes</TD>
3461
<TD ALIGN="LEFT">bigint</TD>
3462
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Number of bytes saved in Job</TD>
3463
</TR>
3464
<TR><TD ALIGN="LEFT">VolErrors</TD>
3465
<TD ALIGN="LEFT">integer</TD>
3466
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Number of errors during Job</TD>
3467
</TR>
3468
<TR><TD ALIGN="LEFT">VolWrites</TD>
3469
<TD ALIGN="LEFT">integer</TD>
3470
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Number of writes to media</TD>
3471
</TR>
3472
<TR><TD ALIGN="LEFT">MaxVolBytes</TD>
3473
<TD ALIGN="LEFT">bigint</TD>
3474
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Maximum bytes to put on this media</TD>
3475
</TR>
3476
<TR><TD ALIGN="LEFT">VolCapacityBytes</TD>
3477
<TD ALIGN="LEFT">bigint</TD>
3478
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Capacity estimate for this volume</TD>
3479
</TR>
3480
<TR><TD ALIGN="LEFT">VolStatus</TD>
3481
<TD ALIGN="LEFT">enum</TD>
3482
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Status of media: Full, Archive, Append, Recycle,
3483
Read-Only, Disabled, Error, Busy</TD>
3484
</TR>
3485
<TR><TD ALIGN="LEFT">Recycle</TD>
3486
<TD ALIGN="LEFT">tinyint</TD>
3487
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Whether or not Bacula can recycle the Volumes:
3488
Yes, No</TD>
3489
</TR>
3490
<TR><TD ALIGN="LEFT">VolRetention</TD>
3491
<TD ALIGN="LEFT">bigint</TD>
3492
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>64 bit seconds until expiration</TD>
3493
</TR>
3494
<TR><TD ALIGN="LEFT">VolUseDuration</TD>
3495
<TD ALIGN="LEFT">bigint</TD>
3496
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>64 bit seconds volume can be used</TD>
3497
</TR>
3498
<TR><TD ALIGN="LEFT">MaxVolJobs</TD>
3499
<TD ALIGN="LEFT">integer</TD>
3500
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>maximum jobs to put on Volume</TD>
3501
</TR>
3502
<TR><TD ALIGN="LEFT">MaxVolFiles</TD>
3503
<TD ALIGN="LEFT">integer</TD>
3504
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>maximume EOF marks to put on Volume</TD>
3505
</TR>
3506
</TABLE>
3507
3508
<P>
3509
The <B>Volume</B> table (internally referred to as the Media table) contains
3510
one entry for each volume, that is each tape, cassette (8mm, DLT, DAT, ...),
3511
or file on which information is or was backed up. There is one Volume record
3512
created for each of the NumVols specified in the Pool resource record.
3513
3514
<P>
3515
3516
<P>
3517
<A NAME="2097"></A>
3518
<TABLE CELLPADDING=3 BORDER="1">
3519
<TR><TD ALIGN="LEFT" COLSPAN=3><B>Pool </B></TD>
3520
</TR>
3521
<TR><TD ALIGN="CENTER" COLSPAN=1><B>Column Name </B></TD>
3522
<TD ALIGN="CENTER" COLSPAN=1><B>Data Type
3523
</B></TD>
3524
<TD ALIGN="CENTER" COLSPAN=1><B>Remark </B></TD>
3525
</TR>
3526
<TR><TD ALIGN="LEFT">PoolId</TD>
3527
<TD ALIGN="LEFT">integer</TD>
3528
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Primary Key</TD>
3529
</TR>
3530
<TR><TD ALIGN="LEFT">Name</TD>
3531
<TD ALIGN="LEFT">Tinyblob</TD>
3532
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Pool Name</TD>
3533
</TR>
3534
<TR><TD ALIGN="LEFT">NumVols</TD>
3535
<TD ALIGN="LEFT">Integer</TD>
3536
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Number of Volumes in the Pool</TD>
3537
</TR>
3538
<TR><TD ALIGN="LEFT">MaxVols</TD>
3539
<TD ALIGN="LEFT">Integer</TD>
3540
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Maximum Volumes in the Pool</TD>
3541
</TR>
3542
<TR><TD ALIGN="LEFT">UseOnce</TD>
3543
<TD ALIGN="LEFT">tinyint</TD>
3544
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Use volume once</TD>
3545
</TR>
3546
<TR><TD ALIGN="LEFT">UseCatalog</TD>
3547
<TD ALIGN="LEFT">tinyint</TD>
3548
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Set to use catalog</TD>
3549
</TR>
3550
<TR><TD ALIGN="LEFT">AcceptAnyVolume</TD>
3551
<TD ALIGN="LEFT">tinyint</TD>
3552
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Accept any volume from Pool</TD>
3553
</TR>
3554
<TR><TD ALIGN="LEFT">VolRetention</TD>
3555
<TD ALIGN="LEFT">bigint</TD>
3556
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>64 bit seconds to retain volume</TD>
3557
</TR>
3558
<TR><TD ALIGN="LEFT">VolUseDuration</TD>
3559
<TD ALIGN="LEFT">bigint</TD>
3560
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>64 bit seconds volume can be used</TD>
3561
</TR>
3562
<TR><TD ALIGN="LEFT">MaxVolJobs</TD>
3563
<TD ALIGN="LEFT">integer</TD>
3564
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>max jobs on volume</TD>
3565
</TR>
3566
<TR><TD ALIGN="LEFT">MaxVolFiles</TD>
3567
<TD ALIGN="LEFT">integer</TD>
3568
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>max EOF marks to put on Volume</TD>
3569
</TR>
3570
<TR><TD ALIGN="LEFT">MaxVolBytes</TD>
3571
<TD ALIGN="LEFT">bigint</TD>
3572
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>max bytes to write on Volume</TD>
3573
</TR>
3574
<TR><TD ALIGN="LEFT">AutoPrune</TD>
3575
<TD ALIGN="LEFT">tinyint</TD>
3576
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>yes|no for autopruning</TD>
3577
</TR>
3578
<TR><TD ALIGN="LEFT">Recycle</TD>
3579
<TD ALIGN="LEFT">tinyint</TD>
3580
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>yes|no for allowing auto recycling of Volume</TD>
3581
</TR>
3582
<TR><TD ALIGN="LEFT">PoolType</TD>
3583
<TD ALIGN="LEFT">enum</TD>
3584
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Backup, Copy, Cloned, Archive, Migration</TD>
3585
</TR>
3586
<TR><TD ALIGN="LEFT">LabelFormat</TD>
3587
<TD ALIGN="LEFT">Tinyblob</TD>
3588
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=173>Label format</TD>
3589
</TR>
3590
</TABLE>
3591
3592
<P>
3593
The <B>Pool</B> table contains one entry for each media pool controlled by
3594
Bacula in this database. One media record exists for each of the NumVols
3595
contained in the Pool. The PoolType is a Bacula defined keyword. The MediaType
3596
is defined by the administrator, and corresponds to the MediaType specified in
3597
the Director's Storage definition record. The CurrentVol is the sequence
3598
number of the Media record for the current volume.
3599
3600
<P>
3601
3602
<P>
3603
<A NAME="2164"></A>
3604
<TABLE CELLPADDING=3 BORDER="1">
3605
<TR><TD ALIGN="LEFT" COLSPAN=3><B>Client </B></TD>
3606
</TR>
3607
<TR><TD ALIGN="CENTER" COLSPAN=1><B>Column Name </B></TD>
3608
<TD ALIGN="CENTER" COLSPAN=1><B>Data Type
3609
</B></TD>
3610
<TD ALIGN="CENTER" COLSPAN=1><B>Remark </B></TD>
3611
</TR>
3612
<TR><TD ALIGN="LEFT">ClientId</TD>
3613
<TD ALIGN="LEFT">integer</TD>
3614
<TD ALIGN="LEFT">Primary Key</TD>
3615
</TR>
3616
<TR><TD ALIGN="LEFT">Name</TD>
3617
<TD ALIGN="LEFT">TinyBlob</TD>
3618
<TD ALIGN="LEFT">File Services Name</TD>
3619
</TR>
3620
<TR><TD ALIGN="LEFT">UName</TD>
3621
<TD ALIGN="LEFT">TinyBlob</TD>
3622
<TD ALIGN="LEFT">uname -a from Client (not yet used)</TD>
3623
</TR>
3624
<TR><TD ALIGN="LEFT">AutoPrune</TD>
3625
<TD ALIGN="LEFT">tinyint</TD>
3626
<TD ALIGN="LEFT">yes|no for autopruning</TD>
3627
</TR>
3628
<TR><TD ALIGN="LEFT">FileRetention</TD>
3629
<TD ALIGN="LEFT">bigint</TD>
3630
<TD ALIGN="LEFT">64 bit seconds to retain Files</TD>
3631
</TR>
3632
<TR><TD ALIGN="LEFT">JobRetention</TD>
3633
<TD ALIGN="LEFT">bigint</TD>
3634
<TD ALIGN="LEFT">64 bit seconds to retain Job</TD>
3635
</TR>
3636
</TABLE>
3637
3638
<P>
3639
The <B>Client</B> table contains one entry for each machine backed up by Bacula
3640
in this database. Normally the Name is a fully qualified domain name.
3641
3642
<P>
3643
3644
<P>
3645
<A NAME="2201"></A>
3646
<TABLE CELLPADDING=3 BORDER="1">
3647
<TR><TD ALIGN="LEFT" COLSPAN=3><B>UnsavedFiles </B></TD>
3648
</TR>
3649
<TR><TD ALIGN="CENTER" COLSPAN=1><B>Column Name </B></TD>
3650
<TD ALIGN="CENTER" COLSPAN=1><B>Data Type
3651
</B></TD>
3652
<TD ALIGN="CENTER" COLSPAN=1><B>Remark </B></TD>
3653
</TR>
3654
<TR><TD ALIGN="LEFT">UnsavedId</TD>
3655
<TD ALIGN="LEFT">integer</TD>
3656
<TD ALIGN="LEFT">Primary Key</TD>
3657
</TR>
3658
<TR><TD ALIGN="LEFT">JobId</TD>
3659
<TD ALIGN="LEFT">integer</TD>
3660
<TD ALIGN="LEFT">JobId corresponding to this record</TD>
3661
</TR>
3662
<TR><TD ALIGN="LEFT">PathId</TD>
3663
<TD ALIGN="LEFT">integer</TD>
3664
<TD ALIGN="LEFT">Id of path</TD>
3665
</TR>
3666
<TR><TD ALIGN="LEFT">FilenameId</TD>
3667
<TD ALIGN="LEFT">integer</TD>
3668
<TD ALIGN="LEFT">Id of filename</TD>
3669
</TR>
3670
</TABLE>
3671
3672
<P>
3673
The <B>UnsavedFiles</B> table contains one entry for each file that was not
3674
saved. Note! This record is not yet implemented.
3675
3676
<P>
3677
3678
<P>
3679
<A NAME="2232"></A>
3680
<TABLE CELLPADDING=3 BORDER="1">
3681
<TR><TD ALIGN="LEFT" COLSPAN=3><B>Counter </B></TD>
3682
</TR>
3683
<TR><TD ALIGN="CENTER" COLSPAN=1><B>Column Name </B></TD>
3684
<TD ALIGN="CENTER" COLSPAN=1><B>Data Type
3685
</B></TD>
3686
<TD ALIGN="CENTER" COLSPAN=1><B>Remark </B></TD>
3687
</TR>
3688
<TR><TD ALIGN="LEFT">Counter</TD>
3689
<TD ALIGN="LEFT">tinyblob</TD>
3690
<TD ALIGN="LEFT">Counter name</TD>
3691
</TR>
3692
<TR><TD ALIGN="LEFT">MinValue</TD>
3693
<TD ALIGN="LEFT">integer</TD>
3694
<TD ALIGN="LEFT">Start/Min value for counter</TD>
3695
</TR>
3696
<TR><TD ALIGN="LEFT">MaxValue</TD>
3697
<TD ALIGN="LEFT">integer</TD>
3698
<TD ALIGN="LEFT">Max value for counter</TD>
3699
</TR>
3700
<TR><TD ALIGN="LEFT">CurrentValue</TD>
3701
<TD ALIGN="LEFT">integer</TD>
3702
<TD ALIGN="LEFT">Current counter value</TD>
3703
</TR>
3704
<TR><TD ALIGN="LEFT">WrapCounter</TD>
3705
<TD ALIGN="LEFT">tinyblob</TD>
3706
<TD ALIGN="LEFT">Name of another counter</TD>
3707
</TR>
3708
</TABLE>
3709
3710
<P>
3711
The <B>Counter</B> table contains one entry for each permanent counter defined
3712
by the user.
3713
3714
<P>
3715
3716
<P>
3717
<A NAME="2266"></A>
3718
<TABLE CELLPADDING=3 BORDER="1">
3719
<TR><TD ALIGN="LEFT" COLSPAN=3><B>Version </B></TD>
3720
</TR>
3721
<TR><TD ALIGN="CENTER" COLSPAN=1><B>Column Name </B></TD>
3722
<TD ALIGN="CENTER" COLSPAN=1><B>Data Type
3723
</B></TD>
3724
<TD ALIGN="CENTER" COLSPAN=1><B>Remark </B></TD>
3725
</TR>
3726
<TR><TD ALIGN="LEFT">VersionId</TD>
3727
<TD ALIGN="LEFT">integer</TD>
3728
<TD ALIGN="LEFT">Primary Key</TD>
3729
</TR>
3730
</TABLE>
3731
3732
<P>
3733
The <B>Version</B> table defines the Bacula database version number. Bacula
3734
checks this number before reading the database to ensure that it is compatible
3735
with the Bacula binary file.
3736
3737
<P>
3738
3739
<P>
3740
<A NAME="2288"></A>
3741
<TABLE CELLPADDING=3 BORDER="1">
3742
<TR><TD ALIGN="LEFT" COLSPAN=3><B>BaseFiles </B></TD>
3743
</TR>
3744
<TR><TD ALIGN="CENTER" COLSPAN=1><B>Column Name </B></TD>
3745
<TD ALIGN="CENTER" COLSPAN=1><B>Data Type
3746
</B></TD>
3747
<TD ALIGN="CENTER" COLSPAN=1><B>Remark </B></TD>
3748
</TR>
3749
<TR><TD ALIGN="LEFT">BaseId</TD>
3750
<TD ALIGN="LEFT">integer</TD>
3751
<TD ALIGN="LEFT">Primary Key</TD>
3752
</TR>
3753
<TR><TD ALIGN="LEFT">BaseJobId</TD>
3754
<TD ALIGN="LEFT">integer</TD>
3755
<TD ALIGN="LEFT">JobId of Base Job</TD>
3756
</TR>
3757
<TR><TD ALIGN="LEFT">JobId</TD>
3758
<TD ALIGN="LEFT">integer</TD>
3759
<TD ALIGN="LEFT">Reference to Job</TD>
3760
</TR>
3761
<TR><TD ALIGN="LEFT">FileId</TD>
3762
<TD ALIGN="LEFT">integer</TD>
3763
<TD ALIGN="LEFT">Reference to File</TD>
3764
</TR>
3765
<TR><TD ALIGN="LEFT">FileIndex</TD>
3766
<TD ALIGN="LEFT">integer</TD>
3767
<TD ALIGN="LEFT">File Index number</TD>
3768
</TR>
3769
</TABLE>
3770
3771
<P>
3772
The <B>BaseFiles</B> table contains all the File references for a particular
3773
JobId that point to a Base file -- i.e. they were previously saved and hence
3774
were not saved in the current JobId but in BaseJobId under FileId. FileIndex
3775
is the index of the file, and is used for optimization of Restore jobs to
3776
prevent the need to read the FileId record when creating the in memory tree.
3777
This record is not yet implemented.
3778
3779
<P>
3780
3781
<P>
3782
3783
<H3><A NAME="SECTION000103100000000000000">
3784
MySQL Table Definition</A>
3785
</H3>
3786
<A NAME="2321"></A>
3787
<A NAME="2322"></A>
3788
<A NAME="2325"></A>
3789
3790
<P>
3791
The commands used to create the MySQL tables are as follows:
3792
3793
<P>
3794
<PRE>
3795
USE bacula;
3796
CREATE TABLE Filename (
3797
FilenameId INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
3798
Name BLOB NOT NULL,
3799
PRIMARY KEY(FilenameId),
3800
INDEX (Name(30))
3801
);
3802
CREATE TABLE Path (
3803
PathId INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
3804
Path BLOB NOT NULL,
3805
PRIMARY KEY(PathId),
3806
INDEX (Path(50))
3807
);
3808
CREATE TABLE File (
3809
FileId INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
3810
FileIndex INTEGER UNSIGNED NOT NULL DEFAULT 0,
3811
JobId INTEGER UNSIGNED NOT NULL REFERENCES Job,
3812
PathId INTEGER UNSIGNED NOT NULL REFERENCES Path,
3813
FilenameId INTEGER UNSIGNED NOT NULL REFERENCES Filename,
3814
MarkId INTEGER UNSIGNED NOT NULL DEFAULT 0,
3815
LStat TINYBLOB NOT NULL,
3816
MD5 TINYBLOB NOT NULL,
3817
PRIMARY KEY(FileId),
3818
INDEX (JobId),
3819
INDEX (PathId),
3820
INDEX (FilenameId)
3821
);
3822
CREATE TABLE Job (
3823
JobId INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
3824
Job TINYBLOB NOT NULL,
3825
Name TINYBLOB NOT NULL,
3826
Type BINARY(1) NOT NULL,
3827
Level BINARY(1) NOT NULL,
3828
ClientId INTEGER NOT NULL REFERENCES Client,
3829
JobStatus BINARY(1) NOT NULL,
3830
SchedTime DATETIME NOT NULL,
3831
StartTime DATETIME NOT NULL,
3832
EndTime DATETIME NOT NULL,
3833
JobTDate BIGINT UNSIGNED NOT NULL,
3834
VolSessionId INTEGER UNSIGNED NOT NULL DEFAULT 0,
3835
VolSessionTime INTEGER UNSIGNED NOT NULL DEFAULT 0,
3836
JobFiles INTEGER UNSIGNED NOT NULL DEFAULT 0,
3837
JobBytes BIGINT UNSIGNED NOT NULL,
3838
JobErrors INTEGER UNSIGNED NOT NULL DEFAULT 0,
3839
JobMissingFiles INTEGER UNSIGNED NOT NULL DEFAULT 0,
3840
PoolId INTEGER UNSIGNED NOT NULL REFERENCES Pool,
3841
FileSetId INTEGER UNSIGNED NOT NULL REFERENCES FileSet,
3842
PurgedFiles TINYINT NOT NULL DEFAULT 0,
3843
HasBase TINYINT NOT NULL DEFAULT 0,
3844
PRIMARY KEY(JobId),
3845
INDEX (Name(128))
3846
);
3847
CREATE TABLE FileSet (
3848
FileSetId INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
3849
FileSet TINYBLOB NOT NULL,
3850
MD5 TINYBLOB NOT NULL,
3851
CreateTime DATETIME NOT NULL,
3852
PRIMARY KEY(FileSetId)
3853
);
3854
CREATE TABLE JobMedia (
3855
JobMediaId INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
3856
JobId INTEGER UNSIGNED NOT NULL REFERENCES Job,
3857
MediaId INTEGER UNSIGNED NOT NULL REFERENCES Media,
3858
FirstIndex INTEGER UNSIGNED NOT NULL DEFAULT 0,
3859
LastIndex INTEGER UNSIGNED NOT NULL DEFAULT 0,
3860
StartFile INTEGER UNSIGNED NOT NULL DEFAULT 0,
3861
EndFile INTEGER UNSIGNED NOT NULL DEFAULT 0,
3862
StartBlock INTEGER UNSIGNED NOT NULL DEFAULT 0,
3863
EndBlock INTEGER UNSIGNED NOT NULL DEFAULT 0,
3864
VolIndex INTEGER UNSIGNED NOT NULL DEFAULT 0,
3865
PRIMARY KEY(JobMediaId),
3866
INDEX (JobId, MediaId)
3867
);
3868
CREATE TABLE Media (
3869
MediaId INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
3870
VolumeName TINYBLOB NOT NULL,
3871
Slot INTEGER NOT NULL DEFAULT 0,
3872
PoolId INTEGER UNSIGNED NOT NULL REFERENCES Pool,
3873
MediaType TINYBLOB NOT NULL,
3874
FirstWritten DATETIME NOT NULL,
3875
LastWritten DATETIME NOT NULL,
3876
LabelDate DATETIME NOT NULL,
3877
VolJobs INTEGER UNSIGNED NOT NULL DEFAULT 0,
3878
VolFiles INTEGER UNSIGNED NOT NULL DEFAULT 0,
3879
VolBlocks INTEGER UNSIGNED NOT NULL DEFAULT 0,
3880
VolMounts INTEGER UNSIGNED NOT NULL DEFAULT 0,
3881
VolBytes BIGINT UNSIGNED NOT NULL DEFAULT 0,
3882
VolErrors INTEGER UNSIGNED NOT NULL DEFAULT 0,
3883
VolWrites INTEGER UNSIGNED NOT NULL DEFAULT 0,
3884
VolCapacityBytes BIGINT UNSIGNED NOT NULL,
3885
VolStatus ENUM('Full', 'Archive', 'Append', 'Recycle', 'Purged',
3886
'Read-Only', 'Disabled', 'Error', 'Busy', 'Used', 'Cleaning') NOT NULL,
3887
Recycle TINYINT NOT NULL DEFAULT 0,
3888
VolRetention BIGINT UNSIGNED NOT NULL DEFAULT 0,
3889
VolUseDuration BIGINT UNSIGNED NOT NULL DEFAULT 0,
3890
MaxVolJobs INTEGER UNSIGNED NOT NULL DEFAULT 0,
3891
MaxVolFiles INTEGER UNSIGNED NOT NULL DEFAULT 0,
3892
MaxVolBytes BIGINT UNSIGNED NOT NULL DEFAULT 0,
3893
InChanger TINYINT NOT NULL DEFAULT 0,
3894
MediaAddressing TINYINT NOT NULL DEFAULT 0,
3895
VolReadTime BIGINT UNSIGNED NOT NULL DEFAULT 0,
3896
VolWriteTime BIGINT UNSIGNED NOT NULL DEFAULT 0,
3897
PRIMARY KEY(MediaId),
3898
INDEX (PoolId)
3899
);
3900
CREATE TABLE Pool (
3901
PoolId INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
3902
Name TINYBLOB NOT NULL,
3903
NumVols INTEGER UNSIGNED NOT NULL DEFAULT 0,
3904
MaxVols INTEGER UNSIGNED NOT NULL DEFAULT 0,
3905
UseOnce TINYINT NOT NULL,
3906
UseCatalog TINYINT NOT NULL,
3907
AcceptAnyVolume TINYINT DEFAULT 0,
3908
VolRetention BIGINT UNSIGNED NOT NULL,
3909
VolUseDuration BIGINT UNSIGNED NOT NULL,
3910
MaxVolJobs INTEGER UNSIGNED NOT NULL DEFAULT 0,
3911
MaxVolFiles INTEGER UNSIGNED NOT NULL DEFAULT 0,
3912
MaxVolBytes BIGINT UNSIGNED NOT NULL,
3913
AutoPrune TINYINT DEFAULT 0,
3914
Recycle TINYINT DEFAULT 0,
3915
PoolType ENUM('Backup', 'Copy', 'Cloned', 'Archive', 'Migration', 'Scratch') NOT NULL,
3916
LabelFormat TINYBLOB,
3917
Enabled TINYINT DEFAULT 1,
3918
ScratchPoolId INTEGER UNSIGNED DEFAULT 0 REFERENCES Pool,
3919
RecyclePoolId INTEGER UNSIGNED DEFAULT 0 REFERENCES Pool,
3920
UNIQUE (Name(128)),
3921
PRIMARY KEY (PoolId)
3922
);
3923
CREATE TABLE Client (
3924
ClientId INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
3925
Name TINYBLOB NOT NULL,
3926
Uname TINYBLOB NOT NULL, /* full uname -a of client */
3927
AutoPrune TINYINT DEFAULT 0,
3928
FileRetention BIGINT UNSIGNED NOT NULL,
3929
JobRetention BIGINT UNSIGNED NOT NULL,
3930
UNIQUE (Name(128)),
3931
PRIMARY KEY(ClientId)
3932
);
3933
CREATE TABLE BaseFiles (
3934
BaseId INTEGER UNSIGNED AUTO_INCREMENT,
3935
BaseJobId INTEGER UNSIGNED NOT NULL REFERENCES Job,
3936
JobId INTEGER UNSIGNED NOT NULL REFERENCES Job,
3937
FileId INTEGER UNSIGNED NOT NULL REFERENCES File,
3938
FileIndex INTEGER UNSIGNED,
3939
PRIMARY KEY(BaseId)
3940
);
3941
CREATE TABLE UnsavedFiles (
3942
UnsavedId INTEGER UNSIGNED AUTO_INCREMENT,
3943
JobId INTEGER UNSIGNED NOT NULL REFERENCES Job,
3944
PathId INTEGER UNSIGNED NOT NULL REFERENCES Path,
3945
FilenameId INTEGER UNSIGNED NOT NULL REFERENCES Filename,
3946
PRIMARY KEY (UnsavedId)
3947
);
3948
CREATE TABLE Version (
3949
VersionId INTEGER UNSIGNED NOT NULL
3950
);
3951
-- Initialize Version
3952
INSERT INTO Version (VersionId) VALUES (7);
3953
CREATE TABLE Counters (
3954
Counter TINYBLOB NOT NULL,
3955
MinValue INTEGER,
3956
MaxValue INTEGER,
3957
CurrentValue INTEGER,
3958
WrapCounter TINYBLOB NOT NULL,
3959
PRIMARY KEY (Counter(128))
3960
);
3961
</PRE>
3962
<P>
3963
3964
<H1><A NAME="SECTION000110000000000000000"></A>
3965
<A NAME="_ChapterStart9"></A><A NAME="2688"></A>
3966
<A NAME="2689"></A>
3967
<BR>
3968
Storage Media Output Format
3969
</H1>
3970
<A NAME="2692"></A>
3971
3972
<P>
3973
3974
<H2><A NAME="SECTION000111000000000000000"></A>
3975
<A NAME="2694"></A>
3976
<BR>
3977
General
3978
</H2>
3979
<A NAME="2697"></A>
3980
3981
<P>
3982
This document describes the media format written by the Storage daemon. The
3983
Storage daemon reads and writes in units of blocks. Blocks contain records.
3984
Each block has a block header followed by records, and each record has a
3985
record header followed by record data.
3986
3987
<P>
3988
This chapter is intended to be a technical discussion of the Media Format and
3989
as such is not targeted at end users but rather at developers and system
3990
administrators that want or need to know more of the working details of <B>Bacula</B>.
3991
3992
<P>
3993
3994
<H2><A NAME="SECTION000112000000000000000"></A>
3995
<A NAME="2700"></A>
3996
<BR>
3997
Definitions
3998
</H2>
3999
<A NAME="2703"></A>
4000
4001
<P>
4002
<DL>
4003
<DT><STRONG>Block</STRONG></DT>
4004
<DD><A NAME="2705"></A>
4005
A block represents the primitive unit of information that the Storage daemon
4006
reads and writes to a physical device. Normally, for a tape device, it will
4007
be the same as a tape block. The Storage daemon always reads and writes
4008
blocks. A block consists of block header information followed by records.
4009
Clients of the Storage daemon (the File daemon) normally never see blocks.
4010
However, some of the Storage tools (bls, bscan, bextract, ...) may be use
4011
block header information. In older Bacula tape versions, a block could
4012
contain records (see record definition below) from multiple jobs. However,
4013
all blocks currently written by Bacula are block level BB02, and a given
4014
block contains records for only a single job. Different jobs simply have
4015
their own private blocks that are intermingled with the other blocks from
4016
other jobs on the Volume (previously the records were intermingled within
4017
the blocks). Having only records from a single job in any give block
4018
permitted moving the VolumeSessionId and VolumeSessionTime (see below) from
4019
each record heading to the Block header. This has two advantages: 1. a block
4020
can be quickly rejected based on the contents of the header without reading
4021
all the records. 2. because there is on the average more than one record per
4022
block, less data is written to the Volume for each job.
4023
4024
<P>
4025
</DD>
4026
<DT><STRONG>Record</STRONG></DT>
4027
<DD><A NAME="2706"></A>
4028
A record consists of a Record Header, which is managed by the Storage daemon
4029
and Record Data, which is the data received from the Client. A record is the
4030
primitive unit of information sent to and from the Storage daemon by the
4031
Client (File daemon) programs. The details are described below.
4032
4033
<P>
4034
</DD>
4035
<DT><STRONG>JobId</STRONG></DT>
4036
<DD><A NAME="2707"></A>
4037
A number assigned by the Director daemon for a particular job. This number
4038
will be unique for that particular Director (Catalog). The daemons use this
4039
number to keep track of individual jobs. Within the Storage daemon, the JobId
4040
may not be unique if several Directors are accessing the Storage daemon
4041
simultaneously.
4042
4043
<P>
4044
</DD>
4045
<DT><STRONG>Session</STRONG></DT>
4046
<DD><A NAME="2708"></A>
4047
A Session is a concept used in the Storage daemon corresponds one to one to a
4048
Job with the exception that each session is uniquely identified within the
4049
Storage daemon by a unique SessionId/SessionTime pair (see below).
4050
4051
<P>
4052
</DD>
4053
<DT><STRONG>VolSessionId</STRONG></DT>
4054
<DD><A NAME="2709"></A>
4055
A unique number assigned by the Storage daemon to a particular session (Job)
4056
it is having with a File daemon. This number by itself is not unique to the
4057
given Volume, but with the VolSessionTime, it is unique.
4058
4059
<P>
4060
</DD>
4061
<DT><STRONG>VolSessionTime</STRONG></DT>
4062
<DD><A NAME="2710"></A>
4063
A unique number assigned by the Storage daemon to a particular Storage daemon
4064
execution. It is actually the Unix time_t value of when the Storage daemon
4065
began execution cast to a 32 bit unsigned integer. The combination of the
4066
<B>VolSessionId</B> and the <B>VolSessionTime</B> for a given Storage daemon is
4067
guaranteed to be unique for each Job (or session).
4068
4069
<P>
4070
</DD>
4071
<DT><STRONG>FileIndex</STRONG></DT>
4072
<DD><A NAME="2713"></A>
4073
A sequential number beginning at one assigned by the File daemon to the files
4074
within a job that are sent to the Storage daemon for backup. The Storage
4075
daemon ensures that this number is greater than zero and sequential. Note,
4076
the Storage daemon uses negative FileIndexes to flag Session Start and End
4077
Labels as well as End of Volume Labels. Thus, the combination of
4078
VolSessionId, VolSessionTime, and FileIndex uniquely identifies the records
4079
for a single file written to a Volume.
4080
4081
<P>
4082
</DD>
4083
<DT><STRONG>Stream</STRONG></DT>
4084
<DD><A NAME="2714"></A>
4085
While writing the information for any particular file to the Volume, there
4086
can be any number of distinct pieces of information about that file, e.g. the
4087
attributes, the file data, ... The Stream indicates what piece of data it
4088
is, and it is an arbitrary number assigned by the File daemon to the parts
4089
(Unix attributes, Win32 attributes, data, compressed data, ...) of a file
4090
that are sent to the Storage daemon. The Storage daemon has no knowledge of
4091
the details of a Stream; it simply represents a numbered stream of bytes. The
4092
data for a given stream may be passed to the Storage daemon in single record,
4093
or in multiple records.
4094
4095
<P>
4096
</DD>
4097
<DT><STRONG>Block Header</STRONG></DT>
4098
<DD><A NAME="2715"></A>
4099
A block header consists of a block identification (``BB02''), a block length
4100
in bytes (typically 64,512) a checksum, and sequential block number. Each
4101
block starts with a Block Header and is followed by Records. Current block
4102
headers also contain the VolSessionId and VolSessionTime for the records
4103
written to that block.
4104
4105
<P>
4106
</DD>
4107
<DT><STRONG>Record Header</STRONG></DT>
4108
<DD><A NAME="2716"></A>
4109
A record header contains the Volume Session Id, the Volume Session Time, the
4110
FileIndex, the Stream, and the size of the data record which follows. The
4111
Record Header is always immediately followed by a Data Record if the size
4112
given in the Header is greater than zero. Note, for Block headers of level
4113
BB02 (version 1.27 and later), the Record header as written to tape does not
4114
contain the Volume Session Id and the Volume Session Time as these two
4115
fields are stored in the BB02 Block header. The in-memory record header does
4116
have those fields for convenience.
4117
4118
<P>
4119
</DD>
4120
<DT><STRONG>Data Record</STRONG></DT>
4121
<DD><A NAME="2717"></A>
4122
A data record consists of a binary stream of bytes and is always preceded by
4123
a Record Header. The details of the meaning of the binary stream of bytes are
4124
unknown to the Storage daemon, but the Client programs (File daemon) defines
4125
and thus knows the details of each record type.
4126
4127
<P>
4128
</DD>
4129
<DT><STRONG>Volume Label</STRONG></DT>
4130
<DD><A NAME="2718"></A>
4131
A label placed by the Storage daemon at the beginning of each storage volume.
4132
It contains general information about the volume. It is written in Record
4133
format. The Storage daemon manages Volume Labels, and if the client wants, he
4134
may also read them.
4135
4136
<P>
4137
</DD>
4138
<DT><STRONG>Begin Session Label</STRONG></DT>
4139
<DD><A NAME="2719"></A>
4140
The Begin Session Label is a special record placed by the Storage daemon on
4141
the storage medium as the first record of an append session job with a File
4142
daemon. This record is useful for finding the beginning of a particular
4143
session (Job), since no records with the same VolSessionId and VolSessionTime
4144
will precede this record. This record is not normally visible outside of the
4145
Storage daemon. The Begin Session Label is similar to the Volume Label except
4146
that it contains additional information pertaining to the Session.
4147
4148
<P>
4149
</DD>
4150
<DT><STRONG>End Session Label</STRONG></DT>
4151
<DD><A NAME="2720"></A>
4152
The End Session Label is a special record placed by the Storage daemon on the
4153
storage medium as the last record of an append session job with a File
4154
daemon. The End Session Record is distinguished by a FileIndex with a value
4155
of minus two (-2). This record is useful for detecting the end of a
4156
particular session since no records with the same VolSessionId and
4157
VolSessionTime will follow this record. This record is not normally visible
4158
outside of the Storage daemon. The End Session Label is similar to the Volume
4159
Label except that it contains additional information pertaining to the
4160
Session.
4161
</DD>
4162
</DL>
4163
4164
<P>
4165
4166
<H2><A NAME="SECTION000113000000000000000"></A>
4167
<A NAME="2723"></A>
4168
<A NAME="2724"></A>
4169
<BR>
4170
Storage Daemon File Output Format
4171
</H2>
4172
<A NAME="2727"></A>
4173
4174
<P>
4175
The file storage and tape storage formats are identical except that tape
4176
records are by default blocked into blocks of 64,512 bytes, except for the
4177
last block, which is the actual number of bytes written rounded up to a
4178
multiple of 1024 whereas the last record of file storage is not rounded up.
4179
The default block size of 64,512 bytes may be overridden by the user (some
4180
older tape drives only support block sizes of 32K). Each Session written to
4181
tape is terminated with an End of File mark (this will be removed later).
4182
Sessions written to file are simply appended to the end of the file.
4183
4184
<P>
4185
4186
<H2><A NAME="SECTION000114000000000000000"></A>
4187
<A NAME="2729"></A>
4188
<A NAME="2730"></A>
4189
<BR>
4190
Overall Format
4191
</H2>
4192
<A NAME="2733"></A>
4193
4194
<P>
4195
A Bacula output file consists of Blocks of data. Each block contains a block
4196
header followed by records. Each record consists of a record header followed
4197
by the record data. The first record on a tape will always be the Volume Label
4198
Record.
4199
4200
<P>
4201
No Record Header will be split across Bacula blocks. However, Record Data may
4202
be split across any number of Bacula blocks. Obviously this will not be the
4203
case for the Volume Label which will always be smaller than the Bacula Block
4204
size.
4205
4206
<P>
4207
To simplify reading tapes, the Start of Session (SOS) and End of Session (EOS)
4208
records are never split across blocks. If this is about to happen, Bacula will
4209
write a short block before writing the session record (actually, the SOS
4210
record should always be the first record in a block, excepting perhaps the
4211
Volume label).
4212
4213
<P>
4214
Due to hardware limitations, the last block written to the tape may not be
4215
fully written. If your drive permits backspace record, Bacula will backup over
4216
the last record written on the tape, re-read it and verify that it was
4217
correctly written.
4218
4219
<P>
4220
When a new tape is mounted Bacula will write the full contents of the
4221
partially written block to the new tape ensuring that there is no loss of
4222
data. When reading a tape, Bacula will discard any block that is not totally
4223
written, thus ensuring that there is no duplication of data. In addition,
4224
since Bacula blocks are sequentially numbered within a Job, it is easy to
4225
ensure that no block is missing or duplicated.
4226
4227
<P>
4228
4229
<H2><A NAME="SECTION000115000000000000000"></A>
4230
<A NAME="2735"></A>
4231
<BR>
4232
Serialization
4233
</H2>
4234
<A NAME="2738"></A>
4235
4236
<P>
4237
All Block Headers, Record Headers, and Label Records are written using
4238
Bacula's serialization routines. These routines guarantee that the data is
4239
written to the output volume in a machine independent format.
4240
4241
<P>
4242
4243
<H2><A NAME="SECTION000116000000000000000"></A>
4244
<A NAME="2740"></A>
4245
<A NAME="2741"></A>
4246
<BR>
4247
Block Header
4248
</H2>
4249
<A NAME="2744"></A>
4250
4251
<P>
4252
The format of the Block Header (version 1.27 and later) is:
4253
4254
<P>
4255
<PRE>
4256
uint32_t CheckSum; /* Block check sum */
4257
uint32_t BlockSize; /* Block byte size including the header */
4258
uint32_t BlockNumber; /* Block number */
4259
char ID[4] = "BB02"; /* Identification and block level */
4260
uint32_t VolSessionId; /* Session Id for Job */
4261
uint32_t VolSessionTime; /* Session Time for Job */
4262
</PRE>
4263
<P>
4264
The Block header is a fixed length and fixed format and is followed by Record
4265
Headers and Record Data. The CheckSum field is a 32 bit checksum of the block
4266
data and the block header but not including the CheckSum field. The Block
4267
Header is always immediately followed by a Record Header. If the tape is
4268
damaged, a Bacula utility will be able to recover as much information as
4269
possible from the tape by recovering blocks which are valid. The Block header
4270
is written using the Bacula serialization routines and thus is guaranteed to
4271
be in machine independent format. See below for version 2 of the block header.
4272
4273
<P>
4274
4275
<H2><A NAME="SECTION000117000000000000000"></A>
4276
<A NAME="2748"></A>
4277
<A NAME="2749"></A>
4278
<BR>
4279
Record Header
4280
</H2>
4281
<A NAME="2752"></A>
4282
4283
<P>
4284
Each binary data record is preceded by a Record Header. The Record Header is
4285
fixed length and fixed format, whereas the binary data record is of variable
4286
length. The Record Header is written using the Bacula serialization routines
4287
and thus is guaranteed to be in machine independent format.
4288
4289
<P>
4290
The format of the Record Header (version 1.27 or later) is:
4291
4292
<P>
4293
<PRE>
4294
int32_t FileIndex; /* File index supplied by File daemon */
4295
int32_t Stream; /* Stream number supplied by File daemon */
4296
uint32_t DataSize; /* size of following data record in bytes */
4297
</PRE>
4298
<P>
4299
This record is followed by the binary Stream data of DataSize bytes, followed
4300
by another Record Header record and the binary stream data. For the definitive
4301
definition of this record, see record.h in the src/stored directory.
4302
4303
<P>
4304
Additional notes on the above:
4305
4306
<P>
4307
<DL>
4308
<DT><STRONG>The <B>VolSessionId</B> </STRONG></DT>
4309
<DD><A NAME="2757"></A>
4310
is a unique sequential number that is assigned by the Storage Daemon to a
4311
particular Job. This number is sequential since the start of execution of the
4312
daemon.
4313
4314
<P>
4315
</DD>
4316
<DT><STRONG>The <B>VolSessionTime</B> </STRONG></DT>
4317
<DD><A NAME="2759"></A>
4318
is the time/date that the current execution of the Storage Daemon started. It
4319
assures that the combination of VolSessionId and VolSessionTime is unique for
4320
every jobs written to the tape, even if there was a machine crash between two
4321
writes.
4322
4323
<P>
4324
</DD>
4325
<DT><STRONG>The <B>FileIndex</B> </STRONG></DT>
4326
<DD><A NAME="2761"></A>
4327
is a sequential file number within a job. The Storage daemon requires this
4328
index to be greater than zero and sequential. Note, however, that the File
4329
daemon may send multiple Streams for the same FileIndex. In addition, the
4330
Storage daemon uses negative FileIndices to hold the Begin Session Label, the
4331
End Session Label, and the End of Volume Label.
4332
4333
<P>
4334
</DD>
4335
<DT><STRONG>The <B>Stream</B> </STRONG></DT>
4336
<DD><A NAME="2763"></A>
4337
is defined by the File daemon and is used to identify separate parts of the
4338
data saved for each file (Unix attributes, Win32 attributes, file data,
4339
compressed file data, sparse file data, ...). The Storage Daemon has no idea
4340
of what a Stream is or what it contains except that the Stream is required to
4341
be a positive integer. Negative Stream numbers are used internally by the
4342
Storage daemon to indicate that the record is a continuation of the previous
4343
record (the previous record would not entirely fit in the block).
4344
4345
<P>
4346
For Start Session and End Session Labels (where the FileIndex is negative),
4347
the Storage daemon uses the Stream field to contain the JobId. The current
4348
stream definitions are:
4349
4350
<P>
4351
<PRE>
4352
#define STREAM_UNIX_ATTRIBUTES 1 /* Generic Unix attributes */
4353
#define STREAM_FILE_DATA 2 /* Standard uncompressed data */
4354
#define STREAM_MD5_SIGNATURE 3 /* MD5 signature for the file */
4355
#define STREAM_GZIP_DATA 4 /* GZip compressed file data */
4356
/* Extended Unix attributes with Win32 Extended data. Deprecated. */
4357
#define STREAM_UNIX_ATTRIBUTES_EX 5 /* Extended Unix attr for Win32 EX */
4358
#define STREAM_SPARSE_DATA 6 /* Sparse data stream */
4359
#define STREAM_SPARSE_GZIP_DATA 7
4360
#define STREAM_PROGRAM_NAMES 8 /* program names for program data */
4361
#define STREAM_PROGRAM_DATA 9 /* Data needing program */
4362
#define STREAM_SHA1_SIGNATURE 10 /* SHA1 signature for the file */
4363
#define STREAM_WIN32_DATA 11 /* Win32 BackupRead data */
4364
#define STREAM_WIN32_GZIP_DATA 12 /* Gzipped Win32 BackupRead data */
4365
#define STREAM_MACOS_FORK_DATA 13 /* Mac resource fork */
4366
#define STREAM_HFSPLUS_ATTRIBUTES 14 /* Mac OS extra attributes */
4367
#define STREAM_UNIX_ATTRIBUTES_ACCESS_ACL 15 /* Standard ACL attributes on UNIX */
4368
#define STREAM_UNIX_ATTRIBUTES_DEFAULT_ACL 16 /* Default ACL attributes on UNIX */
4369
</PRE>
4370
<P>
4371
</DD>
4372
<DT><STRONG>The <B>DataSize</B> </STRONG></DT>
4373
<DD><A NAME="2767"></A>
4374
is the size in bytes of the binary data record that follows the Session
4375
Record header. The Storage Daemon has no idea of the actual contents of the
4376
binary data record. For standard Unix files, the data record typically
4377
contains the file attributes or the file data. For a sparse file the first
4378
64 bits of the file data contains the storage address for the data block.
4379
</DD>
4380
</DL>
4381
4382
<P>
4383
The Record Header is never split across two blocks. If there is not enough
4384
room in a block for the full Record Header, the block is padded to the end
4385
with zeros and the Record Header begins in the next block. The data record, on
4386
the other hand, may be split across multiple blocks and even multiple physical
4387
volumes. When a data record is split, the second (and possibly subsequent)
4388
piece of the data is preceded by a new Record Header. Thus each piece of data
4389
is always immediately preceded by a Record Header. When reading a record, if
4390
Bacula finds only part of the data in the first record, it will automatically
4391
read the next record and concatenate the data record to form a full data
4392
record.
4393
4394
<P>
4395
4396
<H2><A NAME="SECTION000118000000000000000"></A>
4397
<A NAME="2770"></A>
4398
<A NAME="2771"></A>
4399
<BR>
4400
Version BB02 Block Header
4401
</H2>
4402
<A NAME="2774"></A>
4403
4404
<P>
4405
Each session or Job has its own private block. As a consequence, the SessionId
4406
and SessionTime are written once in each Block Header and not in the Record
4407
Header. So, the second and current version of the Block Header BB02 is:
4408
4409
<P>
4410
<PRE>
4411
uint32_t CheckSum; /* Block check sum */
4412
uint32_t BlockSize; /* Block byte size including the header */
4413
uint32_t BlockNumber; /* Block number */
4414
char ID[4] = "BB02"; /* Identification and block level */
4415
uint32_t VolSessionId; /* Applies to all records */
4416
uint32_t VolSessionTime; /* contained in this block */
4417
</PRE>
4418
<P>
4419
As with the previous version, the BB02 Block header is a fixed length and
4420
fixed format and is followed by Record Headers and Record Data. The CheckSum
4421
field is a 32 bit CRC checksum of the block data and the block header but not
4422
including the CheckSum field. The Block Header is always immediately followed
4423
by a Record Header. If the tape is damaged, a Bacula utility will be able to
4424
recover as much information as possible from the tape by recovering blocks
4425
which are valid. The Block header is written using the Bacula serialization
4426
routines and thus is guaranteed to be in machine independent format.
4427
4428
<P>
4429
4430
<H2><A NAME="SECTION000119000000000000000"></A>
4431
<A NAME="2778"></A>
4432
<A NAME="2779"></A>
4433
<BR>
4434
Version 2 Record Header
4435
</H2>
4436
<A NAME="2782"></A>
4437
4438
<P>
4439
Version 2 Record Header is written to the medium when using Version BB02 Block
4440
Headers. The memory representation of the record is identical to the old BB01
4441
Record Header, but on the storage medium, the first two fields, namely
4442
VolSessionId and VolSessionTime are not written. The Block Header is filled
4443
with these values when the First user record is written (i.e. non label
4444
record) so that when the block is written, it will have the current and unique
4445
VolSessionId and VolSessionTime. On reading each record from the Block, the
4446
VolSessionId and VolSessionTime is filled in the Record Header from the Block
4447
Header.
4448
4449
<P>
4450
4451
<H2><A NAME="SECTION0001110000000000000000"></A>
4452
<A NAME="2784"></A>
4453
<A NAME="2785"></A>
4454
<BR>
4455
Volume Label Format
4456
</H2>
4457
<A NAME="2788"></A>
4458
4459
<P>
4460
Tape volume labels are created by the Storage daemon in response to a <B>label</B> command given to the Console program, or alternatively by the <B>btape</B> program. created. Each volume is labeled with the following information
4461
using the Bacula serialization routines, which guarantee machine byte order
4462
independence.
4463
4464
<P>
4465
For Bacula versions 1.27 and later, the Volume Label Format is:
4466
4467
<P>
4468
<PRE>
4469
char Id[32]; /* Bacula 1.0 Immortal\n */
4470
uint32_t VerNum; /* Label version number */
4471
/* VerNum 11 and greater Bacula 1.27 and later */
4472
btime_t label_btime; /* Time/date tape labeled */
4473
btime_t write_btime; /* Time/date tape first written */
4474
/* The following are 0 in VerNum 11 and greater */
4475
float64_t write_date; /* Date this label written */
4476
float64_t write_time; /* Time this label written */
4477
char VolName[128]; /* Volume name */
4478
char PrevVolName[128]; /* Previous Volume Name */
4479
char PoolName[128]; /* Pool name */
4480
char PoolType[128]; /* Pool type */
4481
char MediaType[128]; /* Type of this media */
4482
char HostName[128]; /* Host name of writing computer */
4483
char LabelProg[32]; /* Label program name */
4484
char ProgVersion[32]; /* Program version */
4485
char ProgDate[32]; /* Program build date/time */
4486
</PRE>
4487
<P>
4488
Note, the LabelType (Volume Label, Volume PreLabel, Session Start Label, ...)
4489
is stored in the record FileIndex field of the Record Header and does not
4490
appear in the data part of the record.
4491
4492
<P>
4493
4494
<H2><A NAME="SECTION0001111000000000000000"></A>
4495
<A NAME="2794"></A>
4496
<A NAME="2795"></A>
4497
<BR>
4498
Session Label
4499
</H2>
4500
<A NAME="2798"></A>
4501
4502
<P>
4503
The Session Label is written at the beginning and end of each session as well
4504
as the last record on the physical medium. It has the following binary format:
4505
4506
<P>
4507
<PRE>
4508
char Id[32]; /* Bacula Immortal ... */
4509
uint32_t VerNum; /* Label version number */
4510
uint32_t JobId; /* Job id */
4511
uint32_t VolumeIndex; /* sequence no of vol */
4512
/* Prior to VerNum 11 */
4513
float64_t write_date; /* Date this label written */
4514
/* VerNum 11 and greater */
4515
btime_t write_btime; /* time/date record written */
4516
/* The following is zero VerNum 11 and greater */
4517
float64_t write_time; /* Time this label written */
4518
char PoolName[128]; /* Pool name */
4519
char PoolType[128]; /* Pool type */
4520
char JobName[128]; /* base Job name */
4521
char ClientName[128];
4522
/* Added in VerNum 10 */
4523
char Job[128]; /* Unique Job name */
4524
char FileSetName[128]; /* FileSet name */
4525
uint32_t JobType;
4526
uint32_t JobLevel;
4527
</PRE>
4528
<P>
4529
In addition, the EOS label contains:
4530
4531
<P>
4532
<PRE>
4533
/* The remainder are part of EOS label only */
4534
uint32_t JobFiles;
4535
uint64_t JobBytes;
4536
uint32_t start_block;
4537
uint32_t end_block;
4538
uint32_t start_file;
4539
uint32_t end_file;
4540
uint32_t JobErrors;
4541
</PRE>
4542
<P>
4543
In addition, for VerNum greater than 10, the EOS label contains (in addition
4544
to the above):
4545
4546
<P>
4547
<PRE>
4548
uint32_t JobStatus /* Job termination code */
4549
</PRE>
4550
<P>
4551
: Note, the LabelType (Volume Label, Volume PreLabel, Session Start Label,
4552
...) is stored in the record FileIndex field and does not appear in the data
4553
part of the record. Also, the Stream field of the Record Header contains the
4554
JobId. This permits quick filtering without actually reading all the session
4555
data in many cases.
4556
4557
<P>
4558
4559
<H2><A NAME="SECTION0001112000000000000000"></A>
4560
<A NAME="2806"></A>
4561
<A NAME="2807"></A>
4562
<BR>
4563
Overall Storage Format
4564
</H2>
4565
<A NAME="2810"></A>
4566
4567
<P>
4568
<PRE>
4569
Current Bacula Tape Format
4570
6 June 2001
4571
Version BB02 added 28 September 2002
4572
Version BB01 is the old deprecated format.
4573
A Bacula tape is composed of tape Blocks. Each block
4574
has a Block header followed by the block data. Block
4575
Data consists of Records. Records consist of Record
4576
Headers followed by Record Data.
4577
:=======================================================:
4578
| |
4579
| Block Header (24 bytes) |
4580
| |
4581
|-------------------------------------------------------|
4582
| |
4583
| Record Header (12 bytes) |
4584
| |
4585
|-------------------------------------------------------|
4586
| |
4587
| Record Data |
4588
| |
4589
|-------------------------------------------------------|
4590
| |
4591
| Record Header (12 bytes) |
4592
| |
4593
|-------------------------------------------------------|
4594
| |
4595
| ... |
4596
Block Header: the first item in each block. The format is
4597
shown below.
4598
Partial Data block: occurs if the data from a previous
4599
block spills over to this block (the normal case except
4600
for the first block on a tape). However, this partial
4601
data block is always preceded by a record header.
4602
Record Header: identifies the Volume Session, the Stream
4603
and the following Record Data size. See below for format.
4604
Record data: arbitrary binary data.
4605
Block Header Format BB02
4606
:=======================================================:
4607
| CheckSum (uint32_t) |
4608
|-------------------------------------------------------|
4609
| BlockSize (uint32_t) |
4610
|-------------------------------------------------------|
4611
| BlockNumber (uint32_t) |
4612
|-------------------------------------------------------|
4613
| "BB02" (char [4]) |
4614
|-------------------------------------------------------|
4615
| VolSessionId (uint32_t) |
4616
|-------------------------------------------------------|
4617
| VolSessionTime (uint32_t) |
4618
:=======================================================:
4619
BBO2: Serves to identify the block as a
4620
Bacula block and also servers as a block format identifier
4621
should we ever need to change the format.
4622
BlockSize: is the size in bytes of the block. When reading
4623
back a block, if the BlockSize does not agree with the
4624
actual size read, Bacula discards the block.
4625
CheckSum: a checksum for the Block.
4626
BlockNumber: is the sequential block number on the tape.
4627
VolSessionId: a unique sequential number that is assigned
4628
by the Storage Daemon to a particular Job.
4629
This number is sequential since the start
4630
of execution of the daemon.
4631
VolSessionTime: the time/date that the current execution
4632
of the Storage Daemon started. It assures
4633
that the combination of VolSessionId and
4634
VolSessionTime is unique for all jobs
4635
written to the tape, even if there was a
4636
machine crash between two writes.
4637
Record Header Format BB02
4638
:=======================================================:
4639
| FileIndex (int32_t) |
4640
|-------------------------------------------------------|
4641
| Stream (int32_t) |
4642
|-------------------------------------------------------|
4643
| DataSize (uint32_t) |
4644
:=======================================================:
4645
FileIndex: a sequential file number within a job. The
4646
Storage daemon enforces this index to be
4647
greater than zero and sequential. Note,
4648
however, that the File daemon may send
4649
multiple Streams for the same FileIndex.
4650
The Storage Daemon uses negative FileIndices
4651
to identify Session Start and End labels
4652
as well as the End of Volume labels.
4653
Stream: defined by the File daemon and is intended to be
4654
used to identify separate parts of the data
4655
saved for each file (attributes, file data,
4656
...). The Storage Daemon has no idea of
4657
what a Stream is or what it contains.
4658
DataSize: the size in bytes of the binary data record
4659
that follows the Session Record header.
4660
The Storage Daemon has no idea of the
4661
actual contents of the binary data record.
4662
For standard Unix files, the data record
4663
typically contains the file attributes or
4664
the file data. For a sparse file
4665
the first 64 bits of the data contains
4666
the storage address for the data block.
4667
Volume Label
4668
:=======================================================:
4669
| Id (32 bytes) |
4670
|-------------------------------------------------------|
4671
| VerNum (uint32_t) |
4672
|-------------------------------------------------------|
4673
| label_date (float64_t) |
4674
| label_btime (btime_t VerNum 11 |
4675
|-------------------------------------------------------|
4676
| label_time (float64_t) |
4677
| write_btime (btime_t VerNum 11 |
4678
|-------------------------------------------------------|
4679
| write_date (float64_t) |
4680
| 0 (float64_t) VerNum 11 |
4681
|-------------------------------------------------------|
4682
| write_time (float64_t) |
4683
| 0 (float64_t) VerNum 11 |
4684
|-------------------------------------------------------|
4685
| VolName (128 bytes) |
4686
|-------------------------------------------------------|
4687
| PrevVolName (128 bytes) |
4688
|-------------------------------------------------------|
4689
| PoolName (128 bytes) |
4690
|-------------------------------------------------------|
4691
| PoolType (128 bytes) |
4692
|-------------------------------------------------------|
4693
| MediaType (128 bytes) |
4694
|-------------------------------------------------------|
4695
| HostName (128 bytes) |
4696
|-------------------------------------------------------|
4697
| LabelProg (32 bytes) |
4698
|-------------------------------------------------------|
4699
| ProgVersion (32 bytes) |
4700
|-------------------------------------------------------|
4701
| ProgDate (32 bytes) |
4702
|-------------------------------------------------------|
4703
:=======================================================:
4704
4705
Id: 32 byte Bacula identifier "Bacula 1.0 immortal\n"
4706
(old version also recognized:)
4707
Id: 32 byte Bacula identifier "Bacula 0.9 mortal\n"
4708
LabelType (Saved in the FileIndex of the Header record).
4709
PRE_LABEL -1 Volume label on unwritten tape
4710
VOL_LABEL -2 Volume label after tape written
4711
EOM_LABEL -3 Label at EOM (not currently implemented)
4712
SOS_LABEL -4 Start of Session label (format given below)
4713
EOS_LABEL -5 End of Session label (format given below)
4714
VerNum: 11
4715
label_date: Julian day tape labeled
4716
label_time: Julian time tape labeled
4717
write_date: Julian date tape first used (data written)
4718
write_time: Julian time tape first used (data written)
4719
VolName: "Physical" Volume name
4720
PrevVolName: The VolName of the previous tape (if this tape is
4721
a continuation of the previous one).
4722
PoolName: Pool Name
4723
PoolType: Pool Type
4724
MediaType: Media Type
4725
HostName: Name of host that is first writing the tape
4726
LabelProg: Name of the program that labeled the tape
4727
ProgVersion: Version of the label program
4728
ProgDate: Date Label program built
4729
Session Label
4730
:=======================================================:
4731
| Id (32 bytes) |
4732
|-------------------------------------------------------|
4733
| VerNum (uint32_t) |
4734
|-------------------------------------------------------|
4735
| JobId (uint32_t) |
4736
|-------------------------------------------------------|
4737
| write_btime (btime_t) VerNum 11 |
4738
|-------------------------------------------------------|
4739
| 0 (float64_t) VerNum 11 |
4740
|-------------------------------------------------------|
4741
| PoolName (128 bytes) |
4742
|-------------------------------------------------------|
4743
| PoolType (128 bytes) |
4744
|-------------------------------------------------------|
4745
| JobName (128 bytes) |
4746
|-------------------------------------------------------|
4747
| ClientName (128 bytes) |
4748
|-------------------------------------------------------|
4749
| Job (128 bytes) |
4750
|-------------------------------------------------------|
4751
| FileSetName (128 bytes) |
4752
|-------------------------------------------------------|
4753
| JobType (uint32_t) |
4754
|-------------------------------------------------------|
4755
| JobLevel (uint32_t) |
4756
|-------------------------------------------------------|
4757
| FileSetMD5 (50 bytes) VerNum 11 |
4758
|-------------------------------------------------------|
4759
Additional fields in End Of Session Label
4760
|-------------------------------------------------------|
4761
| JobFiles (uint32_t) |
4762
|-------------------------------------------------------|
4763
| JobBytes (uint32_t) |
4764
|-------------------------------------------------------|
4765
| start_block (uint32_t) |
4766
|-------------------------------------------------------|
4767
| end_block (uint32_t) |
4768
|-------------------------------------------------------|
4769
| start_file (uint32_t) |
4770
|-------------------------------------------------------|
4771
| end_file (uint32_t) |
4772
|-------------------------------------------------------|
4773
| JobErrors (uint32_t) |
4774
|-------------------------------------------------------|
4775
| JobStatus (uint32_t) VerNum 11 |
4776
:=======================================================:
4777
* => fields deprecated
4778
Id: 32 byte Bacula Identifier "Bacula 1.0 immortal\n"
4779
LabelType (in FileIndex field of Header):
4780
EOM_LABEL -3 Label at EOM
4781
SOS_LABEL -4 Start of Session label
4782
EOS_LABEL -5 End of Session label
4783
VerNum: 11
4784
JobId: JobId
4785
write_btime: Bacula time/date this tape record written
4786
write_date: Julian date tape this record written - deprecated
4787
write_time: Julian time tape this record written - deprecated.
4788
PoolName: Pool Name
4789
PoolType: Pool Type
4790
MediaType: Media Type
4791
ClientName: Name of File daemon or Client writing this session
4792
Not used for EOM_LABEL.
4793
</PRE>
4794
<P>
4795
4796
<H2><A NAME="SECTION0001113000000000000000"></A>
4797
<A NAME="2814"></A>
4798
<A NAME="2815"></A>
4799
<BR>
4800
Unix File Attributes
4801
</H2>
4802
<A NAME="2818"></A>
4803
4804
<P>
4805
The Unix File Attributes packet consists of the following:
4806
4807
<P>
4808
<File-Index> <Type>
4809
<Filename>@<File-Attributes>@<Link>
4810
@<Extended-Attributes@> where
4811
4812
<P>
4813
<DL>
4814
<DT><STRONG>@</STRONG></DT>
4815
<DD>represents a byte containing a binary zero.
4816
4817
<P>
4818
</DD>
4819
<DT><STRONG>FileIndex</STRONG></DT>
4820
<DD><A NAME="2832"></A>
4821
is the sequential file index starting from one assigned by the File daemon.
4822
4823
<P>
4824
</DD>
4825
<DT><STRONG>Type</STRONG></DT>
4826
<DD><A NAME="2833"></A>
4827
is one of the following:
4828
4829
<P>
4830
<PRE>
4831
#define FT_LNKSAVED 1 /* hard link to file already saved */
4832
#define FT_REGE 2 /* Regular file but empty */
4833
#define FT_REG 3 /* Regular file */
4834
#define FT_LNK 4 /* Soft Link */
4835
#define FT_DIR 5 /* Directory */
4836
#define FT_SPEC 6 /* Special file -- chr, blk, fifo, sock */
4837
#define FT_NOACCESS 7 /* Not able to access */
4838
#define FT_NOFOLLOW 8 /* Could not follow link */
4839
#define FT_NOSTAT 9 /* Could not stat file */
4840
#define FT_NOCHG 10 /* Incremental option, file not changed */
4841
#define FT_DIRNOCHG 11 /* Incremental option, directory not changed */
4842
#define FT_ISARCH 12 /* Trying to save archive file */
4843
#define FT_NORECURSE 13 /* No recursion into directory */
4844
#define FT_NOFSCHG 14 /* Different file system, prohibited */
4845
#define FT_NOOPEN 15 /* Could not open directory */
4846
#define FT_RAW 16 /* Raw block device */
4847
#define FT_FIFO 17 /* Raw fifo device */
4848
</PRE>
4849
<P>
4850
</DD>
4851
<DT><STRONG>Filename</STRONG></DT>
4852
<DD><A NAME="2836"></A>
4853
is the fully qualified filename.
4854
4855
<P>
4856
</DD>
4857
<DT><STRONG>File-Attributes</STRONG></DT>
4858
<DD><A NAME="2837"></A>
4859
consists of the 13 fields of the stat() buffer in ASCII base64 format
4860
separated by spaces. These fields and their meanings are shown below. This
4861
stat() packet is in Unix format, and MUST be provided (constructed) for ALL
4862
systems.
4863
4864
<P>
4865
</DD>
4866
<DT><STRONG>Link</STRONG></DT>
4867
<DD><A NAME="2838"></A>
4868
when the FT code is FT_LNK or FT_LNKSAVED, the item in question is a Unix
4869
link, and this field contains the fully qualified link name. When the FT code
4870
is not FT_LNK or FT_LNKSAVED, this field is null.
4871
4872
<P>
4873
</DD>
4874
<DT><STRONG>Extended-Attributes</STRONG></DT>
4875
<DD><A NAME="2839"></A>
4876
The exact format of this field is operating system dependent. It contains
4877
additional or extended attributes of a system dependent nature. Currently,
4878
this field is used only on WIN32 systems where it contains a ASCII base64
4879
representation of the WIN32_FILE_ATTRIBUTE_DATA structure as defined by
4880
Windows. The fields in the base64 representation of this structure are like
4881
the File-Attributes separated by spaces.
4882
</DD>
4883
</DL>
4884
4885
<P>
4886
The File-attributes consist of the following:
4887
4888
<P>
4889
<A NAME="2843"></A>
4890
<TABLE CELLPADDING=3 BORDER="1">
4891
<TR><TD ALIGN="CENTER" COLSPAN=1><B>Field No. </B></TD>
4892
<TD ALIGN="CENTER" COLSPAN=1><B>Stat Name </B></TD>
4893
<TD ALIGN="CENTER" COLSPAN=1><B>Unix </B></TD>
4894
<TD ALIGN="CENTER" COLSPAN=1><B>Win98/NT </B></TD>
4895
<TD ALIGN="CENTER" COLSPAN=1><B>MacOS </B></TD>
4896
</TR>
4897
<TR><TD ALIGN="CENTER" COLSPAN=1>1</TD>
4898
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=50>st_dev</TD>
4899
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>Device number of filesystem</TD>
4900
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>Drive number</TD>
4901
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=101>vRefNum</TD>
4902
</TR>
4903
<TR><TD ALIGN="CENTER" COLSPAN=1>2</TD>
4904
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=50>st_ino</TD>
4905
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>Inode number</TD>
4906
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>Always 0</TD>
4907
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=101>fileID/dirID</TD>
4908
</TR>
4909
<TR><TD ALIGN="CENTER" COLSPAN=1>3</TD>
4910
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=50>st_mode</TD>
4911
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>File mode</TD>
4912
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>File mode</TD>
4913
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=101>777 dirs/apps; 666 docs; 444 locked docs</TD>
4914
</TR>
4915
<TR><TD ALIGN="CENTER" COLSPAN=1>4</TD>
4916
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=50>st_nlink</TD>
4917
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>Number of links to the file</TD>
4918
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>Number of link (only on NTFS)</TD>
4919
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=101>Always 1</TD>
4920
</TR>
4921
<TR><TD ALIGN="CENTER" COLSPAN=1>5</TD>
4922
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=50>st_uid</TD>
4923
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>Owner ID</TD>
4924
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>Always 0</TD>
4925
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=101>Always 0</TD>
4926
</TR>
4927
<TR><TD ALIGN="CENTER" COLSPAN=1>6</TD>
4928
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=50>st_gid</TD>
4929
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>Group ID</TD>
4930
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>Always 0</TD>
4931
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=101>Always 0</TD>
4932
</TR>
4933
<TR><TD ALIGN="CENTER" COLSPAN=1>7</TD>
4934
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=50>st_rdev</TD>
4935
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>Device ID for special files</TD>
4936
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>Drive No.</TD>
4937
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=101>Always 0</TD>
4938
</TR>
4939
<TR><TD ALIGN="CENTER" COLSPAN=1>8</TD>
4940
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=50>st_size</TD>
4941
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>File size in bytes</TD>
4942
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>File
4943
size in bytes</TD>
4944
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=101>Data fork file size in bytes</TD>
4945
</TR>
4946
<TR><TD ALIGN="CENTER" COLSPAN=1>9</TD>
4947
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=50>st_blksize</TD>
4948
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>Preferred block size</TD>
4949
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>Always 0</TD>
4950
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=101>Preferred block size</TD>
4951
</TR>
4952
<TR><TD ALIGN="CENTER" COLSPAN=1>10</TD>
4953
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=50>st_blocks</TD>
4954
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>Number of blocks allocated</TD>
4955
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>Always 0</TD>
4956
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=101>Number of blocks allocated</TD>
4957
</TR>
4958
<TR><TD ALIGN="CENTER" COLSPAN=1>11</TD>
4959
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=50>st_atime</TD>
4960
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>Last access time since epoch</TD>
4961
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>Last access time since epoch</TD>
4962
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=101>Last access time -66 years</TD>
4963
</TR>
4964
<TR><TD ALIGN="CENTER" COLSPAN=1>12</TD>
4965
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=50>st_mtime</TD>
4966
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>Last modify time since epoch</TD>
4967
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>Last modify time since epoch</TD>
4968
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=101>Last access time -66 years</TD>
4969
</TR>
4970
<TR><TD ALIGN="CENTER" COLSPAN=1>13</TD>
4971
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=50>st_ctime</TD>
4972
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>Inode change time since epoch</TD>
4973
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=72>File create time since epoch</TD>
4974
<TD ALIGN="LEFT" VALIGN="TOP" WIDTH=101>File create time -66 years</TD>
4975
</TR>
4976
</TABLE>
4977
4978
<P>
4979
4980
<H2><A NAME="SECTION0001114000000000000000"></A>
4981
<A NAME="2958"></A>
4982
<A NAME="2959"></A>
4983
<BR>
4984
Old Depreciated Tape Format
4985
</H2>
4986
<A NAME="2962"></A>
4987
4988
<P>
4989
The format of the Block Header (version 1.26 and earlier) is:
4990
4991
<P>
4992
<PRE>
4993
uint32_t CheckSum; /* Block check sum */
4994
uint32_t BlockSize; /* Block byte size including the header */
4995
uint32_t BlockNumber; /* Block number */
4996
char ID[4] = "BB01"; /* Identification and block level */
4997
</PRE>
4998
<P>
4999
The format of the Record Header (version 1.26 or earlier) is:
5000
5001
<P>
5002
<PRE>
5003
uint32_t VolSessionId; /* Unique ID for this session */
5004
uint32_t VolSessionTime; /* Start time/date of session */
5005
int32_t FileIndex; /* File index supplied by File daemon */
5006
int32_t Stream; /* Stream number supplied by File daemon */
5007
uint32_t DataSize; /* size of following data record in bytes */
5008
</PRE>
5009
<P>
5010
<PRE>
5011
Current Bacula Tape Format
5012
6 June 2001
5013
Version BB01 is the old deprecated format.
5014
A Bacula tape is composed of tape Blocks. Each block
5015
has a Block header followed by the block data. Block
5016
Data consists of Records. Records consist of Record
5017
Headers followed by Record Data.
5018
:=======================================================:
5019
| |
5020
| Block Header |
5021
| (16 bytes version BB01) |
5022
|-------------------------------------------------------|
5023
| |
5024
| Record Header |
5025
| (20 bytes version BB01) |
5026
|-------------------------------------------------------|
5027
| |
5028
| Record Data |
5029
| |
5030
|-------------------------------------------------------|
5031
| |
5032
| Record Header |
5033
| (20 bytes version BB01) |
5034
|-------------------------------------------------------|
5035
| |
5036
| ... |
5037
Block Header: the first item in each block. The format is
5038
shown below.
5039
Partial Data block: occurs if the data from a previous
5040
block spills over to this block (the normal case except
5041
for the first block on a tape). However, this partial
5042
data block is always preceded by a record header.
5043
Record Header: identifies the Volume Session, the Stream
5044
and the following Record Data size. See below for format.
5045
Record data: arbitrary binary data.
5046
Block Header Format BB01 (deprecated)
5047
:=======================================================:
5048
| CheckSum (uint32_t) |
5049
|-------------------------------------------------------|
5050
| BlockSize (uint32_t) |
5051
|-------------------------------------------------------|
5052
| BlockNumber (uint32_t) |
5053
|-------------------------------------------------------|
5054
| "BB01" (char [4]) |
5055
:=======================================================:
5056
BBO1: Serves to identify the block as a
5057
Bacula block and also servers as a block format identifier
5058
should we ever need to change the format.
5059
BlockSize: is the size in bytes of the block. When reading
5060
back a block, if the BlockSize does not agree with the
5061
actual size read, Bacula discards the block.
5062
CheckSum: a checksum for the Block.
5063
BlockNumber: is the sequential block number on the tape.
5064
VolSessionId: a unique sequential number that is assigned
5065
by the Storage Daemon to a particular Job.
5066
This number is sequential since the start
5067
of execution of the daemon.
5068
VolSessionTime: the time/date that the current execution
5069
of the Storage Daemon started. It assures
5070
that the combination of VolSessionId and
5071
VolSessionTime is unique for all jobs
5072
written to the tape, even if there was a
5073
machine crash between two writes.
5074
Record Header Format BB01 (deprecated)
5075
:=======================================================:
5076
| VolSessionId (uint32_t) |
5077
|-------------------------------------------------------|
5078
| VolSessionTime (uint32_t) |
5079
|-------------------------------------------------------|
5080
| FileIndex (int32_t) |
5081
|-------------------------------------------------------|
5082
| Stream (int32_t) |
5083
|-------------------------------------------------------|
5084
| DataSize (uint32_t) |
5085
:=======================================================:
5086
VolSessionId: a unique sequential number that is assigned
5087
by the Storage Daemon to a particular Job.
5088
This number is sequential since the start
5089
of execution of the daemon.
5090
VolSessionTime: the time/date that the current execution
5091
of the Storage Daemon started. It assures
5092
that the combination of VolSessionId and
5093
VolSessionTime is unique for all jobs
5094
written to the tape, even if there was a
5095
machine crash between two writes.
5096
FileIndex: a sequential file number within a job. The
5097
Storage daemon enforces this index to be
5098
greater than zero and sequential. Note,
5099
however, that the File daemon may send
5100
multiple Streams for the same FileIndex.
5101
The Storage Daemon uses negative FileIndices
5102
to identify Session Start and End labels
5103
as well as the End of Volume labels.
5104
Stream: defined by the File daemon and is intended to be
5105
used to identify separate parts of the data
5106
saved for each file (attributes, file data,
5107
...). The Storage Daemon has no idea of
5108
what a Stream is or what it contains.
5109
DataSize: the size in bytes of the binary data record
5110
that follows the Session Record header.
5111
The Storage Daemon has no idea of the
5112
actual contents of the binary data record.
5113
For standard Unix files, the data record
5114
typically contains the file attributes or
5115
the file data. For a sparse file
5116
the first 64 bits of the data contains
5117
the storage address for the data block.
5118
Volume Label
5119
:=======================================================:
5120
| Id (32 bytes) |
5121
|-------------------------------------------------------|
5122
| VerNum (uint32_t) |
5123
|-------------------------------------------------------|
5124
| label_date (float64_t) |
5125
|-------------------------------------------------------|
5126
| label_time (float64_t) |
5127
|-------------------------------------------------------|
5128
| write_date (float64_t) |
5129
|-------------------------------------------------------|
5130
| write_time (float64_t) |
5131
|-------------------------------------------------------|
5132
| VolName (128 bytes) |
5133
|-------------------------------------------------------|
5134
| PrevVolName (128 bytes) |
5135
|-------------------------------------------------------|
5136
| PoolName (128 bytes) |
5137
|-------------------------------------------------------|
5138
| PoolType (128 bytes) |
5139
|-------------------------------------------------------|
5140
| MediaType (128 bytes) |
5141
|-------------------------------------------------------|
5142
| HostName (128 bytes) |
5143
|-------------------------------------------------------|
5144
| LabelProg (32 bytes) |
5145
|-------------------------------------------------------|
5146
| ProgVersion (32 bytes) |
5147
|-------------------------------------------------------|
5148
| ProgDate (32 bytes) |
5149
|-------------------------------------------------------|
5150
:=======================================================:
5151
5152
Id: 32 byte Bacula identifier "Bacula 1.0 immortal\n"
5153
(old version also recognized:)
5154
Id: 32 byte Bacula identifier "Bacula 0.9 mortal\n"
5155
LabelType (Saved in the FileIndex of the Header record).
5156
PRE_LABEL -1 Volume label on unwritten tape
5157
VOL_LABEL -2 Volume label after tape written
5158
EOM_LABEL -3 Label at EOM (not currently implemented)
5159
SOS_LABEL -4 Start of Session label (format given below)
5160
EOS_LABEL -5 End of Session label (format given below)
5161
label_date: Julian day tape labeled
5162
label_time: Julian time tape labeled
5163
write_date: Julian date tape first used (data written)
5164
write_time: Julian time tape first used (data written)
5165
VolName: "Physical" Volume name
5166
PrevVolName: The VolName of the previous tape (if this tape is
5167
a continuation of the previous one).
5168
PoolName: Pool Name
5169
PoolType: Pool Type
5170
MediaType: Media Type
5171
HostName: Name of host that is first writing the tape
5172
LabelProg: Name of the program that labeled the tape
5173
ProgVersion: Version of the label program
5174
ProgDate: Date Label program built
5175
Session Label
5176
:=======================================================:
5177
| Id (32 bytes) |
5178
|-------------------------------------------------------|
5179
| VerNum (uint32_t) |
5180
|-------------------------------------------------------|
5181
| JobId (uint32_t) |
5182
|-------------------------------------------------------|
5183
| *write_date (float64_t) VerNum 10 |
5184
|-------------------------------------------------------|
5185
| *write_time (float64_t) VerNum 10 |
5186
|-------------------------------------------------------|
5187
| PoolName (128 bytes) |
5188
|-------------------------------------------------------|
5189
| PoolType (128 bytes) |
5190
|-------------------------------------------------------|
5191
| JobName (128 bytes) |
5192
|-------------------------------------------------------|
5193
| ClientName (128 bytes) |
5194
|-------------------------------------------------------|
5195
| Job (128 bytes) |
5196
|-------------------------------------------------------|
5197
| FileSetName (128 bytes) |
5198
|-------------------------------------------------------|
5199
| JobType (uint32_t) |
5200
|-------------------------------------------------------|
5201
| JobLevel (uint32_t) |
5202
|-------------------------------------------------------|
5203
| FileSetMD5 (50 bytes) VerNum 11 |
5204
|-------------------------------------------------------|
5205
Additional fields in End Of Session Label
5206
|-------------------------------------------------------|
5207
| JobFiles (uint32_t) |
5208
|-------------------------------------------------------|
5209
| JobBytes (uint32_t) |
5210
|-------------------------------------------------------|
5211
| start_block (uint32_t) |
5212
|-------------------------------------------------------|
5213
| end_block (uint32_t) |
5214
|-------------------------------------------------------|
5215
| start_file (uint32_t) |
5216
|-------------------------------------------------------|
5217
| end_file (uint32_t) |
5218
|-------------------------------------------------------|
5219
| JobErrors (uint32_t) |
5220
|-------------------------------------------------------|
5221
| JobStatus (uint32_t) VerNum 11 |
5222
:=======================================================:
5223
* => fields deprecated
5224
Id: 32 byte Bacula Identifier "Bacula 1.0 immortal\n"
5225
LabelType (in FileIndex field of Header):
5226
EOM_LABEL -3 Label at EOM
5227
SOS_LABEL -4 Start of Session label
5228
EOS_LABEL -5 End of Session label
5229
VerNum: 11
5230
JobId: JobId
5231
write_btime: Bacula time/date this tape record written
5232
write_date: Julian date tape this record written - deprecated
5233
write_time: Julian time tape this record written - deprecated.
5234
PoolName: Pool Name
5235
PoolType: Pool Type
5236
MediaType: Media Type
5237
ClientName: Name of File daemon or Client writing this session
5238
Not used for EOM_LABEL.
5239
</PRE>
5240
<P>
5241
5242
<H1><A NAME="SECTION000120000000000000000"></A>
5243
<A NAME="_ChapterStart1"></A><A NAME="3236"></A>
5244
<A NAME="3237"></A>
5245
<BR>
5246
Bacula Porting Notes
5247
</H1>
5248
<A NAME="3240"></A>
5249
5250
<P>
5251
This document is intended mostly for developers who wish to port Bacula to a
5252
system that is not <B>officially</B> supported.
5253
5254
<P>
5255
It is hoped that Bacula clients will eventually run on every imaginable system
5256
that needs backing up (perhaps even a Palm). It is also hoped that the Bacula
5257
Directory and Storage daemons will run on every system capable of supporting
5258
them.
5259
5260
<P>
5261
5262
<H2><A NAME="SECTION000121000000000000000"></A>
5263
<A NAME="3243"></A>
5264
<A NAME="3244"></A>
5265
<BR>
5266
Porting Requirements
5267
</H2>
5268
<A NAME="3247"></A>
5269
5270
<P>
5271
In General, the following holds true:
5272
5273
<P>
5274
5275
<UL>
5276
<LI><B>Bacula</B> has been compiled and run on Linux RedHat, FreeBSD, and
5277
Solaris systems.
5278
</LI>
5279
<LI>In addition, clients exist on Win32, and Irix
5280
</LI>
5281
<LI>It requires GNU C++ to compile. You can try with other compilers, but
5282
you are on your own. The Irix client is built with the Irix complier, but, in
5283
general, you will need GNU.
5284
</LI>
5285
<LI>Your compiler must provide support for 64 bit signed and unsigned
5286
integers.
5287
</LI>
5288
<LI>You will need a recent copy of the <B>autoconf</B> tools loaded on your
5289
system (version 2.13 or later). The <B>autoconf</B> tools are used to build
5290
the configuration program, but are not part of the Bacula source
5291
distribution.
5292
</LI>
5293
<LI>There are certain third party packages that Bacula needs. Except for
5294
MySQL, they can all be found in the <B>depkgs</B> and <B>depkgs1</B> releases.
5295
</LI>
5296
<LI>To build the Win32 binaries, we use Microsoft VC++ standard
5297
2003. Please see the instructions in
5298
bacula-source/src/win32/README.win32 for more details. If you
5299
want to use VC++ Express, please see README.vc8. Our build is
5300
done under the most recent version of Cygwin, but Cygwin is
5301
not used in the Bacula binaries that are produced.
5302
Unfortunately, we do not have the resources to help you build
5303
your own version of the Win32 FD, so you are pretty much on
5304
your own. You can ask the bacula-devel list for help, but
5305
please don't expect much.
5306
</LI>
5307
<LI><B>Bacula</B> requires a good implementation of pthreads to work.
5308
</LI>
5309
<LI>The source code has been written with portability in mind and is mostly
5310
POSIX compatible. Thus porting to any POSIX compatible operating system
5311
should be relatively easy.
5312
</LI>
5313
</UL>
5314
5315
<P>
5316
5317
<H2><A NAME="SECTION000122000000000000000"></A>
5318
<A NAME="3257"></A>
5319
<A NAME="3258"></A>
5320
<BR>
5321
Steps to Take for Porting
5322
</H2>
5323
<A NAME="3261"></A>
5324
5325
<P>
5326
5327
<UL>
5328
<LI>The first step is to ensure that you have version 2.13 or later of the
5329
<B>autoconf</B> tools loaded. You can skip this step, but making changes to
5330
the configuration program will be difficult or impossible.
5331
</LI>
5332
<LI>The run a <B>./configure</B> command in the main source directory and
5333
examine the output. It should look something like the following:
5334
5335
<P>
5336
<PRE>
5337
Configuration on Mon Oct 28 11:42:27 CET 2002:
5338
Host: i686-pc-linux-gnu -- redhat 7.3
5339
Bacula version: 1.27 (26 October 2002)
5340
Source code location: .
5341
Install binaries: /sbin
5342
Install config files: /etc/bacula
5343
C Compiler: gcc
5344
C++ Compiler: c++
5345
Compiler flags: -g -O2
5346
Linker flags:
5347
Libraries: -lpthread
5348
Statically Linked Tools: no
5349
Database found: no
5350
Database type: Internal
5351
Database lib:
5352
Job Output Email: root@localhost
5353
Traceback Email: root@localhost
5354
SMTP Host Address: localhost
5355
Director Port 9101
5356
File daemon Port 9102
5357
Storage daemon Port 9103
5358
Working directory /etc/bacula/working
5359
SQL binaries Directory
5360
Large file support: yes
5361
readline support: yes
5362
cweb support: yes /home/kern/bacula/depkgs/cweb
5363
TCP Wrappers support: no
5364
ZLIB support: yes
5365
enable-smartalloc: yes
5366
enable-gnome: no
5367
gmp support: yes
5368
</PRE>
5369
<P>
5370
The details depend on your system. The first thing to check is that it
5371
properly identified your host on the <B>Host:</B> line. The first part (added
5372
in version 1.27) is the GNU four part identification of your system. The part
5373
after the -- is your system and the system version. Generally, if your system
5374
is not yet supported, you must correct these.
5375
</LI>
5376
<LI>If the <B>./configure</B> does not function properly, you must determine
5377
the cause and fix it. Generally, it will be because some required system
5378
routine is not available on your machine.
5379
</LI>
5380
<LI>To correct problems with detection of your system type or with routines
5381
and libraries, you must edit the file <B> <bacula-src>/autoconf/configure.in</B>. This is the ``source'' from
5382
which <B>configure</B> is built. In general, most of the changes for your
5383
system will be made in <B>autoconf/aclocal.m4</B> in the routine <B>BA_CHECK_OPSYS</B> or in the routine <B>BA_CHECK_OPSYS_DISTNAME</B>. I have
5384
already added the necessary code for most systems, but if yours shows up as
5385
<B>unknown</B> you will need to make changes. Then as mentioned above, you
5386
will need to set a number of system dependent items in <B>configure.in</B> in
5387
the <B>case</B> statement at approximately line 1050 (depending on the Bacula
5388
release).
5389
</LI>
5390
<LI>The items to in the case statement that corresponds to your system are
5391
the following:
5392
5393
<P>
5394
5395
<UL>
5396
<LI>DISTVER -- set to the version of your operating system. Typically some
5397
form of <B>uname</B> obtains it.
5398
</LI>
5399
<LI>TAPEDRIVE -- the default tape drive. Not too important as the user can
5400
set it as an option.
5401
</LI>
5402
<LI>PSCMD -- set to the <B>ps</B> command that will provide the PID in the
5403
first field and the program name in the second field. If this is not set
5404
properly, the <B>bacula stop</B> script will most likely not be able to stop
5405
Bacula in all cases.
5406
</LI>
5407
<LI>hostname -- command to return the base host name (non-qualified) of
5408
your system. This is generally the machine name. Not too important as the
5409
user can correct this in his configuration file.
5410
</LI>
5411
<LI>CFLAGS -- set any special compiler flags needed. Many systems need a
5412
special flag to make pthreads work. See cygwin for an example.
5413
</LI>
5414
<LI>LDFLAGS -- set any special loader flags. See cygwin for an example.
5415
</LI>
5416
<LI>PTHREAD_LIB -- set for any special pthreads flags needed during
5417
linking. See freebsd as an example.
5418
</LI>
5419
<LI>lld -- set so that a ``long long int'' will be properly edited in a
5420
printf() call.
5421
</LI>
5422
<LI>llu -- set so that a ``long long unsigned'' will be properly edited in
5423
a printf() call.
5424
</LI>
5425
<LI>PFILES -- set to add any files that you may define is your platform
5426
subdirectory. These files are used for installation of automatic system
5427
startup of Bacula daemons.
5428
</LI>
5429
</UL>
5430
5431
<P>
5432
</LI>
5433
<LI>To rebuild a new version of <B>configure</B> from a changed <B> autoconf/configure.in</B> you enter <B>make configure</B> in the top level Bacula
5434
source directory. You must have done a ./configure prior to trying to rebuild
5435
the configure script or it will get into an infinite loop.
5436
</LI>
5437
<LI>If the <B>make configure</B> gets into an infinite loop, ctl-c it, then
5438
do <B>./configure</B> (no options are necessary) and retry the <B>make
5439
configure</B>, which should now work.
5440
</LI>
5441
<LI>To rebuild <B>configure</B> you will need to have <B>autoconf</B> version
5442
2.57-3 or higher loaded. Older versions of autoconf will complain about
5443
unknown or bad options, and won't work.
5444
</LI>
5445
<LI>After you have a working <B>configure</B> script, you may need to make a
5446
few system dependent changes to the way Bacula works. Generally, these are
5447
done in <B>src/baconfig.h</B>. You can find a few examples of system dependent
5448
changes toward the end of this file. For example, on Irix systems, there is
5449
no definition for <B>socklen_t</B>, so it is made in this file. If your
5450
system has structure alignment requirements, check the definition of BALIGN
5451
in this file. Currently, all Bacula allocated memory is aligned on a <B>double</B> boundary.
5452
</LI>
5453
<LI>If you are having problems with Bacula's type definitions, you might
5454
look at <B>src/bc_types.h</B> where all the types such as <B>uint32_t</B>,
5455
<B>uint64_t</B>, etc. that Bacula uses are defined.
5456
</LI>
5457
</UL>
5458
5459
<P>
5460
5461
<H1><A NAME="SECTION000130000000000000000"></A>
5462
<A NAME="_ChapterStart"></A>
5463
<BR>
5464
Implementing a Bacula GUI Interface
5465
</H1>
5466
<A NAME="3404"></A>
5467
<A NAME="3405"></A>
5468
<A NAME="3408"></A>
5469
5470
<P>
5471
5472
<H2><A NAME="SECTION000131000000000000000">
5473
General</A>
5474
</H2>
5475
<A NAME="3410"></A>
5476
<A NAME="3413"></A>
5477
5478
<P>
5479
This document is intended mostly for developers who wish to develop a new GUI
5480
interface to <B>Bacula</B>.
5481
5482
<P>
5483
5484
<H3><A NAME="SECTION000131100000000000000">
5485
Minimal Code in Console Program</A>
5486
</H3>
5487
<A NAME="3416"></A>
5488
<A NAME="3417"></A>
5489
<A NAME="3420"></A>
5490
5491
<P>
5492
Until now, I have kept all the Catalog code in the Directory (with the
5493
exception of dbcheck and bscan). This is because at some point I would like to
5494
add user level security and access. If we have code spread everywhere such as
5495
in a GUI this will be more difficult. The other advantage is that any code you
5496
add to the Director is automatically available to both the tty console program
5497
and the GNOME program. The major disadvantage is it increases the size of the
5498
code -- however, compared to Networker the Bacula Director is really tiny.
5499
5500
<P>
5501
5502
<H3><A NAME="SECTION000131200000000000000">
5503
GUI Interface is Difficult</A>
5504
</H3>
5505
<A NAME="3422"></A>
5506
<A NAME="3423"></A>
5507
<A NAME="3426"></A>
5508
5509
<P>
5510
Interfacing to an interactive program such as Bacula can be very difficult
5511
because the interfacing program must interpret all the prompts that may come.
5512
This can be next to impossible. There are are a number of ways that Bacula is
5513
designed to facilitate this:
5514
5515
<P>
5516
5517
<UL>
5518
<LI>The Bacula network protocol is packet based, and thus pieces of
5519
information sent can be ASCII or binary.
5520
</LI>
5521
<LI>The packet interface permits knowing where the end of a list is.
5522
</LI>
5523
<LI>The packet interface permits special ``signals'' to be passed rather
5524
than data.
5525
</LI>
5526
<LI>The Director has a number of commands that are non-interactive. They
5527
all begin with a period, and provide things such as the list of all Jobs,
5528
list of all Clients, list of all Pools, list of all Storage, ... Thus the GUI
5529
interface can get to virtually all information that the Director has in a
5530
deterministic way. See <bacula-source>/src/dird/ua_dotcmds.c for
5531
more details on this.
5532
</LI>
5533
<LI>Most console commands allow all the arguments to be specified on the
5534
command line: e.g. <B>run job=NightlyBackup level=Full</B>
5535
</LI>
5536
</UL>
5537
5538
<P>
5539
One of the first things to overcome is to be able to establish a conversation
5540
with the Director. Although you can write all your own code, it is probably
5541
easier to use the Bacula subroutines. The following code is used by the
5542
Console program to begin a conversation.
5543
5544
<P>
5545
<PRE>
5546
static BSOCK *UA_sock = NULL;
5547
static JCR *jcr;
5548
...
5549
read-your-config-getting-address-and-pasword;
5550
UA_sock = bnet_connect(NULL, 5, 15, "Director daemon", dir->address,
5551
NULL, dir->DIRport, 0);
5552
if (UA_sock == NULL) {
5553
terminate_console(0);
5554
return 1;
5555
}
5556
jcr.dir_bsock = UA_sock;
5557
if (!authenticate_director(\&jcr, dir)) {
5558
fprintf(stderr, "ERR=%s", UA_sock->msg);
5559
terminate_console(0);
5560
return 1;
5561
}
5562
read_and_process_input(stdin, UA_sock);
5563
if (UA_sock) {
5564
bnet_sig(UA_sock, BNET_TERMINATE); /* send EOF */
5565
bnet_close(UA_sock);
5566
}
5567
exit 0;
5568
</PRE>
5569
<P>
5570
Then the read_and_process_input routine looks like the following:
5571
5572
<P>
5573
<PRE>
5574
get-input-to-send-to-the-Director;
5575
bnet_fsend(UA_sock, "%s", input);
5576
stat = bnet_recv(UA_sock);
5577
process-output-from-the-Director;
5578
</PRE>
5579
<P>
5580
For a GUI program things will be a bit more complicated. Basically in the very
5581
inner loop, you will need to check and see if any output is available on the
5582
UA_sock. For an example, please take a look at the GNOME GUI interface code
5583
in: <bacula-source&gt/src/gnome-console/console.c
5584
5585
<P>
5586
5587
<H1><A NAME="SECTION000140000000000000000"></A>
5588
<A NAME="_Chapter_TLS"></A><A NAME="3477"></A>
5589
<BR>
5590
TLS
5591
</H1>
5592
<A NAME="3480"></A>
5593
5594
<P>
5595
Written by Landon Fuller
5596
5597
<P>
5598
5599
<H2><A NAME="SECTION000141000000000000000"></A>
5600
<A NAME="3482"></A>
5601
<A NAME="3483"></A>
5602
<BR>
5603
Introduction to TLS
5604
</H2>
5605
<A NAME="3486"></A>
5606
5607
<P>
5608
This patch includes all the back-end code necessary to add complete TLS
5609
data encryption support to Bacula. In addition, support for TLS in
5610
Console/Director communications has been added as a proof of concept.
5611
Adding support for the remaining daemons will be straight-forward.
5612
Supported features of this patchset include:
5613
5614
<P>
5615
5616
<UL>
5617
<LI>Client/Server TLS Requirement Negotiation
5618
</LI>
5619
<LI>TLSv1 Connections with Server and Client Certificate
5620
Validation
5621
</LI>
5622
<LI>Forward Secrecy Support via Diffie-Hellman Ephemeral Keying
5623
</LI>
5624
</UL>
5625
5626
<P>
5627
This document will refer to both ``server'' and ``client'' contexts. These
5628
terms refer to the accepting and initiating peer, respectively.
5629
5630
<P>
5631
Diffie-Hellman anonymous ciphers are not supported by this patchset. The
5632
use of DH anonymous ciphers increases the code complexity and places
5633
explicit trust upon the two-way Cram-MD5 implementation. Cram-MD5 is
5634
subject to known plaintext attacks, and is should be considered
5635
considerably less secure than PKI certificate-based authentication.
5636
5637
<P>
5638
Appropriate autoconf macros have been added to detect and use OpenSSL. Two
5639
additional preprocessor defines have been added: <I>HAVE_TLS</I> and
5640
<I>HAVE_OPENSSL</I>. All changes not specific to OpenSSL rely on
5641
<I>HAVE_TLS</I>. OpenSSL-specific code is constrained to
5642
<I>src/lib/tls.c</I> to facilitate the support of alternative TLS
5643
implementations.
5644
5645
<P>
5646
5647
<H2><A NAME="SECTION000142000000000000000"></A>
5648
<A NAME="3494"></A>
5649
<A NAME="3495"></A>
5650
<BR>
5651
New Configuration Directives
5652
</H2>
5653
<A NAME="3498"></A>
5654
5655
<P>
5656
Additional configuration directives have been added to both the Console and
5657
Director resources. These new directives are defined as follows:
5658
5659
<P>
5660
5661
<UL>
5662
<LI><U>TLS Enable</U> <I>(yes/no)</I>
5663
Enable TLS support.
5664
5665
<P>
5666
</LI>
5667
<LI><U>TLS Require</U> <I>(yes/no)</I>
5668
Require TLS connections.
5669
5670
<P>
5671
</LI>
5672
<LI><U>TLS Certificate</U> <I>(path)</I>
5673
Path to PEM encoded TLS certificate. Used as either a client or server
5674
certificate.
5675
5676
<P>
5677
</LI>
5678
<LI><U>TLS Key</U> <I>(path)</I>
5679
Path to PEM encoded TLS private key. Must correspond with the TLS
5680
certificate.
5681
5682
<P>
5683
</LI>
5684
<LI><U>TLS Verify Peer</U> <I>(yes/no)</I>
5685
Verify peer certificate. Instructs server to request and verify the
5686
client's x509 certificate. Any client certificate signed by a known-CA
5687
will be accepted unless the TLS Allowed CN configuration directive is used.
5688
Not valid in a client context.
5689
5690
<P>
5691
</LI>
5692
<LI><U>TLS Allowed CN</U> <I>(string list)</I>
5693
Common name attribute of allowed peer certificates. If directive is
5694
specified, all client certificates will be verified against this list.
5695
This directive may be specified more than once. Not valid in a client
5696
context.
5697
5698
<P>
5699
</LI>
5700
<LI><U>TLS CA Certificate File</U> <I>(path)</I>
5701
Path to PEM encoded TLS CA certificate(s). Multiple certificates are
5702
permitted in the file. One of <I>TLS CA Certificate File</I> or <I>TLS
5703
CA Certificate Dir</I> are required in a server context if <U>TLS
5704
Verify Peer</U> is also specified, and are always required in a client
5705
context.
5706
5707
<P>
5708
</LI>
5709
<LI><U>TLS CA Certificate Dir</U> <I>(path)</I>
5710
Path to TLS CA certificate directory. In the current implementation,
5711
certificates must be stored PEM encoded with OpenSSL-compatible hashes.
5712
One of <I>TLS CA Certificate File</I> or <I>TLS CA Certificate Dir</I> are
5713
required in a server context if <I>TLS Verify Peer</I> is also specified,
5714
and are always required in a client context.
5715
5716
<P>
5717
</LI>
5718
<LI><U>TLS DH File</U> <I>(path)</I>
5719
Path to PEM encoded Diffie-Hellman parameter file. If this directive is
5720
specified, DH ephemeral keying will be enabled, allowing for forward
5721
secrecy of communications. This directive is only valid within a server
5722
context. To generate the parameter file, you may use openssl:
5723
<PRE>
5724
openssl dhparam -out dh1024.pem -5 1024
5725
</PRE>
5726
</LI>
5727
</UL>
5728
5729
<P>
5730
5731
<H2><A NAME="SECTION000143000000000000000"></A>
5732
<A NAME="3528"></A>
5733
<A NAME="3529"></A>
5734
<BR>
5735
TLS API Implementation
5736
</H2>
5737
<A NAME="3532"></A>
5738
5739
<P>
5740
To facilitate the use of additional TLS libraries, all OpenSSL-specific
5741
code has been implemented within <I>src/lib/tls.c</I>. In turn, a generic
5742
TLS API is exported.
5743
5744
<P>
5745
5746
<H3><A NAME="SECTION000143100000000000000"></A>
5747
<A NAME="3535"></A>
5748
<A NAME="3536"></A>
5749
<BR>
5750
Library Initialization and Cleanup
5751
</H3>
5752
<A NAME="3539"></A>
5753
5754
<P>
5755
<PRE>
5756
int init_tls (void);
5757
</PRE>
5758
<P>
5759
Performs TLS library initialization, including seeding of the PRNG. PRNG
5760
seeding has not yet been implemented for win32.
5761
5762
<P>
5763
<PRE>
5764
int cleanup_tls (void);
5765
</PRE>
5766
<P>
5767
Performs TLS library cleanup.
5768
5769
<P>
5770
5771
<H3><A NAME="SECTION000143200000000000000"></A>
5772
<A NAME="3545"></A>
5773
<A NAME="3546"></A>
5774
<BR>
5775
Manipulating TLS Contexts
5776
</H3>
5777
<A NAME="3549"></A>
5778
5779
<P>
5780
<PRE>
5781
TLS_CONTEXT *new_tls_context (const char *ca_certfile,
5782
const char *ca_certdir, const char *certfile,
5783
const char *keyfile, const char *dhfile, bool verify_peer);
5784
</PRE>
5785
<P>
5786
Allocates and initalizes a new opaque <I>TLS_CONTEXT</I> structure. The
5787
<I>TLS_CONTEXT</I> structure maintains default TLS settings from which
5788
<I>TLS_CONNECTION</I> structures are instantiated. In the future the
5789
<I>TLS_CONTEXT</I> structure may be used to maintain the TLS session
5790
cache. <I>ca_certfile</I> and <I>ca_certdir</I> arguments are used to
5791
initialize the CA verification stores. The <I>certfile</I> and
5792
<I>keyfile</I> arguments are used to initialize the local certificate and
5793
private key. If <I>dhfile</I> is non-NULL, it is used to initialize
5794
Diffie-Hellman ephemeral keying. If <I>verify_peer</I> is <I>true</I> ,
5795
client certificate validation is enabled.
5796
5797
<P>
5798
<PRE>
5799
void free_tls_context (TLS_CONTEXT *ctx);
5800
</PRE>
5801
<P>
5802
Deallocated a previously allocated <I>TLS_CONTEXT</I> structure.
5803
5804
<P>
5805
5806
<H3><A NAME="SECTION000143300000000000000"></A>
5807
<A NAME="3567"></A>
5808
<A NAME="3568"></A>
5809
<BR>
5810
Performing Post-Connection Verification
5811
</H3>
5812
<A NAME="3571"></A>
5813
5814
<P>
5815
<PRE>
5816
bool tls_postconnect_verify_host (TLS_CONNECTION *tls, const char *host);
5817
</PRE>
5818
<P>
5819
Performs post-connection verification of the peer-supplied x509
5820
certificate. Checks whether the <I>subjectAltName</I> and
5821
<I>commonName</I> attributes match the supplied <I>host</I> string.
5822
Returns <I>true</I> if there is a match, <I>false</I> otherwise.
5823
5824
<P>
5825
<PRE>
5826
bool tls_postconnect_verify_cn (TLS_CONNECTION *tls, alist *verify_list);
5827
</PRE>
5828
<P>
5829
Performs post-connection verification of the peer-supplied x509
5830
certificate. Checks whether the <I>commonName</I> attribute matches any
5831
strings supplied via the <I>verify_list</I> parameter. Returns
5832
<I>true</I> if there is a match, <I>false</I> otherwise.
5833
5834
<P>
5835
5836
<H3><A NAME="SECTION000143400000000000000"></A>
5837
<A NAME="3586"></A>
5838
<A NAME="3587"></A>
5839
<BR>
5840
Manipulating TLS Connections
5841
</H3>
5842
<A NAME="3590"></A>
5843
5844
<P>
5845
<PRE>
5846
TLS_CONNECTION *new_tls_connection (TLS_CONTEXT *ctx, int fd);
5847
</PRE>
5848
<P>
5849
Allocates and initializes a new <I>TLS_CONNECTION</I> structure with
5850
context <I>ctx</I> and file descriptor <I>fd</I>.
5851
5852
<P>
5853
<PRE>
5854
void free_tls_connection (TLS_CONNECTION *tls);
5855
</PRE>
5856
<P>
5857
Deallocates memory associated with the <I>tls</I> structure.
5858
5859
<P>
5860
<PRE>
5861
bool tls_bsock_connect (BSOCK *bsock);
5862
</PRE>
5863
<P>
5864
Negotiates a a TLS client connection via <I>bsock</I>. Returns <I>true</I>
5865
if successful, <I>false</I> otherwise. Will fail if there is a TLS
5866
protocol error or an invalid certificate is presented
5867
5868
<P>
5869
<PRE>
5870
bool tls_bsock_accept (BSOCK *bsock);
5871
</PRE>
5872
<P>
5873
Accepts a TLS client connection via <I>bsock</I>. Returns <I>true</I> if
5874
successful, <I>false</I> otherwise. Will fail if there is a TLS protocol
5875
error or an invalid certificate is presented.
5876
5877
<P>
5878
<PRE>
5879
bool tls_bsock_shutdown (BSOCK *bsock);
5880
</PRE>
5881
<P>
5882
Issues a blocking TLS shutdown request to the peer via <I>bsock</I>. This function may not wait for the peer's reply.
5883
5884
<P>
5885
<PRE>
5886
int tls_bsock_writen (BSOCK *bsock, char *ptr, int32_t nbytes);
5887
</PRE>
5888
<P>
5889
Writes <I>nbytes</I> from <I>ptr</I> via the <I>TLS_CONNECTION</I>
5890
associated with <I>bsock</I>. Due to OpenSSL's handling of <I>EINTR</I>,
5891
<I>bsock</I> is set non-blocking at the start of the function, and restored
5892
to its original blocking state before the function returns. Less than
5893
<I>nbytes</I> may be written if an error occurs. The actual number of
5894
bytes written will be returned.
5895
5896
<P>
5897
<PRE>
5898
int tls_bsock_readn (BSOCK *bsock, char *ptr, int32_t nbytes);
5899
</PRE>
5900
<P>
5901
Reads <I>nbytes</I> from the <I>TLS_CONNECTION</I> associated with
5902
<I>bsock</I> and stores the result in <I>ptr</I>. Due to OpenSSL's
5903
handling of <I>EINTR</I>, <I>bsock</I> is set non-blocking at the start of
5904
the function, and restored to its original blocking state before the
5905
function returns. Less than <I>nbytes</I> may be read if an error occurs.
5906
The actual number of bytes read will be returned.
5907
5908
<P>
5909
5910
<H2><A NAME="SECTION000144000000000000000"></A>
5911
<A NAME="3631"></A>
5912
<A NAME="3632"></A>
5913
<BR>
5914
Bnet API Changes
5915
</H2>
5916
<A NAME="3635"></A>
5917
5918
<P>
5919
A minimal number of changes were required in the Bnet socket API. The BSOCK
5920
structure was expanded to include an associated TLS_CONNECTION structure,
5921
as well as a flag to designate the current blocking state of the socket.
5922
The blocking state flag is required for win32, where it does not appear
5923
possible to discern the current blocking state of a socket.
5924
5925
<P>
5926
5927
<H3><A NAME="SECTION000144100000000000000"></A>
5928
<A NAME="3637"></A>
5929
<A NAME="3638"></A>
5930
<BR>
5931
Negotiating a TLS Connection
5932
</H3>
5933
<A NAME="3641"></A>
5934
5935
<P>
5936
<I>bnet_tls_server()</I> and <I>bnet_tls_client()</I> were both
5937
implemented using the new TLS API as follows:
5938
5939
<P>
5940
<PRE>
5941
int bnet_tls_client(TLS_CONTEXT *ctx, BSOCK * bsock);
5942
</PRE>
5943
<P>
5944
Negotiates a TLS session via <I>bsock</I> using the settings from
5945
<I>ctx</I>. Returns 1 if successful, 0 otherwise.
5946
5947
<P>
5948
<PRE>
5949
int bnet_tls_server(TLS_CONTEXT *ctx, BSOCK * bsock, alist *verify_list);
5950
</PRE>
5951
<P>
5952
Accepts a TLS client session via <I>bsock</I> using the settings from
5953
<I>ctx</I>. If <I>verify_list</I> is non-NULL, it is passed to
5954
<I>tls_postconnect_verify_cn()</I> for client certificate verification.
5955
5956
<P>
5957
5958
<H3><A NAME="SECTION000144200000000000000"></A>
5959
<A NAME="3655"></A>
5960
<A NAME="3656"></A>
5961
<A NAME="3657"></A>
5962
<BR>
5963
Manipulating Socket Blocking State
5964
</H3>
5965
<A NAME="3660"></A>
5966
5967
<P>
5968
Three functions were added for manipulating the blocking state of a socket
5969
on both Win32 and Unix-like systems. The Win32 code was written according
5970
to the MSDN documentation, but has not been tested.
5971
5972
<P>
5973
These functions are prototyped as follows:
5974
5975
<P>
5976
<PRE>
5977
int bnet_set_nonblocking (BSOCK *bsock);
5978
</PRE>
5979
<P>
5980
Enables non-blocking I/O on the socket associated with <I>bsock</I>.
5981
Returns a copy of the socket flags prior to modification.
5982
5983
<P>
5984
<PRE>
5985
int bnet_set_blocking (BSOCK *bsock);
5986
</PRE>
5987
<P>
5988
Enables blocking I/O on the socket associated with <I>bsock</I>. Returns a
5989
copy of the socket flags prior to modification.
5990
5991
<P>
5992
<PRE>
5993
void bnet_restore_blocking (BSOCK *bsock, int flags);
5994
</PRE>
5995
<P>
5996
Restores blocking or non-blocking IO setting on the socket associated with
5997
<I>bsock</I>. The <I>flags</I> argument must be the return value of either
5998
<I>bnet_set_blocking()</I> or <I>bnet_restore_blocking()</I>.
5999
6000
<P>
6001
6002
<P>
6003
6004
<H2><A NAME="SECTION000145000000000000000"></A>
6005
<A NAME="3674"></A>
6006
<A NAME="3675"></A>
6007
<BR>
6008
Authentication Negotiation
6009
</H2>
6010
<A NAME="3678"></A>
6011
6012
<P>
6013
Backwards compatibility with the existing SSL negotiation hooks implemented
6014
in src/lib/cram-md5.c have been maintained. The
6015
<I>cram_md5_get_auth()</I> function has been modified to accept an
6016
integer pointer argument, tls_remote_need. The TLS requirement
6017
advertised by the remote host is returned via this pointer.
6018
6019
<P>
6020
After exchanging cram-md5 authentication and TLS requirements, both the
6021
client and server independently decide whether to continue:
6022
6023
<P>
6024
<PRE>
6025
if (!cram_md5_get_auth(dir, password, &tls_remote_need) ||
6026
!cram_md5_auth(dir, password, tls_local_need)) {
6027
[snip]
6028
/* Verify that the remote host is willing to meet our TLS requirements */
6029
if (tls_remote_need < tls_local_need && tls_local_need != BNET_TLS_OK &&
6030
tls_remote_need != BNET_TLS_OK) {
6031
sendit(_("Authorization problem:"
6032
" Remote server did not advertise required TLS support.\n"));
6033
auth_success = false;
6034
goto auth_done;
6035
}
6036
6037
/* Verify that we are willing to meet the remote host's requirements */
6038
if (tls_remote_need > tls_local_need && tls_local_need != BNET_TLS_OK &&
6039
tls_remote_need != BNET_TLS_OK) {
6040
sendit(_("Authorization problem:"
6041
" Remote server requires TLS.\n"));
6042
auth_success = false;
6043
goto auth_done;
6044
}
6045
</PRE>
6046
<P>
6047
6048
<H1><A NAME="SECTION000150000000000000000"></A>
6049
<A NAME="_ChapterStart8"></A><A NAME="3912"></A>
6050
<A NAME="3913"></A>
6051
<BR>
6052
Bacula Regression Testing
6053
</H1>
6054
<A NAME="3916"></A>
6055
6056
<P>
6057
6058
<H2><A NAME="SECTION000151000000000000000"></A>
6059
<A NAME="3918"></A>
6060
<BR>
6061
General
6062
</H2>
6063
<A NAME="3921"></A>
6064
6065
<P>
6066
This document is intended mostly for developers who wish to ensure that their
6067
changes to Bacula don't introduce bugs in the base code.
6068
6069
<P>
6070
You can find the existing regression script in the Bacula CVS on the
6071
SourceForge CVS in the project tree named <B>regress</B>.
6072
6073
<P>
6074
There are two different aspects of regression testing that this document will
6075
discuss: 1. Running the Regression Script, 2. Writing a Regression test.
6076
6077
<P>
6078
6079
<H2><A NAME="SECTION000152000000000000000"></A>
6080
<A NAME="3924"></A>
6081
<A NAME="3925"></A>
6082
<BR>
6083
Running the Regression Script
6084
</H2>
6085
<A NAME="3928"></A>
6086
6087
<P>
6088
There are a number of different tests that may be run, such as: the standard
6089
set that uses disk Volumes and runs under any userid; a small set of tests
6090
that write to tape; another set of tests where you must be root to run them.
6091
To date, each subset of tests runs no more than about 15 minutes.
6092
6093
<P>
6094
6095
<H3><A NAME="SECTION000152100000000000000"></A>
6096
<A NAME="3930"></A>
6097
<A NAME="3931"></A>
6098
<BR>
6099
Setting the Configuration Parameters
6100
</H3>
6101
<A NAME="3934"></A>
6102
6103
<P>
6104
Once you have the regression directory loaded, you will first need to create a
6105
custom xxx.conf file for your system. You can either edit <B>prototype.conf</B>
6106
directly or copy it to a new file and edit it. To see a real example of a
6107
configuration file, look at <B>kern.conf</B>. The variables you need to modify
6108
are:
6109
6110
<P>
6111
<PRE>
6112
6113
# Where to get the source to be tested
6114
# BACULA_SOURCE="${HOME}/bacula/branch-1.36.2"
6115
BACULA_SOURCE="${HOME}/bacula/k"
6116
6117
# Where to send email !!!!! Change me !!!!!!!
6118
EMAIL=kern@sibbald.com
6119
SMTP_HOST="localhost"
6120
6121
# Full "default" path where to find sqlite (no quotes!)
6122
#SQLITE_DIR=${HOME}/bacula/depkgs/sqlite3
6123
SQLITE_DIR=${HOME}/bacula/depkgs/sqlite
6124
6125
TAPE_DRIVE="/dev/nst0"
6126
# if you don't have an autochanger set AUTOCHANGER to /dev/null
6127
AUTOCHANGER="/dev/sg0"
6128
# For two drive tests -- set to /dev/null if you do not have it
6129
TAPE_DRIVE1="/dev/nst1"
6130
6131
# This must be the path to the autochanger including its name
6132
AUTOCHANGER_PATH="/usr/sbin/mtx"
6133
6134
# Set your database here
6135
#SQLITE_DIR=${HOME}/bacula/depkgs/sqlite3
6136
WHICHDB?="--with-sqlite=${SQLITE_DIR}"
6137
#WHICHDB="--with-mysql=${HOME}/mysql"
6138
6139
# Set this to "--with-tcp-wrappers" or "--without-tcp-wrappers"
6140
TCPWRAPPERS="--with-tcp-wrappers"
6141
</PRE>
6142
<P>
6143
6144
<UL>
6145
<LI><B>BACULA_SOURCE</B> should be the full path to the Bacula source code
6146
that you wish to test.
6147
</LI>
6148
<LI><B>EMAIL</B> should be your email addres. Please remember to change this
6149
or I will get a flood of unwanted messages. You may or may not want to see
6150
these emails. In my case, I don't need them so I direct it to the bit bucket.
6151
</LI>
6152
<LI><B>SMTP_HOST</B> defines where your SMTP server is.
6153
6154
<P>
6155
</LI>
6156
<LI><B>SQLITE_DIR</B> should be the full path to the sqlite package, must
6157
be build before running a Bacula regression, if you are using SQLite. This
6158
variable is ignored if you are using MySQL or PostgreSQL. To use PostgreSQL,
6159
edit the Makefile and change (or add) WHICHDB?=``<code>--</code>with-postgresql''. For
6160
MySQL use ``WHICHDB?=''<code>--</code>with-mysql``.
6161
6162
<P>
6163
</LI>
6164
<LI><B>TAPE_DRIVE</B> is the full path to your tape drive. The base set of
6165
regression tests do not use a tape, so this is only important if you want to
6166
run the full tests.
6167
6168
<P>
6169
</LI>
6170
<LI><B>TAPE_DRIVE1</B> is the full path to your second tape drive, if
6171
have one. The base set of
6172
regression tests do not use a tape, so this is only important if you want to
6173
run the full two drive tests.
6174
6175
<P>
6176
</LI>
6177
<LI><B>AUTOCHANGER</B> is the name of your autochanger device. Set this to
6178
/dev/null if you do not have one.
6179
6180
<P>
6181
</LI>
6182
<LI><B>AUTOCHANGER_PATH</B> is the full path including the program name for
6183
your autochanger program (normally <B>mtx</B>. Leave the default value if you
6184
do not have one.
6185
6186
<P>
6187
</LI>
6188
<LI><B>TCPWRAPPERS</B> defines whether or not you want the ./configure
6189
to be performed with tcpwrappers enabled.
6190
6191
<P>
6192
</LI>
6193
</UL>
6194
6195
<P>
6196
6197
<H3><A NAME="SECTION000152200000000000000"></A>
6198
<A NAME="3952"></A>
6199
<A NAME="3953"></A>
6200
<BR>
6201
Building the Test Bacula
6202
</H3>
6203
<A NAME="3956"></A>
6204
6205
<P>
6206
Once the above variables are set, you can build Bacula by entering:
6207
6208
<P>
6209
<PRE>
6210
./config xxx.conf
6211
make setup
6212
</PRE>
6213
<P>
6214
Where xxx.conf is the name of the conf file containing your system parameters.
6215
This will build a Makefile from Makefile.in, then copy the source code within
6216
the regression tree (in directory regress/build), configure it, and build it.
6217
There should be no errors. If there are, please correct them before
6218
continuing.
6219
6220
<P>
6221
6222
<H3><A NAME="SECTION000152300000000000000"></A>
6223
<A NAME="3960"></A>
6224
<A NAME="3961"></A>
6225
<BR>
6226
Running the Disk Only Regression
6227
</H3>
6228
<A NAME="3964"></A>
6229
6230
<P>
6231
Once Bacula is built, you can run the basic disk only non-root regression test
6232
by entering:
6233
6234
<P>
6235
<PRE>
6236
make test
6237
</PRE>
6238
<P>
6239
This will run the base set of tests using disk Volumes, currently (19 Dec
6240
2003), there are current 18 separate tests that run. If you are testing on a
6241
non-Linux machine two of the tests will not be run. In any case, as we add new
6242
tests, the number will vary. It will take about 5 or 10 minutes if you have a
6243
fast (2 GHz) machine, and you don't need to be root to run these tests (I run
6244
under my regular userid). The result should be something similar to:
6245
6246
<P>
6247
<PRE>
6248
Test results
6249
6250
===== Backup Bacula Test OK =====
6251
===== Verify Volume Test OK =====
6252
===== sparse-test OK =====
6253
===== compressed-test OK =====
6254
===== sparse-compressed-test OK =====
6255
===== Weird files test OK =====
6256
===== two-jobs-test OK =====
6257
===== two-vol-test OK =====
6258
===== six-vol-test OK =====
6259
===== bscan-test OK =====
6260
===== Weird files2 test OK =====
6261
===== concurrent-jobs-test OK =====
6262
===== four-concurrent-jobs-test OK =====
6263
===== bsr-opt-test OK =====
6264
===== bextract-test OK =====
6265
===== recycle-test OK =====
6266
===== span-vol-test OK =====
6267
===== restore-by-file-test OK =====
6268
===== restore2-by-file-test OK =====
6269
===== four-jobs-test OK =====
6270
===== incremental-test OK =====
6271
</PRE>
6272
<P>
6273
and the working tape tests are:
6274
6275
<P>
6276
<PRE>
6277
Test results
6278
6279
===== Bacula tape test OK =====
6280
===== Small File Size test OK =====
6281
===== restore-by-file-tape test OK =====
6282
===== incremental-tape test OK =====
6283
===== four-concurrent-jobs-tape OK =====
6284
===== four-jobs-tape OK =====
6285
</PRE>
6286
<P>
6287
Each separate test is self contained in that it initializes to run Bacula from
6288
scratch (i.e. newly created database). It will also kill any Bacula session
6289
that is currently running. In addition, it uses ports 8101, 8102, and 8103 so
6290
that it does not intefere with a production system.
6291
6292
<P>
6293
6294
<H3><A NAME="SECTION000152400000000000000"></A>
6295
<A NAME="3972"></A>
6296
<A NAME="3973"></A>
6297
<BR>
6298
Other Tests
6299
</H3>
6300
<A NAME="3976"></A>
6301
6302
<P>
6303
There are a number of other tests that can be run as well. All the tests are a
6304
simply shell script keep in the regress directory. For example the ''make
6305
test`` simply executes <B>./all-non-root-tests</B>. The other tests are:
6306
6307
<P>
6308
<DL>
6309
<DT><STRONG>all_non-root-tests</STRONG></DT>
6310
<DD><A NAME="3979"></A>
6311
All non-tape tests not requiring root. This is the standard set of tests,
6312
that in general, backup some data, then restore it, and finally compares the
6313
restored data with the original data.
6314
6315
<P>
6316
</DD>
6317
<DT><STRONG>all-root-tests</STRONG></DT>
6318
<DD><A NAME="3980"></A>
6319
All non-tape tests requiring root permission. These are a relatively small
6320
number of tests that require running as root. The amount of data backed up
6321
can be quite large. For example, one test backs up /usr, another backs up
6322
/etc. One or more of these tests reports an error -- I'll fix it one day.
6323
6324
<P>
6325
</DD>
6326
<DT><STRONG>all-non-root-tape-tests</STRONG></DT>
6327
<DD><A NAME="3981"></A>
6328
All tape test not requiring root. There are currently three tests, all run
6329
without being root, and backup to a tape. The first two tests use one volume,
6330
and the third test requires an autochanger, and uses two volumes. If you
6331
don't have an autochanger, then this script will probably produce an error.
6332
6333
<P>
6334
</DD>
6335
<DT><STRONG>all-tape-and-file-tests</STRONG></DT>
6336
<DD><A NAME="3982"></A>
6337
All tape and file tests not requiring root. This includes just about
6338
everything, and I don't run it very often.
6339
</DD>
6340
</DL>
6341
6342
<P>
6343
6344
<H3><A NAME="SECTION000152500000000000000"></A>
6345
<A NAME="3985"></A>
6346
<A NAME="3986"></A>
6347
<BR>
6348
If a Test Fails
6349
</H3>
6350
<A NAME="3989"></A>
6351
6352
<P>
6353
If you one or more tests fail, the line output will be similar to:
6354
6355
<P>
6356
<PRE>
6357
!!!!! concurrent-jobs-test failed!!! !!!!!
6358
</PRE>
6359
<P>
6360
If you want to determine why the test failed, you will need to modify the
6361
script so that it prints. Do so by finding the file in <B>regress/tests</B>
6362
that corresponds to the name printed. For example, the script for the above
6363
error message is in: <B>regress/tests/concurrent-jobs-test</B>.
6364
6365
<P>
6366
In order to see the output produced by Bacula, you need only change the lines
6367
that start with <B>@output</B> to <B>@tee</B>, then rerun the test by hand. it
6368
is very important to start the test from the <B>regress</B> directory.
6369
6370
<P>
6371
To modify the test mentioned above so that you can see the output, change
6372
every occurrence of <B>@output</B> to <B>@tee</B> in the file. In rare cases you
6373
might need to remove the <B>2>&1 >/dev/null</B> from the end of the
6374
<B>bacula</B>, <B>bconsole</B>, or <B>diff</B> lines, but this is rare.
6375
6376
<P>
6377
6378
<H2><A NAME="SECTION000153000000000000000"></A>
6379
<A NAME="4005"></A>
6380
<A NAME="4006"></A>
6381
<BR>
6382
Writing a Regression Test
6383
</H2>
6384
<A NAME="4009"></A>
6385
6386
<P>
6387
Any developer, who implements a major new feature, should write a regression
6388
test that exercises and validates the new feature. Each regression test is a
6389
complete test by itself. It terminates any running Bacula, initializes the
6390
database, starts Bacula, then runs the test by using the console program.
6391
6392
<P>
6393
6394
<H3><A NAME="SECTION000153100000000000000"></A>
6395
<A NAME="4011"></A>
6396
<A NAME="4012"></A>
6397
<BR>
6398
Running the Tests by Hand
6399
</H3>
6400
<A NAME="4015"></A>
6401
6402
<P>
6403
You can run any individual test by hand by cd'ing to the <B>regress</B>
6404
directory and entering:
6405
6406
<P>
6407
<PRE>
6408
tests/<test-name>
6409
</PRE>
6410
<P>
6411
6412
<H3><A NAME="SECTION000153200000000000000"></A>
6413
<A NAME="4020"></A>
6414
<A NAME="4021"></A>
6415
<BR>
6416
Directory Structure
6417
</H3>
6418
<A NAME="4024"></A>
6419
6420
<P>
6421
The directory structure of the regression tests is:
6422
6423
<P>
6424
<PRE>
6425
regress - Makefile, scripts to start tests
6426
|------ scripts - Scripts and conf files
6427
|-------tests - All test scripts are here
6428
|
6429
|------------------ -- All directories below this point are used
6430
| for testing, but are created from the
6431
| above directories and are removed with
6432
| "make distclean"
6433
|
6434
|------ bin - This is the install directory for
6435
| Bacula to be used testing
6436
|------ build - Where the Bacula source build tree is
6437
|------ tmp - Most temp files go here
6438
|------ working - Bacula working directory
6439
|------ weird-files - Weird files used in two of the tests.
6440
</PRE>
6441
<P>
6442
6443
<H3><A NAME="SECTION000153300000000000000"></A>
6444
<A NAME="4028"></A>
6445
<A NAME="4029"></A>
6446
<BR>
6447
Adding a New Test
6448
</H3>
6449
<A NAME="4032"></A>
6450
6451
<P>
6452
If you want to write a new regression test, it is best to start with one of
6453
the existing test scripts, and modify it to do the new test.
6454
6455
<P>
6456
When adding a new test, be extremely careful about adding anything to any of
6457
the daemons' configuration files. The reason is that it may change the prompts
6458
that are sent to the console. For example, adding a Pool means that the
6459
current scripts, which assume that Bacula automatically selects a Pool, will
6460
now be presented with a new prompt, so the test will fail. If you need to
6461
enhance the configuration files, consider making your own versions.
6462
6463
<P>
6464
6465
<H1><A NAME="SECTION000160000000000000000"></A>
6466
<A NAME="_ChapterStart"></A>
6467
<BR>
6468
Bacula MD5 Algorithm
6469
</H1>
6470
<A NAME="4230"></A>
6471
6472
<P>
6473
6474
<H2><A NAME="SECTION000161000000000000000"></A>
6475
<A NAME="4232"></A>
6476
<A NAME="4233"></A>
6477
<BR>
6478
Command Line Message Digest Utility
6479
</H2>
6480
<A NAME="4236"></A>
6481
6482
<P>
6483
This page describes <B>md5</B>, a command line utility usable on either Unix or
6484
MS-DOS/Windows, which generates and verifies message digests (digital
6485
signatures) using the MD5 algorithm. This program can be useful when
6486
developing shell scripts or Perl programs for software installation, file
6487
comparison, and detection of file corruption and tampering.
6488
6489
<P>
6490
6491
<H3><A NAME="SECTION000161100000000000000"></A>
6492
<A NAME="4239"></A>
6493
<BR>
6494
Name
6495
</H3>
6496
<A NAME="4242"></A>
6497
6498
<P>
6499
<B>md5</B> - generate / check MD5 message digest
6500
6501
<P>
6502
6503
<H3><A NAME="SECTION000161200000000000000"></A>
6504
<A NAME="4245"></A>
6505
<BR>
6506
Synopsis
6507
</H3>
6508
<A NAME="4248"></A>
6509
6510
<P>
6511
<B>md5</B> [ <B>-c</B><I>signature</I> ] [ <B>-u</B> ] [ <B>-d</B><I>input_text</I>
6512
| <I>infile</I> ] [ <I>outfile</I> ]
6513
6514
<P>
6515
6516
<H3><A NAME="SECTION000161300000000000000"></A>
6517
<A NAME="4258"></A>
6518
<BR>
6519
Description
6520
</H3>
6521
<A NAME="4261"></A>
6522
6523
<P>
6524
A <I>message digest</I> is a compact digital signature for an arbitrarily long
6525
stream of binary data. An ideal message digest algorithm would never generate
6526
the same signature for two different sets of input, but achieving such
6527
theoretical perfection would require a message digest as long as the input
6528
file. Practical message digest algorithms compromise in favour of a digital
6529
signature of modest size created with an algorithm designed to make
6530
preparation of input text with a given signature computationally infeasible.
6531
Message digest algorithms have much in common with techniques used in
6532
encryption, but to a different end; verification that data have not been
6533
altered since the signature was published.
6534
6535
<P>
6536
Many older programs requiring digital signatures employed 16 or 32 bit <I>cyclical redundancy codes</I> (CRC) originally developed to verify correct
6537
transmission in data communication protocols, but these short codes, while
6538
adequate to detect the kind of transmission errors for which they were
6539
intended, are insufficiently secure for applications such as electronic
6540
commerce and verification of security related software distributions.
6541
6542
<P>
6543
The most commonly used present-day message digest algorithm is the 128 bit MD5
6544
algorithm, developed by Ron Rivest of the
6545
<A NAME="tex2html3"
6546
HREF="http://web.mit.edu/">MIT</A>
6547
<A NAME="tex2html4"
6548
HREF="http://www.lcs.mit.edu/">Laboratory for Computer Science</A>
6549
and
6550
<A NAME="tex2html5"
6551
HREF="http://www.rsa.com/">RSA Data Security, Inc.</A>
6552
The algorithm, with a
6553
reference implementation, was published as Internet
6554
<A NAME="tex2html6"
6555
HREF="http://www.fourmilab.ch/md5/rfc1321.html">RFC 1321</A>
6556
in April 1992, and
6557
was placed into the public domain at that time. Message digest algorithms such
6558
as MD5 are not deemed ``encryption technology'' and are not subject to the
6559
export controls some governments impose on other data security products.
6560
(Obviously, the responsibility for obeying the laws in the jurisdiction in
6561
which you reside is entirely your own, but many common Web and Mail utilities
6562
use MD5, and I am unaware of any restrictions on their distribution and use.)
6563
6564
<P>
6565
The MD5 algorithm has been implemented in numerous computer languages
6566
including C,
6567
<A NAME="tex2html7"
6568
HREF="http://www.perl.org/">Perl</A>, and
6569
<A NAME="tex2html8"
6570
HREF="http://www.javasoft.com/">Java</A>; if you're writing a program in such a
6571
language, track down a suitable subroutine and incorporate it into your
6572
program. The program described on this page is a <I>command line</I>
6573
implementation of MD5, intended for use in shell scripts and Perl programs (it
6574
is much faster than computing an MD5 signature directly in Perl). This <B>md5</B> program was originally developed as part of a suite of tools intended to
6575
monitor large collections of files (for example, the contents of a Web site)
6576
to detect corruption of files and inadvertent (or perhaps malicious) changes.
6577
That task is now best accomplished with more comprehensive packages such as
6578
<A NAME="tex2html9"
6579
HREF="ftp://coast.cs.purdue.edu/pub/COAST/Tripwire/">Tripwire</A>, but the
6580
command line <B>md5</B> component continues to prove useful for verifying
6581
correct delivery and installation of software packages, comparing the contents
6582
of two different systems, and checking for changes in specific files.
6583
6584
<P>
6585
6586
<H3><A NAME="SECTION000161400000000000000"></A>
6587
<A NAME="4282"></A>
6588
<BR>
6589
Options
6590
</H3>
6591
<A NAME="4285"></A>
6592
6593
<P>
6594
<DL>
6595
<DT><STRONG><B>-c</B><I>signature</I> </STRONG></DT>
6596
<DD><A NAME="4289"></A>
6597
Computes the signature of the specified <I>infile</I> or the string supplied
6598
by the <B>-d</B> option and compares it against the specified <I>signature</I>.
6599
If the two signatures match, the exit status will be zero, otherwise the exit
6600
status will be 1. No signature is written to <I>outfile</I> or standard
6601
output; only the exit status is set. The signature to be checked must be
6602
specified as 32 hexadecimal digits.
6603
6604
<P>
6605
</DD>
6606
<DT><STRONG><B>-d</B><I>input_text</I> </STRONG></DT>
6607
<DD><A NAME="4296"></A>
6608
A signature is computed for the given <I>input_text</I> (which must be quoted
6609
if it contains white space characters) instead of input from <I>infile</I> or
6610
standard input. If input is specified with the <B>-d</B> option, no <I>infile</I> should be specified.
6611
6612
<P>
6613
</DD>
6614
<DT><STRONG><B>-u</B> </STRONG></DT>
6615
<DD>Print how-to-call information.
6616
6617
</DD>
6618
</DL>
6619
6620
<P>
6621
6622
<H3><A NAME="SECTION000161500000000000000"></A>
6623
<A NAME="4304"></A>
6624
<BR>
6625
Files
6626
</H3>
6627
<A NAME="4307"></A>
6628
6629
<P>
6630
If no <I>infile</I> or <B>-d</B> option is specified or <I>infile</I> is a single
6631
``-'', <B>md5</B> reads from standard input; if no <I>outfile</I> is given, or
6632
<I>outfile</I> is a single ``-'', output is sent to standard output. Input and
6633
output are processed strictly serially; consequently <B>md5</B> may be used in
6634
pipelines.
6635
6636
<P>
6637
6638
<H3><A NAME="SECTION000161600000000000000"></A>
6639
<A NAME="4316"></A>
6640
<BR>
6641
Bugs
6642
</H3>
6643
<A NAME="4319"></A>
6644
6645
<P>
6646
The mechanism used to set standard input to binary mode may be specific to
6647
Microsoft C; if you rebuild the DOS/Windows version of the program from source
6648
using another compiler, be sure to verify binary files work properly when read
6649
via redirection or a pipe.
6650
6651
<P>
6652
This program has not been tested on a machine on which <TT>int</TT> and/or <TT>long</TT> are longer than 32 bits.
6653
6654
<P>
6655
6656
<H2><A NAME="SECTION000162000000000000000"></A>
6657
<A NAME="4324"></A>
6658
<A NAME="4325"></A>
6659
<BR>
6660
6661
<A NAME="tex2html10"
6662
HREF="http://www.fourmilab.ch/md5/md5.zip">Download md5.zip</A>
6663
(Zipped
6664
archive)
6665
</H2>
6666
<A NAME="4328"></A>
6667
6668
<P>
6669
The program is provided as
6670
<A NAME="tex2html11"
6671
HREF="http://www.fourmilab.ch/md5/md5.zip">md5.zip</A>, a
6672
<A NAME="tex2html12"
6673
HREF="http://www.pkware.com/">Zipped</A>
6674
archive containing an ready-to-run
6675
Win32 command-line executable program, <TT>md5.exe</TT> (compiled using Microsoft
6676
Visual C++ 5.0), and in source code form along with a <TT>Makefile</TT> to build
6677
the program under Unix.
6678
6679
<P>
6680
6681
<H3><A NAME="SECTION000162100000000000000"></A>
6682
<A NAME="4336"></A>
6683
<A NAME="4337"></A>
6684
<BR>
6685
See Also
6686
</H3>
6687
<A NAME="4340"></A>
6688
6689
<P>
6690
<B>sum</B>(1)
6691
6692
<P>
6693
6694
<H3><A NAME="SECTION000162200000000000000"></A>
6695
<A NAME="4343"></A>
6696
<A NAME="4344"></A>
6697
<BR>
6698
Exit Status
6699
</H3>
6700
<A NAME="4347"></A>
6701
6702
<P>
6703
<B>md5</B> returns status 0 if processing was completed without errors, 1 if
6704
the <B>-c</B> option was specified and the given signature does not match that
6705
of the input, and 2 if processing could not be performed at all due, for
6706
example, to a nonexistent input file.
6707
6708
<P>
6709
6710
<H3><A NAME="SECTION000162300000000000000"></A>
6711
<A NAME="4351"></A>
6712
<BR>
6713
Copying
6714
</H3>
6715
<A NAME="4354"></A>
6716
6717
<P>
6718
<BLOCKQUOTE>
6719
This software is in the public domain. Permission to use, copy, modify, and
6720
distribute this software and its documentation for any purpose and without
6721
fee is hereby granted, without any conditions or restrictions. This software
6722
is provided ``as is'' without express or implied warranty.
6723
6724
</BLOCKQUOTE>
6725
6726
<P>
6727
6728
<H3><A NAME="SECTION000162400000000000000"></A>
6729
<A NAME="4358"></A>
6730
<BR>
6731
Acknowledgements
6732
</H3>
6733
<A NAME="4361"></A>
6734
6735
<P>
6736
The MD5 algorithm was developed by Ron Rivest. The public domain C language
6737
implementation used in this program was written by Colin Plumb in 1993.
6738
<I><A NAME="tex2html13"
6739
HREF="http://www.fourmilab.ch/">by John Walker</A>
6740
January 6th, MIM </I>
6741
6742
<P>
6743
6744
<H1><A NAME="SECTION000170000000000000000"></A>
6745
<A NAME="_ChapterStart7"></A><A NAME="4518"></A>
6746
<A NAME="4519"></A>
6747
<BR>
6748
Bacula Memory Management
6749
</H1>
6750
<A NAME="4522"></A>
6751
6752
<P>
6753
6754
<H2><A NAME="SECTION000171000000000000000"></A>
6755
<A NAME="4524"></A>
6756
<BR>
6757
General
6758
</H2>
6759
<A NAME="4527"></A>
6760
6761
<P>
6762
This document describes the memory management routines that are used in Bacula
6763
and is meant to be a technical discussion for developers rather than part of
6764
the user manual.
6765
6766
<P>
6767
Since Bacula may be called upon to handle filenames of varying and more or
6768
less arbitrary length, special attention needs to be used in the code to
6769
ensure that memory buffers are sufficiently large. There are four
6770
possibilities for memory usage within <B>Bacula</B>. Each will be described in
6771
turn. They are:
6772
6773
<P>
6774
6775
<UL>
6776
<LI>Statically allocated memory.
6777
</LI>
6778
<LI>Dynamically allocated memory using malloc() and free().
6779
</LI>
6780
<LI>Non-pooled memory.
6781
</LI>
6782
<LI>Pooled memory.
6783
6784
</LI>
6785
</UL>
6786
6787
<P>
6788
6789
<H3><A NAME="SECTION000171100000000000000"></A>
6790
<A NAME="4532"></A>
6791
<A NAME="4533"></A>
6792
<BR>
6793
Statically Allocated Memory
6794
</H3>
6795
<A NAME="4536"></A>
6796
6797
<P>
6798
Statically allocated memory is of the form:
6799
6800
<P>
6801
<PRE>
6802
char buffer[MAXSTRING];
6803
</PRE>
6804
<P>
6805
The use of this kind of memory is discouraged except when you are 100% sure
6806
that the strings to be used will be of a fixed length. One example of where
6807
this is appropriate is for <B>Bacula</B> resource names, which are currently
6808
limited to 127 characters (MAX_NAME_LENGTH). Although this maximum size may
6809
change, particularly to accommodate Unicode, it will remain a relatively small
6810
value.
6811
6812
<P>
6813
6814
<H3><A NAME="SECTION000171200000000000000"></A>
6815
<A NAME="4541"></A>
6816
<A NAME="4542"></A>
6817
<BR>
6818
Dynamically Allocated Memory
6819
</H3>
6820
<A NAME="4545"></A>
6821
6822
<P>
6823
Dynamically allocated memory is obtained using the standard malloc() routines.
6824
As in:
6825
6826
<P>
6827
<PRE>
6828
char *buf;
6829
buf = malloc(256);
6830
</PRE>
6831
<P>
6832
This kind of memory can be released with:
6833
6834
<P>
6835
<PRE>
6836
free(buf);
6837
</PRE>
6838
<P>
6839
It is recommended to use this kind of memory only when you are sure that you
6840
know the memory size needed and the memory will be used for short periods of
6841
time -- that is it would not be appropriate to use statically allocated
6842
memory. An example might be to obtain a large memory buffer for reading and
6843
writing files. When <B>SmartAlloc</B> is enabled, the memory obtained by
6844
malloc() will automatically be checked for buffer overwrite (overflow) during
6845
the free() call, and all malloc'ed memory that is not released prior to
6846
termination of the program will be reported as Orphaned memory.
6847
6848
<P>
6849
6850
<H3><A NAME="SECTION000171300000000000000"></A>
6851
<A NAME="4552"></A>
6852
<A NAME="4553"></A>
6853
<BR>
6854
Pooled and Non-pooled Memory
6855
</H3>
6856
<A NAME="4556"></A>
6857
6858
<P>
6859
In order to facility the handling of arbitrary length filenames and to
6860
efficiently handle a high volume of dynamic memory usage, we have implemented
6861
routines between the C code and the malloc routines. The first is called
6862
``Pooled'' memory, and is memory, which once allocated and then released, is
6863
not returned to the system memory pool, but rather retained in a Bacula memory
6864
pool. The next request to acquire pooled memory will return any free memory
6865
block. In addition, each memory block has its current size associated with the
6866
block allowing for easy checking if the buffer is of sufficient size. This
6867
kind of memory would normally be used in high volume situations (lots of
6868
malloc()s and free()s) where the buffer length may have to frequently change
6869
to adapt to varying filename lengths.
6870
6871
<P>
6872
The non-pooled memory is handled by routines similar to those used for pooled
6873
memory, allowing for easy size checking. However, non-pooled memory is
6874
returned to the system rather than being saved in the Bacula pool. This kind
6875
of memory would normally be used in low volume situations (few malloc()s and
6876
free()s), but where the size of the buffer might have to be adjusted
6877
frequently.
6878
6879
<P>
6880
6881
<H4><A NAME="SECTION000171310000000000000">
6882
Types of Memory Pool:</A>
6883
</H4>
6884
6885
<P>
6886
Currently there are three memory pool types:
6887
6888
<P>
6889
6890
<UL>
6891
<LI>PM_NOPOOL -- non-pooled memory.
6892
</LI>
6893
<LI>PM_FNAME -- a filename pool.
6894
</LI>
6895
<LI>PM_MESSAGE -- a message buffer pool.
6896
</LI>
6897
<LI>PM_EMSG -- error message buffer pool.
6898
6899
</LI>
6900
</UL>
6901
6902
<P>
6903
6904
<H4><A NAME="SECTION000171320000000000000">
6905
Getting Memory:</A>
6906
</H4>
6907
6908
<P>
6909
To get memory, one uses:
6910
6911
<P>
6912
<PRE>
6913
void *get_pool_memory(pool);
6914
</PRE>
6915
<P>
6916
where <B>pool</B> is one of the above mentioned pool names. The size of the
6917
memory returned will be determined by the system to be most appropriate for
6918
the application.
6919
6920
<P>
6921
If you wish non-pooled memory, you may alternatively call:
6922
6923
<P>
6924
<PRE>
6925
void *get_memory(size_t size);
6926
</PRE>
6927
<P>
6928
The buffer length will be set to the size specified, and it will be assigned
6929
to the PM_NOPOOL pool (no pooling).
6930
6931
<P>
6932
6933
<H4><A NAME="SECTION000171330000000000000">
6934
Releasing Memory:</A>
6935
</H4>
6936
6937
<P>
6938
To free memory acquired by either of the above two calls, use:
6939
6940
<P>
6941
<PRE>
6942
void free_pool_memory(void *buffer);
6943
</PRE>
6944
<P>
6945
where buffer is the memory buffer returned when the memory was acquired. If
6946
the memory was originally allocated as type PM_NOPOOL, it will be released to
6947
the system, otherwise, it will be placed on the appropriate Bacula memory pool
6948
free chain to be used in a subsequent call for memory from that pool.
6949
6950
<P>
6951
6952
<H4><A NAME="SECTION000171340000000000000">
6953
Determining the Memory Size:</A>
6954
</H4>
6955
6956
<P>
6957
To determine the memory buffer size, use:
6958
6959
<P>
6960
<PRE>
6961
size_t sizeof_pool_memory(void *buffer);
6962
</PRE>
6963
<P>
6964
6965
<H4><A NAME="SECTION000171350000000000000">
6966
Resizing Pool Memory:</A>
6967
</H4>
6968
6969
<P>
6970
To resize pool memory, use:
6971
6972
<P>
6973
<PRE>
6974
void *realloc_pool_memory(void *buffer);
6975
</PRE>
6976
<P>
6977
The buffer will be reallocated, and the contents of the original buffer will
6978
be preserved, but the address of the buffer may change.
6979
6980
<P>
6981
6982
<H4><A NAME="SECTION000171360000000000000">
6983
Automatic Size Adjustment:</A>
6984
</H4>
6985
6986
<P>
6987
To have the system check and if necessary adjust the size of your pooled
6988
memory buffer, use:
6989
6990
<P>
6991
<PRE>
6992
void *check_pool_memory_size(void *buffer, size_t new-size);
6993
</PRE>
6994
<P>
6995
where <B>new-size</B> is the buffer length needed. Note, if the buffer is
6996
already equal to or larger than <B>new-size</B> no buffer size change will
6997
occur. However, if a buffer size change is needed, the original contents of
6998
the buffer will be preserved, but the buffer address may change. Many of the
6999
low level Bacula subroutines expect to be passed a pool memory buffer and use
7000
this call to ensure the buffer they use is sufficiently large.
7001
7002
<P>
7003
7004
<H4><A NAME="SECTION000171370000000000000">
7005
Releasing All Pooled Memory:</A>
7006
</H4>
7007
7008
<P>
7009
In order to avoid orphaned buffer error messages when terminating the program,
7010
use:
7011
7012
<P>
7013
<PRE>
7014
void close_memory_pool();
7015
</PRE>
7016
<P>
7017
to free all unused memory retained in the Bacula memory pool. Note, any memory
7018
not returned to the pool via free_pool_memory() will not be released by this
7019
call.
7020
7021
<P>
7022
7023
<H4><A NAME="SECTION000171380000000000000">
7024
Pooled Memory Statistics:</A>
7025
</H4>
7026
7027
<P>
7028
For debugging purposes and performance tuning, the following call will print
7029
the current memory pool statistics:
7030
7031
<P>
7032
<PRE>
7033
void print_memory_pool_stats();
7034
</PRE>
7035
<P>
7036
an example output is:
7037
7038
<P>
7039
<PRE>
7040
Pool Maxsize Maxused Inuse
7041
0 256 0 0
7042
1 256 1 0
7043
2 256 1 0
7044
</PRE>
7045
<P>
7046
7047
<H1><A NAME="SECTION000180000000000000000"></A>
7048
<A NAME="_ChapterStart5"></A><A NAME="4754"></A>
7049
<A NAME="4755"></A>
7050
<BR>
7051
TCP/IP Network Protocol
7052
</H1>
7053
<A NAME="4758"></A>
7054
7055
<P>
7056
7057
<H2><A NAME="SECTION000181000000000000000"></A>
7058
<A NAME="4760"></A>
7059
<BR>
7060
General
7061
</H2>
7062
<A NAME="4763"></A>
7063
7064
<P>
7065
This document describes the TCP/IP protocol used by Bacula to communicate
7066
between the various daemons and services. The definitive definition of the
7067
protocol can be found in src/lib/bsock.h, src/lib/bnet.c and
7068
src/lib/bnet_server.c.
7069
7070
<P>
7071
Bacula's network protocol is basically a ``packet oriented'' protocol built on
7072
a standard TCP/IP streams. At the lowest level all packet transfers are done
7073
with read() and write() requests on system sockets. Pipes are not used as they
7074
are considered unreliable for large serial data transfers between various
7075
hosts.
7076
7077
<P>
7078
Using the routines described below (bnet_open, bnet_write, bnet_recv, and
7079
bnet_close) guarantees that the number of bytes you write into the socket
7080
will be received as a single record on the other end regardless of how many
7081
low level write() and read() calls are needed. All data transferred are
7082
considered to be binary data.
7083
7084
<P>
7085
7086
<H2><A NAME="SECTION000182000000000000000"></A>
7087
<A NAME="4765"></A>
7088
<A NAME="4766"></A>
7089
<BR>
7090
bnet and Threads
7091
</H2>
7092
<A NAME="4769"></A>
7093
7094
<P>
7095
These bnet routines work fine in a threaded environment. However, they assume
7096
that there is only one reader or writer on the socket at any time. It is
7097
highly recommended that only a single thread access any BSOCK packet. The
7098
exception to this rule is when the socket is first opened and it is waiting
7099
for a job to start. The wait in the Storage daemon is done in one thread and
7100
then passed to another thread for subsequent handling.
7101
7102
<P>
7103
If you envision having two threads using the same BSOCK, think twice, then you
7104
must implement some locking mechanism. However, it probably would not be
7105
appropriate to put locks inside the bnet subroutines for efficiency reasons.
7106
7107
<P>
7108
7109
<H2><A NAME="SECTION000183000000000000000"></A>
7110
<A NAME="4771"></A>
7111
<BR>
7112
bnet_open
7113
</H2>
7114
<A NAME="4774"></A>
7115
7116
<P>
7117
To establish a connection to a server, use the subroutine:
7118
7119
<P>
7120
BSOCK *bnet_open(void *jcr, char *host, char *service, int port, int *fatal)
7121
bnet_open(), if successful, returns the Bacula sock descriptor pointer to be
7122
used in subsequent bnet_send() and bnet_read() requests. If not successful,
7123
bnet_open() returns a NULL. If fatal is set on return, it means that a fatal
7124
error occurred and that you should not repeatedly call bnet_open(). Any error
7125
message will generally be sent to the JCR.
7126
7127
<P>
7128
7129
<H2><A NAME="SECTION000184000000000000000"></A>
7130
<A NAME="4776"></A>
7131
<BR>
7132
bnet_send
7133
</H2>
7134
<A NAME="4779"></A>
7135
7136
<P>
7137
To send a packet, one uses the subroutine:
7138
7139
<P>
7140
int bnet_send(BSOCK *sock) This routine is equivalent to a write() except
7141
that it handles the low level details. The data to be sent is expected to be
7142
in sock->msg and be sock->msglen bytes. To send a packet, bnet_send()
7143
first writes four bytes in network byte order than indicate the size of the
7144
following data packet. It returns:
7145
7146
<P>
7147
<PRE>
7148
Returns 0 on failure
7149
Returns 1 on success
7150
</PRE>
7151
<P>
7152
In the case of a failure, an error message will be sent to the JCR contained
7153
within the bsock packet.
7154
7155
<P>
7156
7157
<H2><A NAME="SECTION000185000000000000000"></A>
7158
<A NAME="4785"></A>
7159
<BR>
7160
bnet_fsend
7161
</H2>
7162
<A NAME="4788"></A>
7163
7164
<P>
7165
This form uses:
7166
7167
<P>
7168
int bnet_fsend(BSOCK *sock, char *format, ...) and it allows you to send a
7169
formatted messages somewhat like fprintf(). The return status is the same as
7170
bnet_send.
7171
7172
<P>
7173
7174
<H2><A NAME="SECTION000186000000000000000"></A>
7175
<A NAME="4790"></A>
7176
<A NAME="4791"></A>
7177
<BR>
7178
Additional Error information
7179
</H2>
7180
<A NAME="4794"></A>
7181
7182
<P>
7183
Fro additional error information, you can call <B>is_bnet_error(BSOCK
7184
*bsock)</B> which will return 0 if there is no error or non-zero if there is an
7185
error on the last transmission. The <B>is_bnet_stop(BSOCK *bsock)</B>
7186
function will return 0 if there no errors and you can continue sending. It
7187
will return non-zero if there are errors or the line is closed (no more
7188
transmissions should be sent).
7189
7190
<P>
7191
7192
<H2><A NAME="SECTION000187000000000000000"></A>
7193
<A NAME="4798"></A>
7194
<BR>
7195
bnet_recv
7196
</H2>
7197
<A NAME="4801"></A>
7198
7199
<P>
7200
To read a packet, one uses the subroutine:
7201
7202
<P>
7203
int bnet_recv(BSOCK *sock) This routine is similar to a read() except that it
7204
handles the low level details. bnet_read() first reads packet length that
7205
follows as four bytes in network byte order. The data is read into
7206
sock->msg and is sock->msglen bytes. If the sock->msg is not large
7207
enough, bnet_recv() realloc() the buffer. It will return an error (-2) if
7208
maxbytes is less than the record size sent. It returns:
7209
7210
<P>
7211
<PRE>
7212
* Returns number of bytes read
7213
* Returns 0 on end of file
7214
* Returns -1 on hard end of file (i.e. network connection close)
7215
* Returns -2 on error
7216
</PRE>
7217
<P>
7218
It should be noted that bnet_recv() is a blocking read.
7219
7220
<P>
7221
7222
<H2><A NAME="SECTION000188000000000000000"></A>
7223
<A NAME="4808"></A>
7224
<BR>
7225
bnet_sig
7226
</H2>
7227
<A NAME="4811"></A>
7228
7229
<P>
7230
To send a ``signal'' from one daemon to another, one uses the subroutine:
7231
7232
<P>
7233
int bnet_sig(BSOCK *sock, SIGNAL) where SIGNAL is one of the following:
7234
7235
<P>
7236
7237
<OL>
7238
<LI>BNET_EOF - deprecated use BNET_EOD
7239
</LI>
7240
<LI>BNET_EOD - End of data stream, new data may follow
7241
</LI>
7242
<LI>BNET_EOD_POLL - End of data and poll all in one
7243
</LI>
7244
<LI>BNET_STATUS - Request full status
7245
</LI>
7246
<LI>BNET_TERMINATE - Conversation terminated, doing close()
7247
</LI>
7248
<LI>BNET_POLL - Poll request, I'm hanging on a read
7249
</LI>
7250
<LI>BNET_HEARTBEAT - Heartbeat Response requested
7251
</LI>
7252
<LI>BNET_HB_RESPONSE - Only response permitted to HB
7253
</LI>
7254
<LI>BNET_PROMPT - Prompt for UA
7255
7256
</LI>
7257
</OL>
7258
7259
<P>
7260
7261
<H2><A NAME="SECTION000189000000000000000"></A>
7262
<A NAME="4815"></A>
7263
<BR>
7264
bnet_strerror
7265
</H2>
7266
<A NAME="4818"></A>
7267
7268
<P>
7269
Returns a formated string corresponding to the last error that occurred.
7270
7271
<P>
7272
7273
<H2><A NAME="SECTION0001810000000000000000"></A>
7274
<A NAME="4820"></A>
7275
<BR>
7276
bnet_close
7277
</H2>
7278
<A NAME="4823"></A>
7279
7280
<P>
7281
The connection with the server remains open until closed by the subroutine:
7282
7283
<P>
7284
void bnet_close(BSOCK *sock)
7285
7286
<P>
7287
7288
<H2><A NAME="SECTION0001811000000000000000"></A>
7289
<A NAME="4825"></A>
7290
<A NAME="4826"></A>
7291
<BR>
7292
Becoming a Server
7293
</H2>
7294
<A NAME="4829"></A>
7295
7296
<P>
7297
The bnet_open() and bnet_close() routines described above are used on the
7298
client side to establish a connection and terminate a connection with the
7299
server. To become a server (i.e. wait for a connection from a client), use the
7300
routine <B>bnet_thread_server</B>. The calling sequence is a bit complicated,
7301
please refer to the code in bnet_server.c and the code at the beginning of
7302
each daemon as examples of how to call it.
7303
7304
<P>
7305
7306
<H2><A NAME="SECTION0001812000000000000000"></A>
7307
<A NAME="4832"></A>
7308
<A NAME="4833"></A>
7309
<BR>
7310
Higher Level Conventions
7311
</H2>
7312
<A NAME="4836"></A>
7313
7314
<P>
7315
Within Bacula, we have established the convention that any time a single
7316
record is passed, it is sent with bnet_send() and read with bnet_recv().
7317
Thus the normal exchange between the server (S) and the client (C) are:
7318
7319
<P>
7320
<PRE>
7321
S: wait for connection C: attempt connection
7322
S: accept connection C: bnet_send() send request
7323
S: bnet_recv() wait for request
7324
S: act on request
7325
S: bnet_send() send ack C: bnet_recv() wait for ack
7326
</PRE>
7327
<P>
7328
Thus a single command is sent, acted upon by the server, and then
7329
acknowledged.
7330
7331
<P>
7332
In certain cases, such as the transfer of the data for a file, all the
7333
information or data cannot be sent in a single packet. In this case, the
7334
convention is that the client will send a command to the server, who knows
7335
that more than one packet will be returned. In this case, the server will
7336
enter a loop:
7337
7338
<P>
7339
<PRE>
7340
while ((n=bnet_recv(bsock)) > 0) {
7341
act on request
7342
}
7343
if (n < 0)
7344
error
7345
</PRE>
7346
<P>
7347
The client will perform the following:
7348
7349
<P>
7350
<PRE>
7351
bnet_send(bsock);
7352
bnet_send(bsock);
7353
...
7354
bnet_sig(bsock, BNET_EOD);
7355
</PRE>
7356
<P>
7357
Thus the client will send multiple packets and signal to the server when all
7358
the packets have been sent by sending a zero length record.
7359
7360
<P>
7361
<A NAME="4963"></A>
7362
<IMG
7363
WIDTH="442" HEIGHT="67" ALIGN="BOTTOM" BORDER="0"
7364
SRC="smartall.png"
7365
ALT="\includegraphics{./smartall.eps}">
7366
7367
<P>
7368
7369
<H1><A NAME="SECTION000190000000000000000"></A>
7370
<A NAME="_ChapterStart4"></A><A NAME="4967"></A>
7371
<A NAME="4968"></A>
7372
<BR>
7373
Smart Memory Allocation With Orphaned Buffer Detection
7374
</H1>
7375
<A NAME="4971"></A>
7376
7377
<P>
7378
Few things are as embarrassing as a program that leaks, yet few errors are so
7379
easy to commit or as difficult to track down in a large, complicated program
7380
as failure to release allocated memory. SMARTALLOC replaces the standard C
7381
library memory allocation functions with versions which keep track of buffer
7382
allocations and releases and report all orphaned buffers at the end of program
7383
execution. By including this package in your program during development and
7384
testing, you can identify code that loses buffers right when it's added and
7385
most easily fixed, rather than as part of a crisis debugging push when the
7386
problem is identified much later in the testing cycle (or even worse, when the
7387
code is in the hands of a customer). When program testing is complete, simply
7388
recompiling with different flags removes SMARTALLOC from your program,
7389
permitting it to run without speed or storage penalties.
7390
7391
<P>
7392
In addition to detecting orphaned buffers, SMARTALLOC also helps to find other
7393
common problems in management of dynamic storage including storing before the
7394
start or beyond the end of an allocated buffer, referencing data through a
7395
pointer to a previously released buffer, attempting to release a buffer twice
7396
or releasing storage not obtained from the allocator, and assuming the initial
7397
contents of storage allocated by functions that do not guarantee a known
7398
value. SMARTALLOC's checking does not usually add a large amount of overhead
7399
to a program (except for programs which use <TT>realloc()</TT> extensively; see
7400
below). SMARTALLOC focuses on proper storage management rather than internal
7401
consistency of the heap as checked by the malloc_debug facility available on
7402
some systems. SMARTALLOC does not conflict with malloc_debug and both may be
7403
used together, if you wish. SMARTALLOC makes no assumptions regarding the
7404
internal structure of the heap and thus should be compatible with any C
7405
language implementation of the standard memory allocation functions.
7406
7407
<P>
7408
7409
<H3><A NAME="SECTION000190100000000000000"></A>
7410
<A NAME="4974"></A>
7411
<A NAME="4975"></A>
7412
<BR>
7413
Installing SMARTALLOC
7414
</H3>
7415
<A NAME="4978"></A>
7416
7417
<P>
7418
SMARTALLOC is provided as a Zipped archive,
7419
<A NAME="tex2html14"
7420
HREF="http://www.fourmilab.ch/smartall/smartall.zip">smartall.zip</A>; see the
7421
download instructions below.
7422
7423
<P>
7424
To install SMARTALLOC in your program, simply add the statement:
7425
7426
<P>
7427
to every C program file which calls any of the memory allocation functions
7428
(<TT>malloc</TT>, <TT>calloc</TT>, <TT>free</TT>, etc.). SMARTALLOC must be used for
7429
all memory allocation with a program, so include file for your entire program,
7430
if you have such a thing. Next, define the symbol SMARTALLOC in the
7431
compilation before the inclusion of smartall.h. I usually do this by having my
7432
Makefile add the ``<TT>-DSMARTALLOC</TT>'' option to the C compiler for
7433
non-production builds. You can define the symbol manually, if you prefer, by
7434
adding the statement:
7435
7436
<P>
7437
<TT>#define SMARTALLOC</TT>
7438
7439
<P>
7440
At the point where your program is all done and ready to relinquish control to
7441
the operating system, add the call:
7442
7443
<P>
7444
<TT> sm_dump(</TT><I>datadump</I><TT>);</TT>
7445
7446
<P>
7447
where <I>datadump</I> specifies whether the contents of orphaned buffers are to
7448
be dumped in addition printing to their size and place of allocation. The data
7449
are dumped only if <I>datadump</I> is nonzero, so most programs will normally
7450
use ``<TT>sm_dump(0);</TT>''. If a mysterious orphaned buffer appears that can't
7451
be identified from the information this prints about it, replace the statement
7452
with ``<TT>sm_dump(1)</TT>;''. Usually the dump of the buffer's data will
7453
furnish the additional clues you need to excavate and extirpate the elusive
7454
error that left the buffer allocated.
7455
7456
<P>
7457
Finally, add the files ``smartall.h'' and ``smartall.c'' from this release to
7458
your source directory, make dependencies, and linker input. You needn't make
7459
inclusion of smartall.c in your link optional; if compiled with SMARTALLOC not
7460
defined it generates no code, so you may always include it knowing it will
7461
waste no storage in production builds. Now when you run your program, if it
7462
leaves any buffers around when it's done, each will be reported by <TT>sm_dump()</TT> on stderr as follows:
7463
7464
<P>
7465
<PRE>
7466
Orphaned buffer: 120 bytes allocated at line 50 of gutshot.c
7467
</PRE>
7468
<P>
7469
7470
<H3><A NAME="SECTION000190200000000000000"></A>
7471
<A NAME="4997"></A>
7472
<A NAME="4998"></A>
7473
<BR>
7474
Squelching a SMARTALLOC
7475
</H3>
7476
<A NAME="5001"></A>
7477
7478
<P>
7479
Usually, when you first install SMARTALLOC in an existing program you'll find
7480
it nattering about lots of orphaned buffers. Some of these turn out to be
7481
legitimate errors, but some are storage allocated during program
7482
initialisation that, while dynamically allocated, is logically static storage
7483
not intended to be released. Of course, you can get rid of the complaints
7484
about these buffers by adding code to release them, but by doing so you're
7485
adding unnecessary complexity and code size to your program just to silence
7486
the nattering of a SMARTALLOC, so an escape hatch is provided to eliminate the
7487
need to release these buffers.
7488
7489
<P>
7490
Normally all storage allocated with the functions <TT>malloc()</TT>, <TT>calloc()</TT>, and <TT>realloc()</TT> is monitored by SMARTALLOC. If you make the
7491
function call:
7492
7493
<P>
7494
<PRE>
7495
sm_static(1);
7496
</PRE>
7497
<P>
7498
you declare that subsequent storage allocated by <TT>malloc()</TT>, <TT>calloc()</TT>, and <TT>realloc()</TT> should not be considered orphaned if found to
7499
be allocated when <TT>sm_dump()</TT> is called. I use a call on ``<TT>sm_static(1);</TT>'' before I allocate things like program configuration tables
7500
so I don't have to add code to release them at end of program time. After
7501
allocating unmonitored data this way, be sure to add a call to:
7502
7503
<P>
7504
<PRE>
7505
sm_static(0);
7506
</PRE>
7507
<P>
7508
to resume normal monitoring of buffer allocations. Buffers allocated while
7509
<TT>sm_static(1</TT>) is in effect are not checked for having been orphaned but
7510
all the other safeguards provided by SMARTALLOC remain in effect. You may
7511
release such buffers, if you like; but you don't have to.
7512
7513
<P>
7514
7515
<H3><A NAME="SECTION000190300000000000000"></A>
7516
<A NAME="5016"></A>
7517
<A NAME="5017"></A>
7518
<BR>
7519
Living with Libraries
7520
</H3>
7521
<A NAME="5020"></A>
7522
7523
<P>
7524
Some library functions for which source code is unavailable may gratuitously
7525
allocate and return buffers that contain their results, or require you to pass
7526
them buffers which they subsequently release. If you have source code for the
7527
library, by far the best approach is to simply install SMARTALLOC in it,
7528
particularly since this kind of ill-structured dynamic storage management is
7529
the source of so many storage leaks. Without source code, however, there's no
7530
option but to provide a way to bypass SMARTALLOC for the buffers the library
7531
allocates and/or releases with the standard system functions.
7532
7533
<P>
7534
For each function <I>xxx</I> redefined by SMARTALLOC, a corresponding routine
7535
named ``<TT>actually</TT><I>xxx</I>'' is furnished which provides direct access to
7536
the underlying system function, as follows:
7537
7538
<P>
7539
<P>
7540
<BLOCKQUOTE><TABLE CELLPADDING=3>
7541
<TR><TD ALIGN="LEFT" COLSPAN=1><B>Standard function </B></TD>
7542
<TD ALIGN="LEFT" COLSPAN=1><B>Direct
7543
access function </B></TD>
7544
</TR>
7545
<TR><TD ALIGN="LEFT"><TT>malloc(</TT><I>size</I><TT>)</TT></TD>
7546
<TD ALIGN="LEFT"><TT>actuallymalloc(</TT><I>size</I><TT>)</TT></TD>
7547
</TR>
7548
<TR><TD ALIGN="LEFT"><TT>calloc(</TT><I>nelem</I><TT>,</TT> <I>elsize</I><TT>)</TT></TD>
7549
<TD ALIGN="LEFT"><TT>actuallycalloc(</TT><I>nelem</I>, <I>elsize</I><TT>)</TT></TD>
7550
</TR>
7551
<TR><TD ALIGN="LEFT"><TT>realloc(</TT><I>ptr</I><TT>,</TT> <I>size</I><TT>)</TT></TD>
7552
<TD ALIGN="LEFT"><TT>actuallyrealloc(</TT><I>ptr</I>, <I>size</I><TT>)</TT></TD>
7553
</TR>
7554
<TR><TD ALIGN="LEFT"><TT>free(</TT><I>ptr</I><TT>)</TT></TD>
7555
<TD ALIGN="LEFT"><TT>actuallyfree(</TT><I>ptr</I><TT>)</TT>
7556
7557
<P></TD>
7558
</TR>
7559
</TABLE>
7560
</BLOCKQUOTE>
7561
<P>
7562
7563
<P>
7564
For example, suppose there exists a system library function named ``<TT>getimage()</TT>'' which reads a raster image file and returns the address of a
7565
buffer containing it. Since the library routine allocates the image directly
7566
with <TT>malloc()</TT>, you can't use SMARTALLOC's <TT>free()</TT>, as that call
7567
expects information placed in the buffer by SMARTALLOC's special version of
7568
<TT>malloc()</TT>, and hence would report an error. To release the buffer you
7569
should call <TT>actuallyfree()</TT>, as in this code fragment:
7570
7571
<P>
7572
<PRE>
7573
struct image *ibuf = getimage("ratpack.img");
7574
display_on_screen(ibuf);
7575
actuallyfree(ibuf);
7576
</PRE>
7577
<P>
7578
Conversely, suppose we are to call a library function, ``<TT>putimage()</TT>'',
7579
which writes an image buffer into a file and then releases the buffer with
7580
<TT>free()</TT>. Since the system <TT>free()</TT> is being called, we can't pass a
7581
buffer allocated by SMARTALLOC's allocation routines, as it contains special
7582
information that the system <TT>free()</TT> doesn't expect to be there. The
7583
following code uses <TT>actuallymalloc()</TT> to obtain the buffer passed to such
7584
a routine.
7585
7586
<P>
7587
<PRE>
7588
struct image *obuf =
7589
(struct image *) actuallymalloc(sizeof(struct image));
7590
dump_screen_to_image(obuf);
7591
putimage("scrdump.img", obuf); /* putimage() releases obuf */
7592
</PRE>
7593
<P>
7594
It's unlikely you'll need any of the ``actually'' calls except under very odd
7595
circumstances (in four products and three years, I've only needed them once),
7596
but they're there for the rare occasions that demand them. Don't use them to
7597
subvert the error checking of SMARTALLOC; if you want to disable orphaned
7598
buffer detection, use the <TT>sm_static(1)</TT> mechanism described above. That
7599
way you don't forfeit all the other advantages of SMARTALLOC as you do when
7600
using <TT>actuallymalloc()</TT> and <TT>actuallyfree()</TT>.
7601
7602
<P>
7603
7604
<H3><A NAME="SECTION000190400000000000000"></A>
7605
<A NAME="5083"></A>
7606
<A NAME="5084"></A>
7607
<BR>
7608
SMARTALLOC Details
7609
</H3>
7610
<A NAME="5087"></A>
7611
7612
<P>
7613
When you include ``smartall.h'' and define SMARTALLOC, the following standard
7614
system library functions are redefined with the #define mechanism to call
7615
corresponding functions within smartall.c instead. (For details of the
7616
redefinitions, please refer to smartall.h.)
7617
7618
<P>
7619
<PRE>
7620
void *malloc(size_t size)
7621
void *calloc(size_t nelem, size_t elsize)
7622
void *realloc(void *ptr, size_t size)
7623
void free(void *ptr)
7624
void cfree(void *ptr)
7625
</PRE>
7626
<P>
7627
<TT>cfree()</TT> is a historical artifact identical to <TT>free()</TT>.
7628
7629
<P>
7630
In addition to allocating storage in the same way as the standard library
7631
functions, the SMARTALLOC versions expand the buffers they allocate to include
7632
information that identifies where each buffer was allocated and to chain all
7633
allocated buffers together. When a buffer is released, it is removed from the
7634
allocated buffer chain. A call on <TT>sm_dump()</TT> is able, by scanning the
7635
chain of allocated buffers, to find all orphaned buffers. Buffers allocated
7636
while <TT>sm_static(1)</TT> is in effect are specially flagged so that, despite
7637
appearing on the allocated buffer chain, <TT>sm_dump()</TT> will not deem them
7638
orphans.
7639
7640
<P>
7641
When a buffer is allocated by <TT>malloc()</TT> or expanded with <TT>realloc()</TT>,
7642
all bytes of newly allocated storage are set to the hexadecimal value 0x55
7643
(alternating one and zero bits). Note that for <TT>realloc()</TT> this applies
7644
only to the bytes added at the end of buffer; the original contents of the
7645
buffer are not modified. Initializing allocated storage to a distinctive
7646
nonzero pattern is intended to catch code that erroneously assumes newly
7647
allocated buffers are cleared to zero; in fact their contents are random. The
7648
<TT>calloc()</TT> function, defined as returning a buffer cleared to zero,
7649
continues to zero its buffers under SMARTALLOC.
7650
7651
<P>
7652
Buffers obtained with the SMARTALLOC functions contain a special sentinel byte
7653
at the end of the user data area. This byte is set to a special key value
7654
based upon the buffer's memory address. When the buffer is released, the key
7655
is tested and if it has been overwritten an assertion in the <TT>free</TT>
7656
function will fail. This catches incorrect program code that stores beyond the
7657
storage allocated for the buffer. At <TT>free()</TT> time the queue links are
7658
also validated and an assertion failure will occur if the program has
7659
destroyed them by storing before the start of the allocated storage.
7660
7661
<P>
7662
In addition, when a buffer is released with <TT>free()</TT>, its contents are
7663
immediately destroyed by overwriting them with the hexadecimal pattern 0xAA
7664
(alternating bits, the one's complement of the initial value pattern). This
7665
will usually trip up code that keeps a pointer to a buffer that's been freed
7666
and later attempts to reference data within the released buffer. Incredibly,
7667
this is <I>legal</I> in the standard Unix memory allocation package, which
7668
permits programs to free() buffers, then raise them from the grave with <TT>realloc()</TT>. Such program ``logic'' should be fixed, not accommodated, and
7669
SMARTALLOC brooks no such Lazarus buffer`` nonsense.
7670
7671
<P>
7672
Some C libraries allow a zero size argument in calls to <TT>malloc()</TT>. Since
7673
this is far more likely to indicate a program error than a defensible
7674
programming stratagem, SMARTALLOC disallows it with an assertion.
7675
7676
<P>
7677
When the standard library <TT>realloc()</TT> function is called to expand a
7678
buffer, it attempts to expand the buffer in place if possible, moving it only
7679
if necessary. Because SMARTALLOC must place its own private storage in the
7680
buffer and also to aid in error detection, its version of <TT>realloc()</TT>
7681
always moves and copies the buffer except in the trivial case where the size
7682
of the buffer is not being changed. By forcing the buffer to move on every
7683
call and destroying the contents of the old buffer when it is released,
7684
SMARTALLOC traps programs which keep pointers into a buffer across a call on
7685
<TT>realloc()</TT> which may move it. This strategy may prove very costly to
7686
programs which make extensive use of <TT>realloc()</TT>. If this proves to be a
7687
problem, such programs may wish to use <TT>actuallymalloc()</TT>, <TT>actuallyrealloc()</TT>, and <TT>actuallyfree()</TT> for such frequently-adjusted
7688
buffers, trading error detection for performance. Although not specified in
7689
the System V Interface Definition, many C library implementations of <TT>realloc()</TT> permit an old buffer argument of NULL, causing <TT>realloc()</TT> to
7690
allocate a new buffer. The SMARTALLOC version permits this.
7691
7692
<P>
7693
7694
<H3><A NAME="SECTION000190500000000000000"></A>
7695
<A NAME="5115"></A>
7696
<A NAME="5116"></A>
7697
<BR>
7698
When SMARTALLOC is Disabled
7699
</H3>
7700
<A NAME="5119"></A>
7701
7702
<P>
7703
When SMARTALLOC is disabled by compiling a program with the symbol SMARTALLOC
7704
not defined, calls on the functions otherwise redefined by SMARTALLOC go
7705
directly to the system functions. In addition, compile-time definitions
7706
translate calls on the ''<TT>actually</TT>...<TT>()</TT>`` functions into the
7707
corresponding library calls; ''<TT>actuallymalloc(100)</TT>``, for example,
7708
compiles into ''<TT>malloc(100)</TT>``. The two special SMARTALLOC functions,
7709
<TT>sm_dump()</TT> and <TT>sm_static()</TT>, are defined to generate no code
7710
(hence the null statement). Finally, if SMARTALLOC is not defined, compilation
7711
of the file smartall.c generates no code or data at all, effectively removing
7712
it from the program even if named in the link instructions.
7713
7714
<P>
7715
Thus, except for unusual circumstances, a program that works with SMARTALLOC
7716
defined for testing should require no changes when built without it for
7717
production release.
7718
7719
<P>
7720
7721
<H3><A NAME="SECTION000190600000000000000"></A>
7722
<A NAME="5127"></A>
7723
<A NAME="5128"></A>
7724
<BR>
7725
The <TT>alloc()</TT> Function
7726
</H3>
7727
<A NAME="5131"></A>
7728
7729
<P>
7730
Many programs I've worked on use very few direct calls to <TT>malloc()</TT>,
7731
using the identically declared <TT>alloc()</TT> function instead. Alloc detects
7732
out-of-memory conditions and aborts, removing the need for error checking on
7733
every call of <TT>malloc()</TT> (and the temptation to skip checking for
7734
out-of-memory).
7735
7736
<P>
7737
As a convenience, SMARTALLOC supplies a compatible version of <TT>alloc()</TT> in
7738
the file alloc.c, with its definition in the file alloc.h. This version of
7739
<TT>alloc()</TT> is sensitive to the definition of SMARTALLOC and cooperates with
7740
SMARTALLOC's orphaned buffer detection. In addition, when SMARTALLOC is
7741
defined and <TT>alloc()</TT> detects an out of memory condition, it takes
7742
advantage of the SMARTALLOC diagnostic information to identify the file and
7743
line number of the call on <TT>alloc()</TT> that failed.
7744
7745
<P>
7746
7747
<H3><A NAME="SECTION000190700000000000000"></A>
7748
<A NAME="5140"></A>
7749
<A NAME="5141"></A>
7750
<BR>
7751
Overlays and Underhandedness
7752
</H3>
7753
<A NAME="5144"></A>
7754
7755
<P>
7756
String constants in the C language are considered to be static arrays of
7757
characters accessed through a pointer constant. The arrays are potentially
7758
writable even though their pointer is a constant. SMARTALLOC uses the
7759
compile-time definition <TT>./smartall.wml</TT> to obtain the name of the file in
7760
which a call on buffer allocation was performed. Rather than reserve space in
7761
a buffer to save this information, SMARTALLOC simply stores the pointer to the
7762
compiled-in text of the file name. This works fine as long as the program does
7763
not overlay its data among modules. If data are overlayed, the area of memory
7764
which contained the file name at the time it was saved in the buffer may
7765
contain something else entirely when <TT>sm_dump()</TT> gets around to using the
7766
pointer to edit the file name which allocated the buffer.
7767
7768
<P>
7769
If you want to use SMARTALLOC in a program with overlayed data, you'll have to
7770
modify smartall.c to either copy the file name to a fixed-length field added
7771
to the <TT>abufhead</TT> structure, or else allocate storage with <TT>malloc()</TT>,
7772
copy the file name there, and set the <TT>abfname</TT> pointer to that buffer,
7773
then remember to release the buffer in <TT>sm_free</TT>. Either of these
7774
approaches are wasteful of storage and time, and should be considered only if
7775
there is no alternative. Since most initial debugging is done in non-overlayed
7776
environments, the restrictions on SMARTALLOC with data overlaying may never
7777
prove a problem. Note that conventional overlaying of code, by far the most
7778
common form of overlaying, poses no problems for SMARTALLOC; you need only be
7779
concerned if you're using exotic tools for data overlaying on MS-DOS or other
7780
address-space-challenged systems.
7781
7782
<P>
7783
Since a C language ''constant`` string can actually be written into, most C
7784
compilers generate a unique copy of each string used in a module, even if the
7785
same constant string appears many times. In modules that contain many calls on
7786
allocation functions, this results in substantial wasted storage for the
7787
strings that identify the file name. If your compiler permits optimization of
7788
multiple occurrences of constant strings, enabling this mode will eliminate
7789
the overhead for these strings. Of course, it's up to you to make sure
7790
choosing this compiler mode won't wreak havoc on some other part of your
7791
program.
7792
7793
<P>
7794
7795
<H3><A NAME="SECTION000190800000000000000"></A>
7796
<A NAME="5152"></A>
7797
<A NAME="5153"></A>
7798
<BR>
7799
Test and Demonstration Program
7800
</H3>
7801
<A NAME="5156"></A>
7802
7803
<P>
7804
A test and demonstration program, smtest.c, is supplied with SMARTALLOC. You
7805
can build this program with the Makefile included. Please refer to the
7806
comments in smtest.c and the Makefile for information on this program. If
7807
you're attempting to use SMARTALLOC on a new machine or with a new compiler or
7808
operating system, it's a wise first step to check it out with smtest first.
7809
7810
<P>
7811
7812
<H3><A NAME="SECTION000190900000000000000"></A>
7813
<A NAME="5158"></A>
7814
<A NAME="5159"></A>
7815
<BR>
7816
Invitation to the Hack
7817
</H3>
7818
<A NAME="5162"></A>
7819
7820
<P>
7821
SMARTALLOC is not intended to be a panacea for storage management problems,
7822
nor is it universally applicable or effective; it's another weapon in the
7823
arsenal of the defensive professional programmer attempting to create reliable
7824
products. It represents the current state of evolution of expedient debug code
7825
which has been used in several commercial software products which have,
7826
collectively, sold more than third of a million copies in the retail market,
7827
and can be expected to continue to develop through time as it is applied to
7828
ever more demanding projects.
7829
7830
<P>
7831
The version of SMARTALLOC here has been tested on a Sun SPARCStation, Silicon
7832
Graphics Indigo2, and on MS-DOS using both Borland and Microsoft C. Moving
7833
from compiler to compiler requires the usual small changes to resolve disputes
7834
about prototyping of functions, whether the type returned by buffer allocation
7835
is <TT>char *</TT> or <TT>void *</TT>, and so forth, but following those changes
7836
it works in a variety of environments. I hope you'll find SMARTALLOC as useful
7837
for your projects as I've found it in mine.
7838
7839
<P>
7840
7841
<H2><A NAME="SECTION000191000000000000000"></A>
7842
<A NAME="5169"></A>
7843
<A NAME="5170"></A>
7844
<BR>
7845
7846
7847
<A NAME="tex2html16"
7848
HREF="http://www.fourmilab.ch/smartall/smartall.zip">Download smartall.zip</A>
7849
(Zipped archive)
7850
</H2>
7851
<A NAME="5173"></A>
7852
7853
<P>
7854
SMARTALLOC is provided as
7855
<A NAME="tex2html17"
7856
HREF="http://www.fourmilab.ch/smartall/smartall.zip">smartall.zip</A>, a
7857
<A NAME="tex2html18"
7858
HREF="http://www.pkware.com/">Zipped</A>
7859
archive containing source code,
7860
documentation, and a <TT>Makefile</TT> to build the software under Unix.
7861
7862
<P>
7863
7864
<H3><A NAME="SECTION000191100000000000000"></A>
7865
<A NAME="5180"></A>
7866
<BR>
7867
Copying
7868
</H3>
7869
<A NAME="5183"></A>
7870
7871
<P>
7872
<BLOCKQUOTE>
7873
SMARTALLOC is in the public domain. Permission to use, copy, modify, and
7874
distribute this software and its documentation for any purpose and without fee
7875
is hereby granted, without any conditions or restrictions. This software is
7876
provided ''as is`` without express or implied warranty.
7877
7878
</BLOCKQUOTE>
7879
7880
<P>
7881
<I><A NAME="tex2html19"
7882
HREF="http://www.fourmilab.ch">by John Walker</A>
7883
October 30th, 1998 </I>
7884
7885
<P>
7886
7887
<P>
7888
7889
<P>
7890
colorlinks,
7891
citecolor=black,
7892
filecolor=black,
7893
linkcolor=black,
7894
urlcolor=black,
7895
pdftex
7896
7897
<P>
7898
7899
<H1><A NAME="SECTION000200000000000000000">
7900
GNU Free Documentation License</A>
7901
</H1>
7902
<A NAME="5593"></A>
7903
<A NAME="5594"></A>
7904
<A NAME="5597"></A>
7905
7906
<P>
7907
<DIV ALIGN="CENTER">
7908
</DIV>
7909
<P>
7910
<DIV ALIGN="CENTER">Version 1.2, November 2002
7911
</DIV>
7912
<P>
7913
<DIV ALIGN="CENTER">Copyright ©2000,2001,2002 Free Software Foundation, Inc.
7914
</DIV>
7915
<P>
7916
<DIV ALIGN="CENTER"></DIV>
7917
<P><P>
7918
<BR>
7919
<DIV ALIGN="CENTER"></DIV>
7920
<P>
7921
<DIV ALIGN="CENTER">51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
7922
</DIV>
7923
<P>
7924
<DIV ALIGN="CENTER"></DIV>
7925
<P><P>
7926
<BR>
7927
<DIV ALIGN="CENTER"></DIV>
7928
<P>
7929
<DIV ALIGN="CENTER">Everyone is permitted to copy and distribute verbatim copies
7930
of this license document, but changing it is not allowed.
7931
7932
</DIV>
7933
7934
<P>
7935
<DIV ALIGN="CENTER">
7936
<B><FONT SIZE="+1">Preamble</FONT></B>
7937
7938
</DIV>
7939
7940
<P>
7941
The purpose of this License is to make a manual, textbook, or other
7942
functional and useful document "free" in the sense of freedom: to
7943
assure everyone the effective freedom to copy and redistribute it,
7944
with or without modifying it, either commercially or noncommercially.
7945
Secondarily, this License preserves for the author and publisher a way
7946
to get credit for their work, while not being considered responsible
7947
for modifications made by others.
7948
7949
<P>
7950
This License is a kind of "copyleft", which means that derivative
7951
works of the document must themselves be free in the same sense. It
7952
complements the GNU General Public License, which is a copyleft
7953
license designed for free software.
7954
7955
<P>
7956
We have designed this License in order to use it for manuals for free
7957
software, because free software needs free documentation: a free
7958
program should come with manuals providing the same freedoms that the
7959
software does. But this License is not limited to software manuals;
7960
it can be used for any textual work, regardless of subject matter or
7961
whether it is published as a printed book. We recommend this License
7962
principally for works whose purpose is instruction or reference.
7963
7964
<P>
7965
<DIV ALIGN="CENTER">
7966
<FONT SIZE="+2"><B>1. APPLICABILITY AND DEFINITIONS</B></FONT>
7967
<A NAME="5607"></A>
7968
7969
</DIV>
7970
7971
<P>
7972
This License applies to any manual or other work, in any medium, that
7973
contains a notice placed by the copyright holder saying it can be
7974
distributed under the terms of this License. Such a notice grants a
7975
world-wide, royalty-free license, unlimited in duration, to use that
7976
work under the conditions stated herein. The <B>"Document"</B>, below,
7977
refers to any such manual or work. Any member of the public is a
7978
licensee, and is addressed as <B>"you"</B>. You accept the license if you
7979
copy, modify or distribute the work in a way requiring permission
7980
under copyright law.
7981
7982
<P>
7983
A <B>"Modified Version"</B> of the Document means any work containing the
7984
Document or a portion of it, either copied verbatim, or with
7985
modifications and/or translated into another language.
7986
7987
<P>
7988
A <B>"Secondary Section"</B> is a named appendix or a front-matter section of
7989
the Document that deals exclusively with the relationship of the
7990
publishers or authors of the Document to the Document's overall subject
7991
(or to related matters) and contains nothing that could fall directly
7992
within that overall subject. (Thus, if the Document is in part a
7993
textbook of mathematics, a Secondary Section may not explain any
7994
mathematics.) The relationship could be a matter of historical
7995
connection with the subject or with related matters, or of legal,
7996
commercial, philosophical, ethical or political position regarding
7997
them.
7998
7999
<P>
8000
The <B>"Invariant Sections"</B> are certain Secondary Sections whose titles
8001
are designated, as being those of Invariant Sections, in the notice
8002
that says that the Document is released under this License. If a
8003
section does not fit the above definition of Secondary then it is not
8004
allowed to be designated as Invariant. The Document may contain zero
8005
Invariant Sections. If the Document does not identify any Invariant
8006
Sections then there are none.
8007
8008
<P>
8009
The <B>"Cover Texts"</B> are certain short passages of text that are listed,
8010
as Front-Cover Texts or Back-Cover Texts, in the notice that says that
8011
the Document is released under this License. A Front-Cover Text may
8012
be at most 5 words, and a Back-Cover Text may be at most 25 words.
8013
8014
<P>
8015
A <B>"Transparent"</B> copy of the Document means a machine-readable copy,
8016
represented in a format whose specification is available to the
8017
general public, that is suitable for revising the document
8018
straightforwardly with generic text editors or (for images composed of
8019
pixels) generic paint programs or (for drawings) some widely available
8020
drawing editor, and that is suitable for input to text formatters or
8021
for automatic translation to a variety of formats suitable for input
8022
to text formatters. A copy made in an otherwise Transparent file
8023
format whose markup, or absence of markup, has been arranged to thwart
8024
or discourage subsequent modification by readers is not Transparent.
8025
An image format is not Transparent if used for any substantial amount
8026
of text. A copy that is not "Transparent" is called <B>"Opaque"</B>.
8027
8028
<P>
8029
Examples of suitable formats for Transparent copies include plain
8030
ASCII without markup, Texinfo input format, LaTeX input format, SGML
8031
or XML using a publicly available DTD, and standard-conforming simple
8032
HTML, PostScript or PDF designed for human modification. Examples of
8033
transparent image formats include PNG, XCF and JPG. Opaque formats
8034
include proprietary formats that can be read and edited only by
8035
proprietary word processors, SGML or XML for which the DTD and/or
8036
processing tools are not generally available, and the
8037
machine-generated HTML, PostScript or PDF produced by some word
8038
processors for output purposes only.
8039
8040
<P>
8041
The <B>"Title Page"</B> means, for a printed book, the title page itself,
8042
plus such following pages as are needed to hold, legibly, the material
8043
this License requires to appear in the title page. For works in
8044
formats which do not have any title page as such, "Title Page" means
8045
the text near the most prominent appearance of the work's title,
8046
preceding the beginning of the body of the text.
8047
8048
<P>
8049
A section <B>"Entitled XYZ"</B> means a named subunit of the Document whose
8050
title either is precisely XYZ or contains XYZ in parentheses following
8051
text that translates XYZ in another language. (Here XYZ stands for a
8052
specific section name mentioned below, such as <B>"Acknowledgements"</B>,
8053
<B>"Dedications"</B>, <B>"Endorsements"</B>, or <B>"History"</B>.)
8054
To <B>"Preserve the Title"</B>
8055
of such a section when you modify the Document means that it remains a
8056
section "Entitled XYZ" according to this definition.
8057
8058
<P>
8059
The Document may include Warranty Disclaimers next to the notice which
8060
states that this License applies to the Document. These Warranty
8061
Disclaimers are considered to be included by reference in this
8062
License, but only as regards disclaiming warranties: any other
8063
implication that these Warranty Disclaimers may have is void and has
8064
no effect on the meaning of this License.
8065
8066
<P>
8067
<DIV ALIGN="CENTER">
8068
<FONT SIZE="+2"><B>2. VERBATIM COPYING</B></FONT>
8069
<A NAME="5628"></A>
8070
8071
</DIV>
8072
8073
<P>
8074
You may copy and distribute the Document in any medium, either
8075
commercially or noncommercially, provided that this License, the
8076
copyright notices, and the license notice saying this License applies
8077
to the Document are reproduced in all copies, and that you add no other
8078
conditions whatsoever to those of this License. You may not use
8079
technical measures to obstruct or control the reading or further
8080
copying of the copies you make or distribute. However, you may accept
8081
compensation in exchange for copies. If you distribute a large enough
8082
number of copies you must also follow the conditions in section 3.
8083
8084
<P>
8085
You may also lend copies, under the same conditions stated above, and
8086
you may publicly display copies.
8087
8088
<P>
8089
<DIV ALIGN="CENTER">
8090
<FONT SIZE="+2"><B>3. COPYING IN QUANTITY</B></FONT>
8091
<A NAME="5634"></A>
8092
8093
</DIV>
8094
8095
<P>
8096
If you publish printed copies (or copies in media that commonly have
8097
printed covers) of the Document, numbering more than 100, and the
8098
Document's license notice requires Cover Texts, you must enclose the
8099
copies in covers that carry, clearly and legibly, all these Cover
8100
Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on
8101
the back cover. Both covers must also clearly and legibly identify
8102
you as the publisher of these copies. The front cover must present
8103
the full title with all words of the title equally prominent and
8104
visible. You may add other material on the covers in addition.
8105
Copying with changes limited to the covers, as long as they preserve
8106
the title of the Document and satisfy these conditions, can be treated
8107
as verbatim copying in other respects.
8108
8109
<P>
8110
If the required texts for either cover are too voluminous to fit
8111
legibly, you should put the first ones listed (as many as fit
8112
reasonably) on the actual cover, and continue the rest onto adjacent
8113
pages.
8114
8115
<P>
8116
If you publish or distribute Opaque copies of the Document numbering
8117
more than 100, you must either include a machine-readable Transparent
8118
copy along with each Opaque copy, or state in or with each Opaque copy
8119
a computer-network location from which the general network-using
8120
public has access to download using public-standard network protocols
8121
a complete Transparent copy of the Document, free of added material.
8122
If you use the latter option, you must take reasonably prudent steps,
8123
when you begin distribution of Opaque copies in quantity, to ensure
8124
that this Transparent copy will remain thus accessible at the stated
8125
location until at least one year after the last time you distribute an
8126
Opaque copy (directly or through your agents or retailers) of that
8127
edition to the public.
8128
8129
<P>
8130
It is requested, but not required, that you contact the authors of the
8131
Document well before redistributing any large number of copies, to give
8132
them a chance to provide you with an updated version of the Document.
8133
8134
<P>
8135
<DIV ALIGN="CENTER">
8136
<FONT SIZE="+2"><B>4. MODIFICATIONS</B></FONT>
8137
<A NAME="5640"></A>
8138
8139
</DIV>
8140
8141
<P>
8142
You may copy and distribute a Modified Version of the Document under
8143
the conditions of sections 2 and 3 above, provided that you release
8144
the Modified Version under precisely this License, with the Modified
8145
Version filling the role of the Document, thus licensing distribution
8146
and modification of the Modified Version to whoever possesses a copy
8147
of it. In addition, you must do these things in the Modified Version:
8148
8149
<P>
8150
<DL COMPACT>
8151
<DT>A.</DT>
8152
<DD>Use in the Title Page (and on the covers, if any) a title distinct
8153
from that of the Document, and from those of previous versions
8154
(which should, if there were any, be listed in the History section
8155
of the Document). You may use the same title as a previous version
8156
if the original publisher of that version gives permission.
8157
8158
<P>
8159
</DD>
8160
<DT>B.</DT>
8161
<DD>List on the Title Page, as authors, one or more persons or entities
8162
responsible for authorship of the modifications in the Modified
8163
Version, together with at least five of the principal authors of the
8164
Document (all of its principal authors, if it has fewer than five),
8165
unless they release you from this requirement.
8166
8167
<P>
8168
</DD>
8169
<DT>C.</DT>
8170
<DD>State on the Title page the name of the publisher of the
8171
Modified Version, as the publisher.
8172
8173
<P>
8174
</DD>
8175
<DT>D.</DT>
8176
<DD>Preserve all the copyright notices of the Document.
8177
8178
<P>
8179
</DD>
8180
<DT>E.</DT>
8181
<DD>Add an appropriate copyright notice for your modifications
8182
adjacent to the other copyright notices.
8183
8184
<P>
8185
</DD>
8186
<DT>F.</DT>
8187
<DD>Include, immediately after the copyright notices, a license notice
8188
giving the public permission to use the Modified Version under the
8189
terms of this License, in the form shown in the Addendum below.
8190
8191
<P>
8192
</DD>
8193
<DT>G.</DT>
8194
<DD>Preserve in that license notice the full lists of Invariant Sections
8195
and required Cover Texts given in the Document's license notice.
8196
8197
<P>
8198
</DD>
8199
<DT>H.</DT>
8200
<DD>Include an unaltered copy of this License.
8201
8202
<P>
8203
</DD>
8204
<DT>I.</DT>
8205
<DD>Preserve the section Entitled "History", Preserve its Title, and add
8206
to it an item stating at least the title, year, new authors, and
8207
publisher of the Modified Version as given on the Title Page. If
8208
there is no section Entitled "History" in the Document, create one
8209
stating the title, year, authors, and publisher of the Document as
8210
given on its Title Page, then add an item describing the Modified
8211
Version as stated in the previous sentence.
8212
8213
<P>
8214
</DD>
8215
<DT>J.</DT>
8216
<DD>Preserve the network location, if any, given in the Document for
8217
public access to a Transparent copy of the Document, and likewise
8218
the network locations given in the Document for previous versions
8219
it was based on. These may be placed in the "History" section.
8220
You may omit a network location for a work that was published at
8221
least four years before the Document itself, or if the original
8222
publisher of the version it refers to gives permission.
8223
8224
<P>
8225
</DD>
8226
<DT>K.</DT>
8227
<DD>For any section Entitled "Acknowledgements" or "Dedications",
8228
Preserve the Title of the section, and preserve in the section all
8229
the substance and tone of each of the contributor acknowledgements
8230
and/or dedications given therein.
8231
8232
<P>
8233
</DD>
8234
<DT>L.</DT>
8235
<DD>Preserve all the Invariant Sections of the Document,
8236
unaltered in their text and in their titles. Section numbers
8237
or the equivalent are not considered part of the section titles.
8238
8239
<P>
8240
</DD>
8241
<DT>M.</DT>
8242
<DD>Delete any section Entitled "Endorsements". Such a section
8243
may not be included in the Modified Version.
8244
8245
<P>
8246
</DD>
8247
<DT>N.</DT>
8248
<DD>Do not retitle any existing section to be Entitled "Endorsements"
8249
or to conflict in title with any Invariant Section.
8250
8251
<P>
8252
</DD>
8253
<DT>O.</DT>
8254
<DD>Preserve any Warranty Disclaimers.
8255
</DD>
8256
</DL>
8257
8258
<P>
8259
If the Modified Version includes new front-matter sections or
8260
appendices that qualify as Secondary Sections and contain no material
8261
copied from the Document, you may at your option designate some or all
8262
of these sections as invariant. To do this, add their titles to the
8263
list of Invariant Sections in the Modified Version's license notice.
8264
These titles must be distinct from any other section titles.
8265
8266
<P>
8267
You may add a section Entitled "Endorsements", provided it contains
8268
nothing but endorsements of your Modified Version by various
8269
parties--for example, statements of peer review or that the text has
8270
been approved by an organization as the authoritative definition of a
8271
standard.
8272
8273
<P>
8274
You may add a passage of up to five words as a Front-Cover Text, and a
8275
passage of up to 25 words as a Back-Cover Text, to the end of the list
8276
of Cover Texts in the Modified Version. Only one passage of
8277
Front-Cover Text and one of Back-Cover Text may be added by (or
8278
through arrangements made by) any one entity. If the Document already
8279
includes a cover text for the same cover, previously added by you or
8280
by arrangement made by the same entity you are acting on behalf of,
8281
you may not add another; but you may replace the old one, on explicit
8282
permission from the previous publisher that added the old one.
8283
8284
<P>
8285
The author(s) and publisher(s) of the Document do not by this License
8286
give permission to use their names for publicity for or to assert or
8287
imply endorsement of any Modified Version.
8288
8289
<P>
8290
<DIV ALIGN="CENTER">
8291
<FONT SIZE="+2"><B>5. COMBINING DOCUMENTS</B></FONT>
8292
<A NAME="5648"></A>
8293
8294
</DIV>
8295
8296
<P>
8297
You may combine the Document with other documents released under this
8298
License, under the terms defined in section 4 above for modified
8299
versions, provided that you include in the combination all of the
8300
Invariant Sections of all of the original documents, unmodified, and
8301
list them all as Invariant Sections of your combined work in its
8302
license notice, and that you preserve all their Warranty Disclaimers.
8303
8304
<P>
8305
The combined work need only contain one copy of this License, and
8306
multiple identical Invariant Sections may be replaced with a single
8307
copy. If there are multiple Invariant Sections with the same name but
8308
different contents, make the title of each such section unique by
8309
adding at the end of it, in parentheses, the name of the original
8310
author or publisher of that section if known, or else a unique number.
8311
Make the same adjustment to the section titles in the list of
8312
Invariant Sections in the license notice of the combined work.
8313
8314
<P>
8315
In the combination, you must combine any sections Entitled "History"
8316
in the various original documents, forming one section Entitled
8317
"History"; likewise combine any sections Entitled "Acknowledgements",
8318
and any sections Entitled "Dedications". You must delete all sections
8319
Entitled "Endorsements".
8320
8321
<P>
8322
<DIV ALIGN="CENTER">
8323
<FONT SIZE="+2"><B>6. COLLECTIONS OF DOCUMENTS</B></FONT>
8324
<A NAME="5654"></A>
8325
8326
</DIV>
8327
8328
<P>
8329
You may make a collection consisting of the Document and other documents
8330
released under this License, and replace the individual copies of this
8331
License in the various documents with a single copy that is included in
8332
the collection, provided that you follow the rules of this License for
8333
verbatim copying of each of the documents in all other respects.
8334
8335
<P>
8336
You may extract a single document from such a collection, and distribute
8337
it individually under this License, provided you insert a copy of this
8338
License into the extracted document, and follow this License in all
8339
other respects regarding verbatim copying of that document.
8340
8341
<P>
8342
<DIV ALIGN="CENTER">
8343
<FONT SIZE="+2"><B>7. AGGREGATION WITH INDEPENDENT WORKS</B></FONT>
8344
<A NAME="5660"></A>
8345
8346
</DIV>
8347
8348
<P>
8349
A compilation of the Document or its derivatives with other separate
8350
and independent documents or works, in or on a volume of a storage or
8351
distribution medium, is called an "aggregate" if the copyright
8352
resulting from the compilation is not used to limit the legal rights
8353
of the compilation's users beyond what the individual works permit.
8354
When the Document is included in an aggregate, this License does not
8355
apply to the other works in the aggregate which are not themselves
8356
derivative works of the Document.
8357
8358
<P>
8359
If the Cover Text requirement of section 3 is applicable to these
8360
copies of the Document, then if the Document is less than one half of
8361
the entire aggregate, the Document's Cover Texts may be placed on
8362
covers that bracket the Document within the aggregate, or the
8363
electronic equivalent of covers if the Document is in electronic form.
8364
Otherwise they must appear on printed covers that bracket the whole
8365
aggregate.
8366
8367
<P>
8368
<DIV ALIGN="CENTER">
8369
<FONT SIZE="+2"><B>8. TRANSLATION</B></FONT>
8370
<A NAME="5666"></A>
8371
8372
</DIV>
8373
8374
<P>
8375
Translation is considered a kind of modification, so you may
8376
distribute translations of the Document under the terms of section 4.
8377
Replacing Invariant Sections with translations requires special
8378
permission from their copyright holders, but you may include
8379
translations of some or all Invariant Sections in addition to the
8380
original versions of these Invariant Sections. You may include a
8381
translation of this License, and all the license notices in the
8382
Document, and any Warranty Disclaimers, provided that you also include
8383
the original English version of this License and the original versions
8384
of those notices and disclaimers. In case of a disagreement between
8385
the translation and the original version of this License or a notice
8386
or disclaimer, the original version will prevail.
8387
8388
<P>
8389
If a section in the Document is Entitled "Acknowledgements",
8390
"Dedications", or "History", the requirement (section 4) to Preserve
8391
its Title (section 1) will typically require changing the actual
8392
title.
8393
8394
<P>
8395
<DIV ALIGN="CENTER">
8396
<FONT SIZE="+2"><B>9. TERMINATION</B></FONT>
8397
<A NAME="5672"></A>
8398
8399
</DIV>
8400
8401
<P>
8402
You may not copy, modify, sublicense, or distribute the Document except
8403
as expressly provided for under this License. Any other attempt to
8404
copy, modify, sublicense or distribute the Document is void, and will
8405
automatically terminate your rights under this License. However,
8406
parties who have received copies, or rights, from you under this
8407
License will not have their licenses terminated so long as such
8408
parties remain in full compliance.
8409
8410
<P>
8411
<DIV ALIGN="CENTER">
8412
<FONT SIZE="+2"><B>10. FUTURE REVISIONS OF THIS LICENSE</B></FONT>
8413
<A NAME="5678"></A>
8414
8415
</DIV>
8416
8417
<P>
8418
The Free Software Foundation may publish new, revised versions
8419
of the GNU Free Documentation License from time to time. Such new
8420
versions will be similar in spirit to the present version, but may
8421
differ in detail to address new problems or concerns. See
8422
http://www.gnu.org/copyleft/.
8423
8424
<P>
8425
Each version of the License is given a distinguishing version number.
8426
If the Document specifies that a particular numbered version of this
8427
License "or any later version" applies to it, you have the option of
8428
following the terms and conditions either of that specified version or
8429
of any later version that has been published (not as a draft) by the
8430
Free Software Foundation. If the Document does not specify a version
8431
number of this License, you may choose any version ever published (not
8432
as a draft) by the Free Software Foundation.
8433
8434
<P>
8435
<DIV ALIGN="CENTER">
8436
<FONT SIZE="+2"><B>ADDENDUM: How to use this License for your documents</B></FONT>
8437
<A NAME="5684"></A>
8438
8439
</DIV>
8440
8441
<P>
8442
To use this License in a document you have written, include a copy of
8443
the License in the document and put the following copyright and
8444
license notices just after the title page:
8445
8446
<P>
8447
8448
<P><P>
8449
<BR>
8450
<BLOCKQUOTE>
8451
Copyright ©YEAR YOUR NAME.
8452
Permission is granted to copy, distribute and/or modify this document
8453
under the terms of the GNU Free Documentation License, Version 1.2
8454
or any later version published by the Free Software Foundation;
8455
with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
8456
A copy of the license is included in the section entitled "GNU
8457
Free Documentation License".
8458
8459
</BLOCKQUOTE>
8460
8461
<P><P>
8462
<BR>
8463
8464
<P>
8465
If you have Invariant Sections, Front-Cover Texts and Back-Cover Texts,
8466
replace the "with...Texts." line with this:
8467
8468
<P>
8469
8470
<P><P>
8471
<BR>
8472
<BLOCKQUOTE>
8473
with the Invariant Sections being LIST THEIR TITLES, with the
8474
Front-Cover Texts being LIST, and with the Back-Cover Texts being LIST.
8475
8476
</BLOCKQUOTE>
8477
8478
<P><P>
8479
<BR>
8480
8481
<P>
8482
If you have Invariant Sections without Cover Texts, or some other
8483
combination of the three, merge those two alternatives to suit the
8484
situation.
8485
8486
<P>
8487
If your document contains nontrivial examples of program code, we
8488
recommend releasing these examples in parallel under your choice of
8489
free software license, such as the GNU General Public License,
8490
to permit their use in free software.
8491
8492
<P>
8493
8494
<P>
8495
8496
<BR>
8497
8498
<H2><A NAME="SECTION000210000000000000000">
8499
General Index</A>
8500
</H2><DL COMPACT>
8501
<DT><STRONG>Catalog Services </STRONG>
8502
<DD><A HREF="developers.html#1599">Catalog Services</A>
8503
<DT><STRONG>Catalog, Internal Bacula </STRONG>
8504
<DD><A HREF="developers.html#1647">Internal Bacula Catalog</A>
8505
<DT><STRONG>Database Table Design </STRONG>
8506
<DD><A HREF="developers.html#1656">Database Table Design</A>
8507
<DT><STRONG>Database Tables </STRONG>
8508
<DD><A HREF="developers.html#1669">Database Tables</A>
8509
<DT><STRONG>Definition, MySQL Table </STRONG>
8510
<DD><A HREF="developers.html#2322">MySQL Table Definition</A>
8511
<DT><STRONG>Design, Database Table </STRONG>
8512
<DD><A HREF="developers.html#1655">Database Table Design</A>
8513
<DT><STRONG>Difficult, GUI Interface is </STRONG>
8514
<DD><A HREF="developers.html#3423">GUI Interface is Difficult</A>
8515
<DT><STRONG>Filenames and Maximum Filename Length </STRONG>
8516
<DD><A HREF="developers.html#1617">Filenames and Maximum Filename Length</A>
8517
<DT><STRONG>General </STRONG>
8518
<DD><A HREF="developers.html#1604">General</A>
8519
| <A HREF="developers.html#3410">General</A>
8520
<DT><STRONG>GNU ree Documentation License</STRONG>
8521
<DD><A HREF="developers.html#5593">GNU Free Documentation License</A>
8522
<DT><STRONG>GUI Interface is Difficult </STRONG>
8523
<DD><A HREF="developers.html#3422">GUI Interface is Difficult</A>
8524
<DT><STRONG>Implementing a Bacula GUI Interface </STRONG>
8525
<DD><A HREF="developers.html#3405">Implementing a Bacula GUI Interface</A>
8526
<DT><STRONG>Installing and Configuring MySQL </STRONG>
8527
<DD><A HREF="developers.html#1624">Installing and Configuring MySQL</A>
8528
<DT><STRONG>Installing and Configuring PostgreSQL </STRONG>
8529
<DD><A HREF="developers.html#1632">Installing and Configuring PostgreSQL</A>
8530
<DT><STRONG>Installing and Configuring SQLite </STRONG>
8531
<DD><A HREF="developers.html#1639">Installing and Configuring SQLite</A>
8532
<DT><STRONG>Interface, Implementing a Bacula GUI </STRONG>
8533
<DD><A HREF="developers.html#3404">Implementing a Bacula GUI Interface</A>
8534
<DT><STRONG>Internal Bacula Catalog </STRONG>
8535
<DD><A HREF="developers.html#1648">Internal Bacula Catalog</A>
8536
<DT><STRONG>Job, Sequence of Creation of Records for a Save </STRONG>
8537
<DD><A HREF="developers.html#1662">Sequence of Creation of Records for a Save Job</A>
8538
<DT><STRONG>Length, Filenames and Maximum Filename </STRONG>
8539
<DD><A HREF="developers.html#1618">Filenames and Maximum Filename Length</A>
8540
<DT><STRONG>License, GNU ree Documentation</STRONG>
8541
<DD><A HREF="developers.html#5594">GNU Free Documentation License</A>
8542
<DT><STRONG>Minimal Code in Console Program </STRONG>
8543
<DD><A HREF="developers.html#3417">Minimal Code in Console Program</A>
8544
<DT><STRONG>MySQL Table Definition </STRONG>
8545
<DD><A HREF="developers.html#2321">MySQL Table Definition</A>
8546
<DT><STRONG>MySQL, Installing and Configuring </STRONG>
8547
<DD><A HREF="developers.html#1623">Installing and Configuring MySQL</A>
8548
<DT><STRONG>PostgreSQL, Installing and Configuring </STRONG>
8549
<DD><A HREF="developers.html#1631">Installing and Configuring PostgreSQL</A>
8550
<DT><STRONG>Program, Minimal Code in Console </STRONG>
8551
<DD><A HREF="developers.html#3416">Minimal Code in Console Program</A>
8552
<DT><STRONG>Sequence of Creation of Records for a Save Job </STRONG>
8553
<DD><A HREF="developers.html#1661">Sequence of Creation of Records for a Save Job</A>
8554
<DT><STRONG>Services, Catalog </STRONG>
8555
<DD><A HREF="developers.html#1598">Catalog Services</A>
8556
<DT><STRONG>SQLite, Installing and Configuring </STRONG>
8557
<DD><A HREF="developers.html#1640">Installing and Configuring SQLite</A>
8558
<DT><STRONG>Tables, Database </STRONG>
8559
<DD><A HREF="developers.html#1670">Database Tables</A>
8560
8561
</DL>
8562
<P>
8563
8564
<H1><A NAME="SECTION000220000000000000000">
8565
About this document ...</A>
8566
</H1>
8567
<STRONG><IMG
8568
WIDTH="458" HEIGHT="99" ALIGN="BOTTOM" BORDER="0"
8569
SRC="bacula-logo.png"
8570
ALT="\includegraphics{./bacula-logo.eps}">
8571
<BR><P><P>
8572
<BR>
8573
<DIV ALIGN="CENTER">
8574
<FONT SIZE="+1">It comes in the night and sucks
8575
the essence from your computers.
8576
8577
</FONT></DIV></STRONG><P>
8578
This document was generated using the
8579
<A HREF="http://www.latex2html.org/"><STRONG>LaTeX</STRONG>2<tt>HTML</tt></A> translator Version 2002-2-1 (1.70)
8580
<P>
8581
Copyright © 1993, 1994, 1995, 1996,
8582
<A HREF="http://cbl.leeds.ac.uk/nikos/personal.html">Nikos Drakos</A>,
8583
Computer Based Learning Unit, University of Leeds.
8584
<BR>
8585
Copyright © 1997, 1998, 1999,
8586
<A HREF="http://www.maths.mq.edu.au/~ross/">Ross Moore</A>,
8587
Mathematics Department, Macquarie University, Sydney.
8588
<P>
8589
The command line arguments were: <BR>
8590
<STRONG>latex2html</STRONG> <TT>-white -no_subdir -split 0 -toc_stars -white -notransparent developers</TT>
8591
<P>
8592
The translation was initiated by on 2006-07-01<HR>
8593
<!--Navigation Panel-->
8594
<IMG WIDTH="81" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next_inactive"
8595
SRC="file:/usr/share/latex2html/icons/nx_grp_g.png">
8596
<IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up"
8597
SRC="file:/usr/share/latex2html/icons/up_g.png">
8598
<IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous"
8599
SRC="file:/usr/share/latex2html/icons/prev_g.png">
8600
<BR>
8601
<!--End of Navigation Panel-->
8602
<ADDRESS>
8603
8604
2006-07-01
8605
</ADDRESS>
8606
</BODY>
8607
</HTML>
Loggerhead is a web-based interface for Breezy
Version: 2.0.1