4
\section*{Storage Media Output Format}
6
\index{Format!Storage Media Output}
7
\index{Storage Media Output Format}
8
\addcontentsline{toc}{section}{Storage Media Output Format}
12
\addcontentsline{toc}{subsection}{General}
14
This document describes the media format written by the Storage daemon. The
15
Storage daemon reads and writes in units of blocks. Blocks contain records.
16
Each block has a block header followed by records, and each record has a
17
record header followed by record data.
19
This chapter is intended to be a technical discussion of the Media Format and
20
as such is not targeted at end users but rather at developers and system
21
administrators that want or need to know more of the working details of {\bf
24
\subsection*{Definitions}
26
\addcontentsline{toc}{subsection}{Definitions}
32
A block represents the primitive unit of information that the Storage daemon
33
reads and writes to a physical device. Normally, for a tape device, it will
34
be the same as a tape block. The Storage daemon always reads and writes
35
blocks. A block consists of block header information followed by records.
36
Clients of the Storage daemon (the File daemon) normally never see blocks.
37
However, some of the Storage tools (bls, bscan, bextract, ...) may be use
38
block header information. In older Bacula tape versions, a block could
39
contain records (see record definition below) from multiple jobs. However,
40
all blocks currently written by Bacula are block level BB02, and a given
41
block contains records for only a single job. Different jobs simply have
42
their own private blocks that are intermingled with the other blocks from
43
other jobs on the Volume (previously the records were intermingled within
44
the blocks). Having only records from a single job in any give block
45
permitted moving the VolumeSessionId and VolumeSessionTime (see below) from
46
each record heading to the Block header. This has two advantages: 1. a block
47
can be quickly rejected based on the contents of the header without reading
48
all the records. 2. because there is on the average more than one record per
49
block, less data is written to the Volume for each job.
53
A record consists of a Record Header, which is managed by the Storage daemon
54
and Record Data, which is the data received from the Client. A record is the
55
primitive unit of information sent to and from the Storage daemon by the
56
Client (File daemon) programs. The details are described below.
60
A number assigned by the Director daemon for a particular job. This number
61
will be unique for that particular Director (Catalog). The daemons use this
62
number to keep track of individual jobs. Within the Storage daemon, the JobId
63
may not be unique if several Directors are accessing the Storage daemon
68
A Session is a concept used in the Storage daemon corresponds one to one to a
69
Job with the exception that each session is uniquely identified within the
70
Storage daemon by a unique SessionId/SessionTime pair (see below).
74
A unique number assigned by the Storage daemon to a particular session (Job)
75
it is having with a File daemon. This number by itself is not unique to the
76
given Volume, but with the VolSessionTime, it is unique.
78
\item [VolSessionTime]
79
\index{VolSessionTime}
80
A unique number assigned by the Storage daemon to a particular Storage daemon
81
execution. It is actually the Unix time\_t value of when the Storage daemon
82
began execution cast to a 32 bit unsigned integer. The combination of the
83
{\bf VolSessionId} and the {\bf VolSessionTime} for a given Storage daemon is
84
guaranteed to be unique for each Job (or session).
88
A sequential number beginning at one assigned by the File daemon to the files
89
within a job that are sent to the Storage daemon for backup. The Storage
90
daemon ensures that this number is greater than zero and sequential. Note,
91
the Storage daemon uses negative FileIndexes to flag Session Start and End
92
Labels as well as End of Volume Labels. Thus, the combination of
93
VolSessionId, VolSessionTime, and FileIndex uniquely identifies the records
94
for a single file written to a Volume.
98
While writing the information for any particular file to the Volume, there
99
can be any number of distinct pieces of information about that file, e.g. the
100
attributes, the file data, ... The Stream indicates what piece of data it
101
is, and it is an arbitrary number assigned by the File daemon to the parts
102
(Unix attributes, Win32 attributes, data, compressed data,\ ...) of a file
103
that are sent to the Storage daemon. The Storage daemon has no knowledge of
104
the details of a Stream; it simply represents a numbered stream of bytes. The
105
data for a given stream may be passed to the Storage daemon in single record,
106
or in multiple records.
110
A block header consists of a block identification (``BB02''), a block length
111
in bytes (typically 64,512) a checksum, and sequential block number. Each
112
block starts with a Block Header and is followed by Records. Current block
113
headers also contain the VolSessionId and VolSessionTime for the records
114
written to that block.
116
\item [Record Header]
117
\index{Record Header}
118
A record header contains the Volume Session Id, the Volume Session Time, the
119
FileIndex, the Stream, and the size of the data record which follows. The
120
Record Header is always immediately followed by a Data Record if the size
121
given in the Header is greater than zero. Note, for Block headers of level
122
BB02 (version 1.27 and later), the Record header as written to tape does not
123
contain the Volume Session Id and the Volume Session Time as these two
124
fields are stored in the BB02 Block header. The in-memory record header does
125
have those fields for convenience.
129
A data record consists of a binary stream of bytes and is always preceded by
130
a Record Header. The details of the meaning of the binary stream of bytes are
131
unknown to the Storage daemon, but the Client programs (File daemon) defines
132
and thus knows the details of each record type.
136
A label placed by the Storage daemon at the beginning of each storage volume.
137
It contains general information about the volume. It is written in Record
138
format. The Storage daemon manages Volume Labels, and if the client wants, he
141
\item [Begin Session Label]
142
\index{Begin Session Label}
143
The Begin Session Label is a special record placed by the Storage daemon on
144
the storage medium as the first record of an append session job with a File
145
daemon. This record is useful for finding the beginning of a particular
146
session (Job), since no records with the same VolSessionId and VolSessionTime
147
will precede this record. This record is not normally visible outside of the
148
Storage daemon. The Begin Session Label is similar to the Volume Label except
149
that it contains additional information pertaining to the Session.
151
\item [End Session Label]
152
\index{End Session Label}
153
The End Session Label is a special record placed by the Storage daemon on the
154
storage medium as the last record of an append session job with a File
155
daemon. The End Session Record is distinguished by a FileIndex with a value
156
of minus two (-2). This record is useful for detecting the end of a
157
particular session since no records with the same VolSessionId and
158
VolSessionTime will follow this record. This record is not normally visible
159
outside of the Storage daemon. The End Session Label is similar to the Volume
160
Label except that it contains additional information pertaining to the
164
\subsection*{Storage Daemon File Output Format}
165
\index{Format!Storage Daemon File Output}
166
\index{Storage Daemon File Output Format}
167
\addcontentsline{toc}{subsection}{Storage Daemon File Output Format}
169
The file storage and tape storage formats are identical except that tape
170
records are by default blocked into blocks of 64,512 bytes, except for the
171
last block, which is the actual number of bytes written rounded up to a
172
multiple of 1024 whereas the last record of file storage is not rounded up.
173
The default block size of 64,512 bytes may be overridden by the user (some
174
older tape drives only support block sizes of 32K). Each Session written to
175
tape is terminated with an End of File mark (this will be removed later).
176
Sessions written to file are simply appended to the end of the file.
178
\subsection*{Overall Format}
179
\index{Format!Overall}
180
\index{Overall Format}
181
\addcontentsline{toc}{subsection}{Overall Format}
183
A Bacula output file consists of Blocks of data. Each block contains a block
184
header followed by records. Each record consists of a record header followed
185
by the record data. The first record on a tape will always be the Volume Label
188
No Record Header will be split across Bacula blocks. However, Record Data may
189
be split across any number of Bacula blocks. Obviously this will not be the
190
case for the Volume Label which will always be smaller than the Bacula Block
193
To simplify reading tapes, the Start of Session (SOS) and End of Session (EOS)
194
records are never split across blocks. If this is about to happen, Bacula will
195
write a short block before writing the session record (actually, the SOS
196
record should always be the first record in a block, excepting perhaps the
199
Due to hardware limitations, the last block written to the tape may not be
200
fully written. If your drive permits backspace record, Bacula will backup over
201
the last record written on the tape, re-read it and verify that it was
204
When a new tape is mounted Bacula will write the full contents of the
205
partially written block to the new tape ensuring that there is no loss of
206
data. When reading a tape, Bacula will discard any block that is not totally
207
written, thus ensuring that there is no duplication of data. In addition,
208
since Bacula blocks are sequentially numbered within a Job, it is easy to
209
ensure that no block is missing or duplicated.
211
\subsection*{Serialization}
212
\index{Serialization}
213
\addcontentsline{toc}{subsection}{Serialization}
215
All Block Headers, Record Headers, and Label Records are written using
216
Bacula's serialization routines. These routines guarantee that the data is
217
written to the output volume in a machine independent format.
219
\subsection*{Block Header}
222
\addcontentsline{toc}{subsection}{Block Header}
224
The format of the Block Header (version 1.27 and later) is:
228
uint32_t CheckSum; /* Block check sum */
229
uint32_t BlockSize; /* Block byte size including the header */
230
uint32_t BlockNumber; /* Block number */
231
char ID[4] = "BB02"; /* Identification and block level */
232
uint32_t VolSessionId; /* Session Id for Job */
233
uint32_t VolSessionTime; /* Session Time for Job */
237
The Block header is a fixed length and fixed format and is followed by Record
238
Headers and Record Data. The CheckSum field is a 32 bit checksum of the block
239
data and the block header but not including the CheckSum field. The Block
240
Header is always immediately followed by a Record Header. If the tape is
241
damaged, a Bacula utility will be able to recover as much information as
242
possible from the tape by recovering blocks which are valid. The Block header
243
is written using the Bacula serialization routines and thus is guaranteed to
244
be in machine independent format. See below for version 2 of the block header.
247
\subsection*{Record Header}
248
\index{Header!Record}
249
\index{Record Header}
250
\addcontentsline{toc}{subsection}{Record Header}
252
Each binary data record is preceded by a Record Header. The Record Header is
253
fixed length and fixed format, whereas the binary data record is of variable
254
length. The Record Header is written using the Bacula serialization routines
255
and thus is guaranteed to be in machine independent format.
257
The format of the Record Header (version 1.27 or later) is:
261
int32_t FileIndex; /* File index supplied by File daemon */
262
int32_t Stream; /* Stream number supplied by File daemon */
263
uint32_t DataSize; /* size of following data record in bytes */
267
This record is followed by the binary Stream data of DataSize bytes, followed
268
by another Record Header record and the binary stream data. For the definitive
269
definition of this record, see record.h in the src/stored directory.
271
Additional notes on the above:
275
\item [The {\bf VolSessionId} ]
277
is a unique sequential number that is assigned by the Storage Daemon to a
278
particular Job. This number is sequential since the start of execution of the
281
\item [The {\bf VolSessionTime} ]
282
\index{VolSessionTime}
283
is the time/date that the current execution of the Storage Daemon started. It
284
assures that the combination of VolSessionId and VolSessionTime is unique for
285
every jobs written to the tape, even if there was a machine crash between two
288
\item [The {\bf FileIndex} ]
290
is a sequential file number within a job. The Storage daemon requires this
291
index to be greater than zero and sequential. Note, however, that the File
292
daemon may send multiple Streams for the same FileIndex. In addition, the
293
Storage daemon uses negative FileIndices to hold the Begin Session Label, the
294
End Session Label, and the End of Volume Label.
296
\item [The {\bf Stream} ]
298
is defined by the File daemon and is used to identify separate parts of the
299
data saved for each file (Unix attributes, Win32 attributes, file data,
300
compressed file data, sparse file data, ...). The Storage Daemon has no idea
301
of what a Stream is or what it contains except that the Stream is required to
302
be a positive integer. Negative Stream numbers are used internally by the
303
Storage daemon to indicate that the record is a continuation of the previous
304
record (the previous record would not entirely fit in the block).
306
For Start Session and End Session Labels (where the FileIndex is negative),
307
the Storage daemon uses the Stream field to contain the JobId. The current
308
stream definitions are:
312
#define STREAM_UNIX_ATTRIBUTES 1 /* Generic Unix attributes */
313
#define STREAM_FILE_DATA 2 /* Standard uncompressed data */
314
#define STREAM_MD5_SIGNATURE 3 /* MD5 signature for the file */
315
#define STREAM_GZIP_DATA 4 /* GZip compressed file data */
316
/* Extended Unix attributes with Win32 Extended data. Deprecated. */
317
#define STREAM_UNIX_ATTRIBUTES_EX 5 /* Extended Unix attr for Win32 EX */
318
#define STREAM_SPARSE_DATA 6 /* Sparse data stream */
319
#define STREAM_SPARSE_GZIP_DATA 7
320
#define STREAM_PROGRAM_NAMES 8 /* program names for program data */
321
#define STREAM_PROGRAM_DATA 9 /* Data needing program */
322
#define STREAM_SHA1_SIGNATURE 10 /* SHA1 signature for the file */
323
#define STREAM_WIN32_DATA 11 /* Win32 BackupRead data */
324
#define STREAM_WIN32_GZIP_DATA 12 /* Gzipped Win32 BackupRead data */
325
#define STREAM_MACOS_FORK_DATA 13 /* Mac resource fork */
326
#define STREAM_HFSPLUS_ATTRIBUTES 14 /* Mac OS extra attributes */
327
#define STREAM_UNIX_ATTRIBUTES_ACCESS_ACL 15 /* Standard ACL attributes on UNIX */
328
#define STREAM_UNIX_ATTRIBUTES_DEFAULT_ACL 16 /* Default ACL attributes on UNIX */
332
\item [The {\bf DataSize} ]
334
is the size in bytes of the binary data record that follows the Session
335
Record header. The Storage Daemon has no idea of the actual contents of the
336
binary data record. For standard Unix files, the data record typically
337
contains the file attributes or the file data. For a sparse file the first
338
64 bits of the file data contains the storage address for the data block.
341
The Record Header is never split across two blocks. If there is not enough
342
room in a block for the full Record Header, the block is padded to the end
343
with zeros and the Record Header begins in the next block. The data record, on
344
the other hand, may be split across multiple blocks and even multiple physical
345
volumes. When a data record is split, the second (and possibly subsequent)
346
piece of the data is preceded by a new Record Header. Thus each piece of data
347
is always immediately preceded by a Record Header. When reading a record, if
348
Bacula finds only part of the data in the first record, it will automatically
349
read the next record and concatenate the data record to form a full data
352
\subsection*{Version BB02 Block Header}
353
\index{Version BB02 Block Header}
354
\index{Header!Version BB02 Block}
355
\addcontentsline{toc}{subsection}{Version BB02 Block Header}
357
Each session or Job has its own private block. As a consequence, the SessionId
358
and SessionTime are written once in each Block Header and not in the Record
359
Header. So, the second and current version of the Block Header BB02 is:
363
uint32_t CheckSum; /* Block check sum */
364
uint32_t BlockSize; /* Block byte size including the header */
365
uint32_t BlockNumber; /* Block number */
366
char ID[4] = "BB02"; /* Identification and block level */
367
uint32_t VolSessionId; /* Applies to all records */
368
uint32_t VolSessionTime; /* contained in this block */
372
As with the previous version, the BB02 Block header is a fixed length and
373
fixed format and is followed by Record Headers and Record Data. The CheckSum
374
field is a 32 bit CRC checksum of the block data and the block header but not
375
including the CheckSum field. The Block Header is always immediately followed
376
by a Record Header. If the tape is damaged, a Bacula utility will be able to
377
recover as much information as possible from the tape by recovering blocks
378
which are valid. The Block header is written using the Bacula serialization
379
routines and thus is guaranteed to be in machine independent format.
381
\subsection*{Version 2 Record Header}
382
\index{Version 2 Record Header}
383
\index{Header!Version 2 Record}
384
\addcontentsline{toc}{subsection}{Version 2 Record Header}
386
Version 2 Record Header is written to the medium when using Version BB02 Block
387
Headers. The memory representation of the record is identical to the old BB01
388
Record Header, but on the storage medium, the first two fields, namely
389
VolSessionId and VolSessionTime are not written. The Block Header is filled
390
with these values when the First user record is written (i.e. non label
391
record) so that when the block is written, it will have the current and unique
392
VolSessionId and VolSessionTime. On reading each record from the Block, the
393
VolSessionId and VolSessionTime is filled in the Record Header from the Block
396
\subsection*{Volume Label Format}
397
\index{Volume Label Format}
398
\index{Format!Volume Label}
399
\addcontentsline{toc}{subsection}{Volume Label Format}
401
Tape volume labels are created by the Storage daemon in response to a {\bf
402
label} command given to the Console program, or alternatively by the {\bf
403
btape} program. created. Each volume is labeled with the following information
404
using the Bacula serialization routines, which guarantee machine byte order
407
For Bacula versions 1.27 and later, the Volume Label Format is:
411
char Id[32]; /* Bacula 1.0 Immortal\n */
412
uint32_t VerNum; /* Label version number */
413
/* VerNum 11 and greater Bacula 1.27 and later */
414
btime_t label_btime; /* Time/date tape labeled */
415
btime_t write_btime; /* Time/date tape first written */
416
/* The following are 0 in VerNum 11 and greater */
417
float64_t write_date; /* Date this label written */
418
float64_t write_time; /* Time this label written */
419
char VolName[128]; /* Volume name */
420
char PrevVolName[128]; /* Previous Volume Name */
421
char PoolName[128]; /* Pool name */
422
char PoolType[128]; /* Pool type */
423
char MediaType[128]; /* Type of this media */
424
char HostName[128]; /* Host name of writing computer */
425
char LabelProg[32]; /* Label program name */
426
char ProgVersion[32]; /* Program version */
427
char ProgDate[32]; /* Program build date/time */
431
Note, the LabelType (Volume Label, Volume PreLabel, Session Start Label, ...)
432
is stored in the record FileIndex field of the Record Header and does not
433
appear in the data part of the record.
435
\subsection*{Session Label}
436
\index{Label!Session}
437
\index{Session Label}
438
\addcontentsline{toc}{subsection}{Session Label}
440
The Session Label is written at the beginning and end of each session as well
441
as the last record on the physical medium. It has the following binary format:
446
char Id[32]; /* Bacula Immortal ... */
447
uint32_t VerNum; /* Label version number */
448
uint32_t JobId; /* Job id */
449
uint32_t VolumeIndex; /* sequence no of vol */
450
/* Prior to VerNum 11 */
451
float64_t write_date; /* Date this label written */
452
/* VerNum 11 and greater */
453
btime_t write_btime; /* time/date record written */
454
/* The following is zero VerNum 11 and greater */
455
float64_t write_time; /* Time this label written */
456
char PoolName[128]; /* Pool name */
457
char PoolType[128]; /* Pool type */
458
char JobName[128]; /* base Job name */
459
char ClientName[128];
460
/* Added in VerNum 10 */
461
char Job[128]; /* Unique Job name */
462
char FileSetName[128]; /* FileSet name */
468
In addition, the EOS label contains:
472
/* The remainder are part of EOS label only */
475
uint32_t start_block;
483
In addition, for VerNum greater than 10, the EOS label contains (in addition
488
uint32_t JobStatus /* Job termination code */
492
: Note, the LabelType (Volume Label, Volume PreLabel, Session Start Label,
493
...) is stored in the record FileIndex field and does not appear in the data
494
part of the record. Also, the Stream field of the Record Header contains the
495
JobId. This permits quick filtering without actually reading all the session
498
\subsection*{Overall Storage Format}
499
\index{Format!Overall Storage}
500
\index{Overall Storage Format}
501
\addcontentsline{toc}{subsection}{Overall Storage Format}
505
Current Bacula Tape Format
507
Version BB02 added 28 September 2002
508
Version BB01 is the old deprecated format.
509
A Bacula tape is composed of tape Blocks. Each block
510
has a Block header followed by the block data. Block
511
Data consists of Records. Records consist of Record
512
Headers followed by Record Data.
513
:=======================================================:
515
| Block Header (24 bytes) |
517
|-------------------------------------------------------|
519
| Record Header (12 bytes) |
521
|-------------------------------------------------------|
525
|-------------------------------------------------------|
527
| Record Header (12 bytes) |
529
|-------------------------------------------------------|
532
Block Header: the first item in each block. The format is
534
Partial Data block: occurs if the data from a previous
535
block spills over to this block (the normal case except
536
for the first block on a tape). However, this partial
537
data block is always preceded by a record header.
538
Record Header: identifies the Volume Session, the Stream
539
and the following Record Data size. See below for format.
540
Record data: arbitrary binary data.
541
Block Header Format BB02
542
:=======================================================:
543
| CheckSum (uint32_t) |
544
|-------------------------------------------------------|
545
| BlockSize (uint32_t) |
546
|-------------------------------------------------------|
547
| BlockNumber (uint32_t) |
548
|-------------------------------------------------------|
549
| "BB02" (char [4]) |
550
|-------------------------------------------------------|
551
| VolSessionId (uint32_t) |
552
|-------------------------------------------------------|
553
| VolSessionTime (uint32_t) |
554
:=======================================================:
555
BBO2: Serves to identify the block as a
556
Bacula block and also servers as a block format identifier
557
should we ever need to change the format.
558
BlockSize: is the size in bytes of the block. When reading
559
back a block, if the BlockSize does not agree with the
560
actual size read, Bacula discards the block.
561
CheckSum: a checksum for the Block.
562
BlockNumber: is the sequential block number on the tape.
563
VolSessionId: a unique sequential number that is assigned
564
by the Storage Daemon to a particular Job.
565
This number is sequential since the start
566
of execution of the daemon.
567
VolSessionTime: the time/date that the current execution
568
of the Storage Daemon started. It assures
569
that the combination of VolSessionId and
570
VolSessionTime is unique for all jobs
571
written to the tape, even if there was a
572
machine crash between two writes.
573
Record Header Format BB02
574
:=======================================================:
575
| FileIndex (int32_t) |
576
|-------------------------------------------------------|
578
|-------------------------------------------------------|
579
| DataSize (uint32_t) |
580
:=======================================================:
581
FileIndex: a sequential file number within a job. The
582
Storage daemon enforces this index to be
583
greater than zero and sequential. Note,
584
however, that the File daemon may send
585
multiple Streams for the same FileIndex.
586
The Storage Daemon uses negative FileIndices
587
to identify Session Start and End labels
588
as well as the End of Volume labels.
589
Stream: defined by the File daemon and is intended to be
590
used to identify separate parts of the data
591
saved for each file (attributes, file data,
592
...). The Storage Daemon has no idea of
593
what a Stream is or what it contains.
594
DataSize: the size in bytes of the binary data record
595
that follows the Session Record header.
596
The Storage Daemon has no idea of the
597
actual contents of the binary data record.
598
For standard Unix files, the data record
599
typically contains the file attributes or
600
the file data. For a sparse file
601
the first 64 bits of the data contains
602
the storage address for the data block.
604
:=======================================================:
606
|-------------------------------------------------------|
607
| VerNum (uint32_t) |
608
|-------------------------------------------------------|
609
| label_date (float64_t) |
610
| label_btime (btime_t VerNum 11 |
611
|-------------------------------------------------------|
612
| label_time (float64_t) |
613
| write_btime (btime_t VerNum 11 |
614
|-------------------------------------------------------|
615
| write_date (float64_t) |
616
| 0 (float64_t) VerNum 11 |
617
|-------------------------------------------------------|
618
| write_time (float64_t) |
619
| 0 (float64_t) VerNum 11 |
620
|-------------------------------------------------------|
621
| VolName (128 bytes) |
622
|-------------------------------------------------------|
623
| PrevVolName (128 bytes) |
624
|-------------------------------------------------------|
625
| PoolName (128 bytes) |
626
|-------------------------------------------------------|
627
| PoolType (128 bytes) |
628
|-------------------------------------------------------|
629
| MediaType (128 bytes) |
630
|-------------------------------------------------------|
631
| HostName (128 bytes) |
632
|-------------------------------------------------------|
633
| LabelProg (32 bytes) |
634
|-------------------------------------------------------|
635
| ProgVersion (32 bytes) |
636
|-------------------------------------------------------|
637
| ProgDate (32 bytes) |
638
|-------------------------------------------------------|
639
:=======================================================:
641
Id: 32 byte Bacula identifier "Bacula 1.0 immortal\n"
642
(old version also recognized:)
643
Id: 32 byte Bacula identifier "Bacula 0.9 mortal\n"
644
LabelType (Saved in the FileIndex of the Header record).
645
PRE_LABEL -1 Volume label on unwritten tape
646
VOL_LABEL -2 Volume label after tape written
647
EOM_LABEL -3 Label at EOM (not currently implemented)
648
SOS_LABEL -4 Start of Session label (format given below)
649
EOS_LABEL -5 End of Session label (format given below)
651
label_date: Julian day tape labeled
652
label_time: Julian time tape labeled
653
write_date: Julian date tape first used (data written)
654
write_time: Julian time tape first used (data written)
655
VolName: "Physical" Volume name
656
PrevVolName: The VolName of the previous tape (if this tape is
657
a continuation of the previous one).
660
MediaType: Media Type
661
HostName: Name of host that is first writing the tape
662
LabelProg: Name of the program that labeled the tape
663
ProgVersion: Version of the label program
664
ProgDate: Date Label program built
666
:=======================================================:
668
|-------------------------------------------------------|
669
| VerNum (uint32_t) |
670
|-------------------------------------------------------|
672
|-------------------------------------------------------|
673
| write_btime (btime_t) VerNum 11 |
674
|-------------------------------------------------------|
675
| 0 (float64_t) VerNum 11 |
676
|-------------------------------------------------------|
677
| PoolName (128 bytes) |
678
|-------------------------------------------------------|
679
| PoolType (128 bytes) |
680
|-------------------------------------------------------|
681
| JobName (128 bytes) |
682
|-------------------------------------------------------|
683
| ClientName (128 bytes) |
684
|-------------------------------------------------------|
686
|-------------------------------------------------------|
687
| FileSetName (128 bytes) |
688
|-------------------------------------------------------|
689
| JobType (uint32_t) |
690
|-------------------------------------------------------|
691
| JobLevel (uint32_t) |
692
|-------------------------------------------------------|
693
| FileSetMD5 (50 bytes) VerNum 11 |
694
|-------------------------------------------------------|
695
Additional fields in End Of Session Label
696
|-------------------------------------------------------|
697
| JobFiles (uint32_t) |
698
|-------------------------------------------------------|
699
| JobBytes (uint32_t) |
700
|-------------------------------------------------------|
701
| start_block (uint32_t) |
702
|-------------------------------------------------------|
703
| end_block (uint32_t) |
704
|-------------------------------------------------------|
705
| start_file (uint32_t) |
706
|-------------------------------------------------------|
707
| end_file (uint32_t) |
708
|-------------------------------------------------------|
709
| JobErrors (uint32_t) |
710
|-------------------------------------------------------|
711
| JobStatus (uint32_t) VerNum 11 |
712
:=======================================================:
713
* => fields deprecated
714
Id: 32 byte Bacula Identifier "Bacula 1.0 immortal\n"
715
LabelType (in FileIndex field of Header):
716
EOM_LABEL -3 Label at EOM
717
SOS_LABEL -4 Start of Session label
718
EOS_LABEL -5 End of Session label
721
write_btime: Bacula time/date this tape record written
722
write_date: Julian date tape this record written - deprecated
723
write_time: Julian time tape this record written - deprecated.
726
MediaType: Media Type
727
ClientName: Name of File daemon or Client writing this session
728
Not used for EOM_LABEL.
732
\subsection*{Unix File Attributes}
733
\index{Unix File Attributes}
734
\index{Attributes!Unix File}
735
\addcontentsline{toc}{subsection}{Unix File Attributes}
737
The Unix File Attributes packet consists of the following:
739
\lt{}File-Index\gt{} \lt{}Type\gt{}
740
\lt{}Filename\gt{}@\lt{}File-Attributes\gt{}@\lt{}Link\gt{}
741
@\lt{}Extended-Attributes@\gt{} where
746
represents a byte containing a binary zero.
750
is the sequential file index starting from one assigned by the File daemon.
754
is one of the following:
758
#define FT_LNKSAVED 1 /* hard link to file already saved */
759
#define FT_REGE 2 /* Regular file but empty */
760
#define FT_REG 3 /* Regular file */
761
#define FT_LNK 4 /* Soft Link */
762
#define FT_DIR 5 /* Directory */
763
#define FT_SPEC 6 /* Special file -- chr, blk, fifo, sock */
764
#define FT_NOACCESS 7 /* Not able to access */
765
#define FT_NOFOLLOW 8 /* Could not follow link */
766
#define FT_NOSTAT 9 /* Could not stat file */
767
#define FT_NOCHG 10 /* Incremental option, file not changed */
768
#define FT_DIRNOCHG 11 /* Incremental option, directory not changed */
769
#define FT_ISARCH 12 /* Trying to save archive file */
770
#define FT_NORECURSE 13 /* No recursion into directory */
771
#define FT_NOFSCHG 14 /* Different file system, prohibited */
772
#define FT_NOOPEN 15 /* Could not open directory */
773
#define FT_RAW 16 /* Raw block device */
774
#define FT_FIFO 17 /* Raw fifo device */
780
is the fully qualified filename.
782
\item [File-Attributes]
783
\index{File-Attributes}
784
consists of the 13 fields of the stat() buffer in ASCII base64 format
785
separated by spaces. These fields and their meanings are shown below. This
786
stat() packet is in Unix format, and MUST be provided (constructed) for ALL
791
when the FT code is FT\_LNK or FT\_LNKSAVED, the item in question is a Unix
792
link, and this field contains the fully qualified link name. When the FT code
793
is not FT\_LNK or FT\_LNKSAVED, this field is null.
795
\item [Extended-Attributes]
796
\index{Extended-Attributes}
797
The exact format of this field is operating system dependent. It contains
798
additional or extended attributes of a system dependent nature. Currently,
799
this field is used only on WIN32 systems where it contains a ASCII base64
800
representation of the WIN32\_FILE\_ATTRIBUTE\_DATA structure as defined by
801
Windows. The fields in the base64 representation of this structure are like
802
the File-Attributes separated by spaces.
805
The File-attributes consist of the following:
807
\addcontentsline{lot}{table}{File Attributes}
808
\begin{longtable}{|p{0.6in}|p{0.7in}|p{1in}|p{1in}|p{1.4in}|}
810
\multicolumn{1}{|c|}{\bf Field No. } & \multicolumn{1}{c|}{\bf Stat Name }
811
& \multicolumn{1}{c|}{\bf Unix } & \multicolumn{1}{c|}{\bf Win98/NT } &
812
\multicolumn{1}{c|}{\bf MacOS } \\
814
\multicolumn{1}{|c|}{1 } & {st\_dev } & {Device number of filesystem } &
815
{Drive number } & {vRefNum } \\
817
\multicolumn{1}{|c|}{2 } & {st\_ino } & {Inode number } & {Always 0 } &
820
\multicolumn{1}{|c|}{3 } & {st\_mode } & {File mode } & {File mode } &
821
{777 dirs/apps; 666 docs; 444 locked docs } \\
823
\multicolumn{1}{|c|}{4 } & {st\_nlink } & {Number of links to the file } &
824
{Number of link (only on NTFS) } & {Always 1 } \\
826
\multicolumn{1}{|c|}{5 } & {st\_uid } & {Owner ID } & {Always 0 } &
829
\multicolumn{1}{|c|}{6 } & {st\_gid } & {Group ID } & {Always 0 } &
832
\multicolumn{1}{|c|}{7 } & {st\_rdev } & {Device ID for special files } &
833
{Drive No. } & {Always 0 } \\
835
\multicolumn{1}{|c|}{8 } & {st\_size } & {File size in bytes } & {File
836
size in bytes } & {Data fork file size in bytes } \\
838
\multicolumn{1}{|c|}{9 } & {st\_blksize } & {Preferred block size } &
839
{Always 0 } & {Preferred block size } \\
841
\multicolumn{1}{|c|}{10 } & {st\_blocks } & {Number of blocks allocated }
842
& {Always 0 } & {Number of blocks allocated } \\
844
\multicolumn{1}{|c|}{11 } & {st\_atime } & {Last access time since epoch }
845
& {Last access time since epoch } & {Last access time -66 years } \\
847
\multicolumn{1}{|c|}{12 } & {st\_mtime } & {Last modify time since epoch }
848
& {Last modify time since epoch } & {Last access time -66 years } \\
850
\multicolumn{1}{|c|}{13 } & {st\_ctime } & {Inode change time since epoch
851
} & {File create time since epoch } & {File create time -66 years}
856
\subsection*{Old Depreciated Tape Format}
857
\index{Old Depreciated Tape Format}
858
\index{Format!Old Depreciated Tape}
859
\addcontentsline{toc}{subsection}{Old Depreciated Tape Format}
861
The format of the Block Header (version 1.26 and earlier) is:
865
uint32_t CheckSum; /* Block check sum */
866
uint32_t BlockSize; /* Block byte size including the header */
867
uint32_t BlockNumber; /* Block number */
868
char ID[4] = "BB01"; /* Identification and block level */
872
The format of the Record Header (version 1.26 or earlier) is:
876
uint32_t VolSessionId; /* Unique ID for this session */
877
uint32_t VolSessionTime; /* Start time/date of session */
878
int32_t FileIndex; /* File index supplied by File daemon */
879
int32_t Stream; /* Stream number supplied by File daemon */
880
uint32_t DataSize; /* size of following data record in bytes */
886
Current Bacula Tape Format
888
Version BB01 is the old deprecated format.
889
A Bacula tape is composed of tape Blocks. Each block
890
has a Block header followed by the block data. Block
891
Data consists of Records. Records consist of Record
892
Headers followed by Record Data.
893
:=======================================================:
896
| (16 bytes version BB01) |
897
|-------------------------------------------------------|
900
| (20 bytes version BB01) |
901
|-------------------------------------------------------|
905
|-------------------------------------------------------|
908
| (20 bytes version BB01) |
909
|-------------------------------------------------------|
912
Block Header: the first item in each block. The format is
914
Partial Data block: occurs if the data from a previous
915
block spills over to this block (the normal case except
916
for the first block on a tape). However, this partial
917
data block is always preceded by a record header.
918
Record Header: identifies the Volume Session, the Stream
919
and the following Record Data size. See below for format.
920
Record data: arbitrary binary data.
921
Block Header Format BB01 (deprecated)
922
:=======================================================:
923
| CheckSum (uint32_t) |
924
|-------------------------------------------------------|
925
| BlockSize (uint32_t) |
926
|-------------------------------------------------------|
927
| BlockNumber (uint32_t) |
928
|-------------------------------------------------------|
929
| "BB01" (char [4]) |
930
:=======================================================:
931
BBO1: Serves to identify the block as a
932
Bacula block and also servers as a block format identifier
933
should we ever need to change the format.
934
BlockSize: is the size in bytes of the block. When reading
935
back a block, if the BlockSize does not agree with the
936
actual size read, Bacula discards the block.
937
CheckSum: a checksum for the Block.
938
BlockNumber: is the sequential block number on the tape.
939
VolSessionId: a unique sequential number that is assigned
940
by the Storage Daemon to a particular Job.
941
This number is sequential since the start
942
of execution of the daemon.
943
VolSessionTime: the time/date that the current execution
944
of the Storage Daemon started. It assures
945
that the combination of VolSessionId and
946
VolSessionTime is unique for all jobs
947
written to the tape, even if there was a
948
machine crash between two writes.
949
Record Header Format BB01 (deprecated)
950
:=======================================================:
951
| VolSessionId (uint32_t) |
952
|-------------------------------------------------------|
953
| VolSessionTime (uint32_t) |
954
|-------------------------------------------------------|
955
| FileIndex (int32_t) |
956
|-------------------------------------------------------|
958
|-------------------------------------------------------|
959
| DataSize (uint32_t) |
960
:=======================================================:
961
VolSessionId: a unique sequential number that is assigned
962
by the Storage Daemon to a particular Job.
963
This number is sequential since the start
964
of execution of the daemon.
965
VolSessionTime: the time/date that the current execution
966
of the Storage Daemon started. It assures
967
that the combination of VolSessionId and
968
VolSessionTime is unique for all jobs
969
written to the tape, even if there was a
970
machine crash between two writes.
971
FileIndex: a sequential file number within a job. The
972
Storage daemon enforces this index to be
973
greater than zero and sequential. Note,
974
however, that the File daemon may send
975
multiple Streams for the same FileIndex.
976
The Storage Daemon uses negative FileIndices
977
to identify Session Start and End labels
978
as well as the End of Volume labels.
979
Stream: defined by the File daemon and is intended to be
980
used to identify separate parts of the data
981
saved for each file (attributes, file data,
982
...). The Storage Daemon has no idea of
983
what a Stream is or what it contains.
984
DataSize: the size in bytes of the binary data record
985
that follows the Session Record header.
986
The Storage Daemon has no idea of the
987
actual contents of the binary data record.
988
For standard Unix files, the data record
989
typically contains the file attributes or
990
the file data. For a sparse file
991
the first 64 bits of the data contains
992
the storage address for the data block.
994
:=======================================================:
996
|-------------------------------------------------------|
997
| VerNum (uint32_t) |
998
|-------------------------------------------------------|
999
| label_date (float64_t) |
1000
|-------------------------------------------------------|
1001
| label_time (float64_t) |
1002
|-------------------------------------------------------|
1003
| write_date (float64_t) |
1004
|-------------------------------------------------------|
1005
| write_time (float64_t) |
1006
|-------------------------------------------------------|
1007
| VolName (128 bytes) |
1008
|-------------------------------------------------------|
1009
| PrevVolName (128 bytes) |
1010
|-------------------------------------------------------|
1011
| PoolName (128 bytes) |
1012
|-------------------------------------------------------|
1013
| PoolType (128 bytes) |
1014
|-------------------------------------------------------|
1015
| MediaType (128 bytes) |
1016
|-------------------------------------------------------|
1017
| HostName (128 bytes) |
1018
|-------------------------------------------------------|
1019
| LabelProg (32 bytes) |
1020
|-------------------------------------------------------|
1021
| ProgVersion (32 bytes) |
1022
|-------------------------------------------------------|
1023
| ProgDate (32 bytes) |
1024
|-------------------------------------------------------|
1025
:=======================================================:
1027
Id: 32 byte Bacula identifier "Bacula 1.0 immortal\n"
1028
(old version also recognized:)
1029
Id: 32 byte Bacula identifier "Bacula 0.9 mortal\n"
1030
LabelType (Saved in the FileIndex of the Header record).
1031
PRE_LABEL -1 Volume label on unwritten tape
1032
VOL_LABEL -2 Volume label after tape written
1033
EOM_LABEL -3 Label at EOM (not currently implemented)
1034
SOS_LABEL -4 Start of Session label (format given below)
1035
EOS_LABEL -5 End of Session label (format given below)
1036
label_date: Julian day tape labeled
1037
label_time: Julian time tape labeled
1038
write_date: Julian date tape first used (data written)
1039
write_time: Julian time tape first used (data written)
1040
VolName: "Physical" Volume name
1041
PrevVolName: The VolName of the previous tape (if this tape is
1042
a continuation of the previous one).
1045
MediaType: Media Type
1046
HostName: Name of host that is first writing the tape
1047
LabelProg: Name of the program that labeled the tape
1048
ProgVersion: Version of the label program
1049
ProgDate: Date Label program built
1051
:=======================================================:
1053
|-------------------------------------------------------|
1054
| VerNum (uint32_t) |
1055
|-------------------------------------------------------|
1056
| JobId (uint32_t) |
1057
|-------------------------------------------------------|
1058
| *write_date (float64_t) VerNum 10 |
1059
|-------------------------------------------------------|
1060
| *write_time (float64_t) VerNum 10 |
1061
|-------------------------------------------------------|
1062
| PoolName (128 bytes) |
1063
|-------------------------------------------------------|
1064
| PoolType (128 bytes) |
1065
|-------------------------------------------------------|
1066
| JobName (128 bytes) |
1067
|-------------------------------------------------------|
1068
| ClientName (128 bytes) |
1069
|-------------------------------------------------------|
1071
|-------------------------------------------------------|
1072
| FileSetName (128 bytes) |
1073
|-------------------------------------------------------|
1074
| JobType (uint32_t) |
1075
|-------------------------------------------------------|
1076
| JobLevel (uint32_t) |
1077
|-------------------------------------------------------|
1078
| FileSetMD5 (50 bytes) VerNum 11 |
1079
|-------------------------------------------------------|
1080
Additional fields in End Of Session Label
1081
|-------------------------------------------------------|
1082
| JobFiles (uint32_t) |
1083
|-------------------------------------------------------|
1084
| JobBytes (uint32_t) |
1085
|-------------------------------------------------------|
1086
| start_block (uint32_t) |
1087
|-------------------------------------------------------|
1088
| end_block (uint32_t) |
1089
|-------------------------------------------------------|
1090
| start_file (uint32_t) |
1091
|-------------------------------------------------------|
1092
| end_file (uint32_t) |
1093
|-------------------------------------------------------|
1094
| JobErrors (uint32_t) |
1095
|-------------------------------------------------------|
1096
| JobStatus (uint32_t) VerNum 11 |
1097
:=======================================================:
1098
* => fields deprecated
1099
Id: 32 byte Bacula Identifier "Bacula 1.0 immortal\n"
1100
LabelType (in FileIndex field of Header):
1101
EOM_LABEL -3 Label at EOM
1102
SOS_LABEL -4 Start of Session label
1103
EOS_LABEL -5 End of Session label
1106
write_btime: Bacula time/date this tape record written
1107
write_date: Julian date tape this record written - deprecated
1108
write_time: Julian time tape this record written - deprecated.
1111
MediaType: Media Type
1112
ClientName: Name of File daemon or Client writing this session
1113
Not used for EOM_LABEL.
added
removed
developers/mediaformat.tex
