3
MP4V2 LIBRARY INTERNALS
4
=======================
6
This document provides an overview of the interals of the mp4v2 library
7
to aid those who wish to modify and extend it. Before reading this document,
8
I recommend familiarizing yourself with the MP4 (or Quicktime) file format
9
standard and the mp4v2 library API. The API is described in a set of man pages
10
in mpeg4ip/doc/mp4v2, or if you prefer by looking at mp4.h.
12
All the library code is written in C++, however the library API follows uses
13
C calling conventions hence is linkable by both C and C++ programs. The
14
library has been compiled and used on Linux, BSD, Windows, and Mac OS X.
15
Other than libc, the library has no external dependencies, and hence can
16
be used independently of the mpeg4ip package if desired. The library is
17
used for both real-time recording and playback in mpeg4ip, and its runtime
18
performance is up to those tasks. On the IA32 architecture compiled with gcc,
19
the stripped library is approximately 600 KB code and initialized data.
21
It is useful to think of the mp4v2 library as consisting of four layers:
22
infrastructure, file format, generic tracks, and type specific track helpers.
23
A description of each layer follows, from the fundamental to the optional.
29
The infrastructure layer provides basic file I/O, memory allocation,
30
error handling, string utilities, and protected arrays. The source files
31
for this layer are mp4file_io, mp4util, and mp4array.
33
Note that the array classes uses preprocessor macros instead of C++
34
templates. The rationale for this is to increase portability given the
35
sometimes incomplete support by some compilers for templates.
41
The file format layer provides the translation from the on-disk MP4 file
42
format to in-memory C++ structures and back to disk. It is intended
43
to exactly match the MP4 specification in syntax and semantics. It
44
represents the majority of the code.
46
There are three key structures at the file format layer: atoms, properties,
49
Atoms are the primary containers within an mp4 file. They can contain
50
any combination of properties, other atoms, or descriptors.
52
The mp4atom files contain the base class for all the atoms, and provide
53
generic functions that cover most cases. However, each atom has it's own
54
subclass contained in file atom_<name>.cpp, where <name> is the four
55
letter name of the atom defined in the MP4 specification. Typically this
56
atom file just specifies the properties of the atom or the possible child
57
atoms in the case of a container atom. In more specialized cases the atom
58
specific file provides routines to initialize, read, or write the atom.
60
Properties are the atomic pieces of information. The basic types of
61
properties are integers, floats, strings, and byte arrays. For integers
62
and floats there are subclasses that represent the different storage sizes,
63
e.g. 8, 16, 24, 32, and 64 bit integers. For strings, there is 1 property
64
class with a number of options regarding exact storage details, e.g. null
65
terminated, fixed length, counted.
67
For implementation reasons, there are also two special properties, table
68
and descriptor, that are actually containers for groups of properties.
69
I.e by making these containers provide a property interface much code can
70
be written in a generic fashion.
72
The mp4property files contain all the property related classes.
74
Descriptors are containers that derive from the MPEG conventions and use
75
different encoding rules than the atoms derived from the QuickTime file
76
format. This means more use of bitfields and conditional existence with
77
an emphasis on bit efficiency at the cost of encoding/decoding complexity.
78
Descriptors can contain other descriptors and/or properties.
80
The mp4descriptor files contain the generic base class for descriptors.
81
Also the mp4property files have a descriptor wrapper class that allows a
82
descriptor to behave as if it were a property. The specific descriptors
83
are implemented as subclasses of the base class descriptor in manner similar
84
to that of atoms. The descriptors, ocidescriptors, and qosqualifiers files
85
contain these implementations.
87
Each atom/property/descriptor has a name closely related to that in the
88
MP4 specification. The difference being that the mp4v2 library doesn't
89
use '-' or '_' in property names and capitalizes the first letter of each
90
word, e.g. "thisIsAPropertyName". A complete name specifies the complete
91
container path. The names follow the C/C++ syntax for elements and array
96
"moov.trak[2].tkhd.duration"
97
"moov.trak[3].minf.mdia.stbl.stsz[101].sampleSize"
99
Note "*" can be used as a wildcard for an atom name (only). This is most
100
useful when dealing with the stsd atom which contains child atoms with
101
various names, but shared property names.
103
Note that internally when performance matters the code looks up a property
104
by name once, and then stores the returned pointer to the property class.
110
The two entities at this level are the mp4 file as a whole and the tracks
111
which are contained with it. The mp4file and mp4track files contain the
114
The critical work done by this layer is to map the collection of atoms,
115
properties, and descriptors that represent a media track into a useful,
116
and consistent set of operations. For example, reading or writing a media
117
sample of a track is a relatively simple operation from the library API
118
perspective. However there are numerous pieces of information in the mp4
119
file that need to be properly used and updated to do this. This layer
120
handles all those details.
122
Given familiarity with the mp4 spec, the code should be straight-forward.
123
What may not be immediately obvious are the functions to handle chunks of
124
media samples. These exist to allow optimization of the mp4 file layout by
125
reordering the chunks on disk to interleave the media sample chunks of
126
multiple tracks in time order. (See MP4Optimize API doc).
129
Type Specific Track Helpers
130
===========================
132
This specialized code goes beyond the meta-information about tracks in
133
the mp4 file to understanding and manipulating the information in the
134
track samples. There are currently two helpers in the library:
135
the MPEG-4 Systems Helper, and the RTP Hint Track Helper.
137
The MPEG-4 Systems Helper is currently limited to creating the OD, BIFS,
138
and SDP information about a minimal audio/video scene consistent with
139
the Internet Streaming Media Alliance (ISMA) specifications. We will be
140
evaluating how best to generalize the library's helper functions for
141
MPEG-4 Systems without overburdening the implementation. The code for
142
this helper is found in the isma and odcommands files.
144
The RTP Hint Track Helper is more extensive in its support. The hint
145
tracks contain the track packetization information needed to build
146
RTP packets for streaming. The library can construct RTP packets based
147
on the hint track making RTP based servers significantly easier to write.
149
All code related to rtp hint tracks is in the rtphint files. It would also
150
be useful to look at test/mp4broadcaster and mpeg4ip/server/mp4creator for
151
examples of how this part of the library API can be used.
157
The library API is defined and implemented in the mp4 files. The API uses
158
C linkage conventions, and the mp4.h file adapts itself according to whether
159
C or C++ is the compilation mode.
161
All API calls are implemented in mp4.cpp and basically pass thru's to the
162
MP4File member functions. This ensures that the library has internal access
163
to the same functions as available via the API. All the calls in mp4.cpp use
164
C++ try/catch blocks to protect against any runtime errors in the library.
165
Upon error the library will print a diagnostic message if the verbostiy level
166
has MP4_DETAILS_ERROR set, and return a distinguished error value, typically
169
The test and util subdirectories contain useful examples of how to
170
use the library. Also the mp4creator and mp4live programs within
171
mpeg4ip demonstrate more complete usage of the library API.
177
Since mp4 files are fairly complicated, extensive debugging support is
178
built into the library. Multi-level diagnostic messages are available
179
under the control of a verbosity bitmask described in the API.
181
Also the library provides the MP4Dump() call which provides an ASCII
182
version of the mp4 file meta-information. The mp4dump utilitity is a
183
wrapper executable around this function.
185
The mp4extract program is also provided in the utilities directory
186
which is useful for extracting a track from an mp4file and putting the
187
media data back into it's own file. It can also extract each sample of
188
a track into its own file it that is desired.
190
When all else fails, mp4 files are amenable to debugging by direct
191
examination. Since the atom names are four letter ASCII codes finding
192
reference points in a hex dump is feasible. On UNIX, the od command
193
is your friend: "od -t x1z -A x [-j 0xXXXXXX] foo.mp4" will print
194
a hex and ASCII dump, with hex addresses, starting optionally from
195
a specified offset. The library diagnostic messages can provide
196
information on where the library is reading or writing.
202
The coding convention is to use the C++ throw operator whenever an
203
unrecoverable error occurs. This throw is caught at the API layer
204
in mp4.cpp and translated into an error value.
206
Be careful about indices. Internally, we follow the C/C++ convention
207
to use zero-based indices. However the MP4 spec uses one-based indices
208
for things like samples and hence the library API uses this convention.
added
removed
extras/faad2/common/mp4v2/INTERNALS
