1
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
4
<title>LibRaw: General Notes on API</title>
6
<meta http-equiv="content-type" content="text/html; charset=windows-1251">
9
<a href=index-eng.html>[back to Index]</a>
10
<h1>LibRaw: General Notes on API</h1>
14
<li><a href="#versions">LibRaw editions</a></li>
15
<li><a href="#errors">Error Code Conventions and Error Handling</a></li>
16
<li><a href="#warnings">Nonstandard Situations That Are Not Errors</a></li>
17
<li><a href="#io">Input Layer Abstraction</a></li>
18
<li><a href="threads">Thread Safety</a></li>
19
<li><a href="#CXX">The Use of C++</a></li>
20
<li><a href="#imgdata_params">Parameters of the LibRaw::imgdata.params Structure Affecting the Behavior of
21
open_file/unpack/unpack_thumb</a></li>
22
<li><a href="#filtering">RAW-data filtering</a></li>
23
<li><a href="#masked-border">Masked pixels storage</a></li>
24
<li><a href="#memory">Memory Usage</a>
26
<li><a href="#stack">Stack Usage</a></li>
27
<li><a href="#memmgr">Dynamic Memory Management</a></li>
28
<li><a href="#memuse">Dynamic Memory Usage</a>
30
<li><a href="#memimage">Memory for the Decoded Image</a></li>
31
<li><a href="#memthumb">Memory for the Decoded Thumbnail</a></li>
32
<li><a href="#memprofile">Memory for the Decoded ICC Profile</a></li>
33
<li><a name="#memraw">Memory for RAW Unpacking</a></li>
34
<li><a href="#mempostproces">Memory for Postprocessing</a></li>
35
<li><a href="#memwrite">Memory for File Writing</a></li>
36
<li><a href="#memunpack">Unpacking into memory buffer</a></li>
41
<li><a href="#incompat">Incompatibilities with dcraw</a>
43
<li><a href="#incompat_kodak">Processing of Thumbnails from Kodak cameras</a></li>
49
<h2>LibRaw Versions</h2>
50
<p>Since version 0.9, there is only one LibRaw variants. Older versions have three separate editions (normal,
51
-Lite and -Commercial versions).
55
<h2>Error Code Conventions and Error Handling</h2>
57
The following conventions concern the returned errors:
60
<li>All functions that can return an error code have integer type of return data.</li>
61
<li>If there is no error, the return value is 0 (LIBRAW_SUCCESS).</li>
62
<li>If an error has happened in a system call, the return value is errno
63
(a positive number), which can be analyzed using strerror() or similar means.</li>
64
<li>All LibRaw's own error codes are negative; each of these errors belongs to one of two types:
66
<dt><b>Non-fatal errors</b></dt>
68
Non-fatal errors do not forbid execution of other functions in the processing succession
69
(e.g., <a href="API-CXX-eng.html#unpack_thumb">unpack_thumb()</a> can easily return the code corresponding to "preview is absent"
70
but this does not prevent further call of <a href="API-CXX-eng.html#unpack">unpack()</a>.
72
<dt><b>Fatal errors</b></dt>
74
In the case of fatal errors (memory shortage, input data error, data unpacking failure), the current stage of processing
75
is terminated and all allocated resouces are freed.<br/>
76
If an attempt to continue processing is made, all subsequent API calls will return the LIBRAW_OUT_OF_ORDER_CALL error.<br/>
77
At the same time, the LibRaw instance in which a fatal error has occurred can process the next RAW
78
files in the usual way (by calling <a href="API-CXX-eng.html#open_file">open_file()</a> (or other input methods), then <a href="API-CXX-eng.html#unpack">unpack()</a>,
82
<li>The macro LIBRAW_FATAL_ERROR(error code) checks if an error is fatal or not.</li>
83
<li>The error codes are <a href="API-datastruct-eng.html#errors">listed and deciphered here</a>.</li>
87
<h2>Nonstandard Situations That Are Not Errors</h2>
88
<p>If the program has encountered a nonstandard situation that does not prevent retrieval of some data
89
from a file, it sends a signal by setting the corresponding bit in <a href="API-datastruct-eng.html#libraw_data_t">imgdata.process_warnings</a>.
90
The possible types of warnings are <a href="API-datastruct-eng.html#warnings">listed and deciphered here</a>.
94
<h2>Input Layer Abstraction</h2>
96
LibRaw uses objects derived from
97
<a href="API-CXX-eng.html#datastream">LibRaw_abstract_datastream</a> for data input.
98
Semantics of these objects is similar to 'file with arbitrary seek' object: both read and seek
102
Some RAW formats requires temporary switch to another data stream created on top on memory buffer for metadata
103
read. Methods for doing so are implemented in base class
104
<a href="API-CXX-eng.html#datastream">LibRaw_abstract_datastream</a> by internal data field <b>substream</b>.
105
Look into source code of <a href="API-CXX-eng.html#file_datastream">LibRaw_file_datastream</a> class
106
in <b>libraw/libraw_datastream.h</b> file for more details.
108
When implementing own datastream classes, you need to take <b>substream</b> into account and pass control to
109
methods of this field if it is active (not NULL).
112
If datastream implementaton knows name of input file, it should provide fname() call. This name will be used
113
in <a href="API-CXX-eng.html#callbacks">error callbacks</a> and in guessing name of JPEG file with metadata
114
(for RAW files with external metadata).
117
For external metadata support input class should implement
118
<b>subfile_open()/subfile_close()</b> methods.
119
���������� ��� ������.
121
Sample of these methods implementation may be found in
122
<a href="API-CXX-eng.html#file_datastream">LibRaw_file_datastream</a> class (look into
123
<b>libraw/libraw_datastream.h</b> file for details).
126
<a name="threads"></a>
127
<h2>Thread safety</h2>
129
Thread safety is ensured if a LibRaw object is created and used within one thread. At the same time, the number
130
of threads (each with its own LibRaw object) is not limited in any way (except by memory requirements).
133
If a LibRaw object is created in one execution thread and used in another, external synchronization is
137
There is two libraries under Unix enviroment (Linux/FreeBSD/MacOS): libraw_r.a (thread-safe) and libraw.a
138
(single-threaded, slightly faster).
141
Thread-safe library version stores intermediate unpacker data into LibRaw class data. So, several copies of
142
LibRaw, working in parallel, is possible.
145
Not thread-safe library uses global variable for intermediate data store which is faster but not reenterant.
146
This library may be used in multi-threaded apps, but only if exactly one LibRaw class copy exists in program.
149
Windows version is similar to multi-threaded Unix one.
152
<h2>The Use of C++</h2>
154
Exception situations within LibRaw are handled using the C++ exception mechanism. All exceptions are caught inside
155
the library functions and should not penetrate outside.
158
Memory is allocated/freed using functions malloc(calloc)/free rather than new/delete.
161
No specific libraries (STL, Boost, smart pointers) are used. </p>
163
If C API is used, references to C++ calls new/delete still remain, and so linking with libstdc++(Unix)/....(Windows) is necessary.
165
<a name="imgdata_params"></a>
166
<h2>Parameters of the LibRaw::imgdata.params Structure Affecting the Behavior of open_file/unpack/unpack_thumb</h2>
168
Most data fields of structure LibRaw::imgdata.params affect only <a href="API-CXX-eng.html#dcrawemu">data
169
postprocessing</a>, but there are some exceptions, which have been inherited by the current version of LibRaw
170
from/ dcraw source texts (these dependences will be gradually removed).
172
<dt><b>imgdata.params.use_camera_matrix and imgdata.params.use_camera_wb</b></dt>
174
These fields affect loading of RAW data for cameras with a color matrix.<br/>
175
<b>Attention!</b> If parameter <b>imgdata.params.use_camera_matrix</b> is not set by the user, it is copied from
176
<b>imgdata.params.use_camera_wb</b> at the stage of file opening.
178
<dt><b>imgdata.params.user_flip</b></dt>
180
If this parameter is greater than or equal to zero, assignment <code>imgdata.sizes.flip = imgdata.params.user_flip</code> is
181
performed at the <a href="API-CXX-eng.html#open_file">open_file()</a> stage.
183
<dt><b>imgdata.params.shot_select</b></dt>
185
This parameter makes it possible to select the number of the extracted image for data formats in which storage
186
of several RAW images in one data file is possible.
188
<dt><b>imgdata.params.half_size</b></dt>
190
Affects RAW data loading for Phase One and Sinar backs. Also, it this parameter is set then image bitmap
191
will be reduced by half in each dimension. In later case, all 4 components of bitmap will be filled during
192
data extraction phase.
194
<dt><b>imgdata.params.threshold, imgdata.params.aber</b></dt>
196
If these parameters used, then half-sized bitmap will be used for data unpacking. See above for details.
199
<dt><b>imgdata.params.filtering_mode</b></dt>
201
Affects RAW data loading for cameras with black level subtraction/de-banding during RAW read phase:
202
Canon A600, A5, CRW/CR2; cameras with LJPEG-compression (some Nikon, Kodak and Hasselblad cameras).
204
<dt><b>imgdata.params.use_camera_wb</b></dt>
206
Affects loading of white balance matrix for Leaf backs.
208
<dt><b>imgdata.params.document_mode</b></dt>
210
<b>Does not affect data reading in the current version of LibRaw</b>. In dcraw, this parameter affects the thumbnail data for certain Kodak cameras.
214
<a name="filtering"></a>
215
<h2>RAW-data filtration</h2>
217
During RAW unpacking and post-process stages LibRaw can filter RAW data:
220
<li>Black level subtraction (required if not already done by camera firmware).</li>
221
<li>Zero pixels removal (averaging with nearest pixel values).</li>
222
<li>Tone curve operations on RAW-data.</li>
226
Current LibRaw version allows tuning of filtration process by setting <b>imgdata.params.filtering_mode</b>
227
option to one of <a href="API-datastruct-eng.html#LibRaw_filtering">enum LibRaw_filtering</a> vales:
230
<li>Dcraw-compatible filtration. This mode is default for old LibRaw versions compatibility. This mode
231
is not recommended for new applications.</li>
232
<li>One or several (or all) filtering stages can be turned off. This mode recommended for
233
RAW analysers and advanced data filtration made by calling program (this programs may
234
calculate black level by analyzing <a href=#masked-border>masked frame data</a>.</li>
235
<li>Automatic selection of best filtration avaliable in LibRaw (e.g. camera/model specific
236
routines). This mode is recommended for most applications.
239
<a name="masked-border"></a>
240
<h2>Masked pixels storage</h2>
242
Some of RAW images contains data for not-active ("black","masked") pixels. For some of these formats
243
LibRaw can extract these pixels values and store it in
244
<a href="API-datastruct-eng.html#libraw_masked_t">imgdata.masked_pixels</a> data structure.
245
These values can be explored for:
248
<li>Black level calculation (with banding suppression on some cameras).</li>
249
<li>Noise level calculation, e.g. per-channel noise. This is useful for cameras with per-channel
252
<p>Masked pixel data avaliable only for bayer-pattern data (one component per pixel) and only for cameras
253
with masked frame. Masked pixels extraction for other color models (Canon sRAW, Kodak YRGB, Sinar 4-shot
254
files) will possibly be added in future LibRaw versions.</p>
256
LibRaw provides interface for merging active pixel data and masked pixels into one bitmap:
257
<a href="API-CXX-eng.html#add_masked_borders_to_bitmap">add_masked_borders_to_bitmap()</a>.
258
This call is useful for RAW analyzers.
261
<a name="memory"></a>
262
<h2>Memory Usage</h2>
266
An instance of the LibRaw class has its own size about <b>100 Kb</b>; if constructions like <code>LibRaw
267
imageProcessor;</code> are used, this memory is stack-allocated.
270
Methods of class LibRaw (and C API calls) may allocate up to 130-140 Kb of data on the stack (to place auto
271
variables) during their work.
273
<p>Thus, the work of one LibRaw instance may require about 250 Kb of stack memory. This is not a problem for most
274
contemporary architectures. However, when working in a multithreaded environment, one should not forget to
275
allocate a sufficient amount of memory for the thread stack.
277
<p>In the case of dynamic allocation (<code>LibRaw *iProcessor = new LibRaw;</code>), the requirements to stack
278
memory will decrease by 100 Kb, which is the size of a class instance). If <a href="API-C-eng.html">C API</a> is
279
used, the LibRaw instance is allocated dynamically.
282
<a name="memmgr"></a>
283
<h3>Dynamic Memory Management</h3>
284
<p>LibRaw keeps record of all allocated dynamic memory blocks; in the case of an exceptional situation (fatal
285
error), they are all freed. The code for keeping this record is fairly primitive and not designed to consider
286
allocation of many blocks (in the normal situation, allocation takes place from 2 to 6 times during file
287
processing); this fact should be taken into account by developers trying to add new methods to LibRaw.
289
<a name="memuse"></a>
290
<h3>Dynamic Memory Usage</h3>
291
<p>LibRaw uses dynamic memory</p>
293
<li>for the decoded image;</li>
294
<li>for the decoded thumbnail;</li>
295
<li>for the ICC profile retrieved from the RAW file (if available);</li>
296
<li>for temporary data at the stage of RAW file unpacking;</li>
297
<li>for temporary data at the stage of postprocessing and result output;</li>
298
<li>for reading of the RAW source file (only under Win32).</li>
300
<a name="memimage"></a>
301
<h4>Memory for the Decoded Image</h4>
303
To simplify further processing, memory for the extracted RAW data is allocated with a fourfold (for Bayer sensor
304
cameras) excess: <b>for each pixel, four 16-bit components are available</b> (three of them will be zero after
305
RAW unpacking). Thus, one can perform debayer and other postprocessing actions directly in the same buffer as
306
the one used for data extraction, but the required amount of memory becomes four times higher. <br/>
307
<b>Hence, the size of memory for the image buffer is 6-10 times greater than the size of the source RAW
308
file.</b><br/> <i>It is quite likely that allocation of this buffer in next versions of LibRaw will be more
309
economical, under the condition that postprocessing calls inherited from dcraw will not be used.</i>.
311
<p>The buffer for the decoded image is allocated upon calling <a href="API-CXX-eng.html#unpack">unpack()</a> and
312
freed upon calling <a href="API-CXX-eng.html#recycle">recycle()</a>.
314
<a name="memthumb"></a>
315
<h4>Memory for the Decoded Thumbnail</h4>
317
Memory for the thumbmail is allocated upon calling <a href="API-CXX-eng.html#unpack_thumb">unpack_thumb()</a> and freed upon
318
calling <a href="API-CXX-eng.html#recycle">recycle()</a>. The size of the allocated buffer is precisely adjusted to the thumbnail size,
319
i.e., up to several Mb.
322
<a name="memprofile"></a>
323
<h4>Memory for the Decoded ICC Profile</h4>
325
Memory for the ICC profile is allocated upon calling <a href="API-CXX-eng.html#unpack_profile">unpack_profile()</a> and freed upon
326
calling <a href="API-CXX-eng.html#recycle">recycle()</a>. The size of the allocated buffer is precisely adjusted to the ICC profile size,
327
i.e., up to several hundred Kb.
329
<a name="memraw"></a>
330
<h4>Memory for RAW Unpacking</h4>
332
Memory for temporary buffer needed during RAW data unpacking may be allocated during the work of <a
333
href="API-CXX-eng.html#unpack">unpack()</a> and freed before completion of this function. The sizes of the allocated
334
buffers are small, up to tens of Kb.
337
<a name="mempostproces"></a>
338
<h4>Memory for Postprocessing</h4>
339
<p>During image postprocessing (inherited from dcraw), memory for the histogram (128 Kb) is allocated.
340
This memory is allocated upon calling <a
341
href="API-CXX-eng.html#dcraw_document_mode_processing">dcraw_document_mode_processing()</a> and
342
<a href="API-CXX-eng.html#dcraw_process">dcraw_process()</a> and freed upon calling
343
<a href="API-CXX-eng.html#recycle">recycle()</a>.
346
In addition, during the work of <a href="API-CXX-eng.html#dcraw_process">dcraw_process()</a> and during the
347
usage of some available possibilities, like
350
<li>rotation of images from FUJI cameras;</li>
351
<li>correction of chromatic aberrations;</li>
352
<li>image size changes (including correction of non-square pixels);</li>
353
<li>highlight recovery;</li>
356
a temporary buffer with the size equal to the size of the resultant image (6-8 bytes per pixel for various processing stages)
357
will be allocated. As soon as the intermediate substage of processing is completed, the buffer with the previous copy
358
of the image will be freed.<br/>
359
If postprocessing is not used, then temporary buffers are not allocated.
362
<a name="memwrite"></a>
363
<h4>Memory for File Writing</h4>
365
Upon calling <a href="API-CXX-eng.html#dcraw_ppm_tiff_writer">dcraw_ppm_tiff_writer()</a>, memory for a single row of the
366
output image is allocated. The allocated memory is freed before the end of this call.
368
<a name="memunpack"></a>
369
<h4>Unpacking into memory buffer</h4>
371
Functons <a href="API-CXX-eng.html#dcraw_make_mem_image">dcraw_make_mem_image()</a> �
372
<a href="API-CXX-eng.html#dcraw_make_mem_thumb">dcraw_make_mem_thumb()</a> (and complementary calls in C-API)
373
allocates memory for entire output datasets (full RGB bitmap and thumbnail, respectively).
374
<b>Calling function should free() this memory themself.</b>
378
<a name="incompat"></a>
379
<h2>Incompatibilities with dcraw</h2>
380
<a name="incompat_kodak"></a>
381
<h3>Processing of Thumbnails from Kodak cameras</h3>
382
<p>In some Kodak cameras, the preview (thumbnail) is stored in the form of uncorrected image. During its extraction using
383
<b>dcraw -e</b>, the white balance, color conversion, and other settings are the same as those used for extraction of the main RAW data
384
(including defect removal and dark frame subtraction, which is erroneous, since the image size is different).
386
In LibRaw::unpack_thumb() calls, the white balance taken from the camera ("as shot") is used and no settings from
387
imgdata.params are considered.
390
For all other cameras, thumbnails are extracted "as is," without any color conversions, both in dcraw and in LibRaw.
393
<a href=index-eng.html>[back to Index]</a>
395
<address><a href="mailto:info@libraw.org">LibRaw Team</a></address>
396
<!-- Created: Sun Mar 16 10:05:08 MSK 2008 -->
398
Last modified: Sun Mar 28 22:08:30 MSD 2010