1
commit cfabdfe48df2add3d1f30cf4370efd0b31990ab0
2
Author: Sergey Sharybin <sergey.vfx@gmail.com>
3
Date: Thu Dec 20 05:46:53 2012 +0600
5
Assorted fixes for keyframe selection:
7
- Biggest error was in cost functors used for F and H refirement,
10
- Use natural logarithms, since it's actually makes sense from
11
math papers point of view and error is somewhere else.
13
- Disabled rho for GRIC, for now use non-clamped error for tests.
15
- Made SymmetricEpipolarDistance returning non-squared distance
16
Keyframe selection is currently the only used of this function
17
and it seems using non-squared distance makes much more sense.
19
Also would think to append suffix "Squared" to functions which
20
returns squared distances.
22
- Removed templated version of SymmetricEpipolarDistance, since
23
it's not needed actually.
25
This is actually even worse working than previous implementation,
26
but commit it needed for further review.
28
commit 35d8c57626ad74818f155e6e5960c663ea84e032
29
Author: Sergey Sharybin <sergey.vfx@gmail.com>
30
Date: Thu Dec 20 03:00:40 2012 +0600
32
Euclidean resection cost function didn't use correct constructor
34
It was storing a reference to initial rotation passed by value,
35
leading to pointer being pointing to a stack variable, leading to
36
wrong memory access in residuals computing.
38
Apparently was visible in optimized builds only with inline
41
commit 0798d3162bb49cee7e1c423ceccbca1326ad5650
42
Author: Sergey Sharybin <sergey.vfx@gmail.com>
43
Date: Thu Dec 20 02:50:52 2012 +0600
45
Automatic keyframe selection based on Pollefeys's criteria
47
This commit implements automatic keyframe selection algorithm
48
based on Pollefeys's criteria (F-GRIC is smaller than H-GRIC
49
and correspondence ratio is more then 90%).
51
It is implemented as a part of simple pipeline and returns
52
vector of keyframe images for a given Tracks structure.
54
For simple pipeline reconstruction two best keyframes are
55
expected to be selected from all detected candidates.
56
Criteria for this selection could be reprojection error of
57
solution from two candidate keyfames.
59
Unfortunately, it's not fully workable yet, hopefully would
62
commit e943985552f0598ae122252876f305d72c25c2f9
63
Author: Sergey Sharybin <sergey.vfx@gmail.com>
64
Date: Thu Dec 6 17:47:11 2012 +0600
66
Camera Tracking: allow fallback to reprojection resection
69
This fixes some "regressions" introduced in previous commit
70
which lead to much worse solution in some cases. Now it's
71
possible to bring old behavior back.
73
Perhaps it's more like temporal solution for time being smarter
74
solution is found. But finding such a solution isn't so fast,
75
so let's bring manual control over reprojection usage.
77
But anyway, imo it's now nice to have a structure which could
78
be used to pass different settings to the solver.
80
commit 5a23d01dd531d1e0798298d17ba42a3397effb82
81
Author: Keir Mierle <mierle@gmail.com>
82
Date: Thu Sep 20 18:55:44 2012 +0000
84
Make Euclidean resection "always" succeed.
86
The Euclidean resection code had a magical constant, 1e-3, used to
87
compare the results of solving an equation. This failure detection
88
was well-intended, trying to prevent poor solutions from getting
89
made without notifying the caller. Unfortunately in practice, this
90
threshold is too conservative. Furthermore, it is not clear the
91
threshold should exist at all; the purpose of the Euclidean
92
resection is to come up with the best solution it can; other
93
methods (e.g. reprojection error) should be used to compare
94
whether the method succeeded.
96
This commit changes the Euclidean EPnP code to always succeed,
97
causing the previous fallback to projective resection to never
98
run. In most cases, this will result in better reconstructions.
100
This should, in most cases, fix the dreaded "flipping" problem.
102
commit 57dad861d2a7f9d058c6d8edde1a2d51d7225a51
103
Author: Keir Mierle <mierle@gmail.com>
104
Date: Thu Sep 20 02:27:34 2012 +0000
106
Fix variable naming in the planar tracker.
108
commit e9392fd3b46f5668662935696e7d9afac3390ca4
109
Author: Keir Mierle <mierle@gmail.com>
110
Date: Thu Sep 20 02:10:33 2012 +0000
112
Add smarter tolerance checking in the planar tracker.
114
The planar tracker uses Ceres for the refinement stage. During
115
refinement, Ceres iteratively updates the parameters with the
116
latest best guess. If the change in the parameters falls below a
117
threshold, Ceres will abort successfully ("converged").
119
For the case of pure translation tracking, the parameters are
120
exactly the two pixel shifts (dx, dy), and measuring the change in
121
these parameters gives a meaningful termination criterion.
122
However, for all the other parameterizations like affine, where
123
the parameterization involves affine parameters that have no
124
physical interpretation, Ceres is left with no way to terminate
125
the solver early. With the existing code, often many iterations
126
are run long after Ceres has found a solution sufficiently
127
accurate for all tracking needs. No one needs tracking with
128
a quadrillionth of a pixel accuracy; that time is wasted.
130
This patch extends the existing iteration callback that is passed
131
in to Ceres to check if the pattern has fallen out of the search
132
window, to also check if the optimizer has made a tiny step. In
133
particular, if the maximum shift of any patch corner between two
134
successful optimizer steps is less than a threshold (currently
135
0.005 pixels), the track is declared successful and tracking
138
This leads to dramatic speed increases in some cases, with little
139
to no loss in track quality. This is especially apparent when
140
tracking patches with affine or perspective motion models. For
141
example, on some tracking cases I tried, the iterations Ceres took
144
commit 36729c19bf90cb767e9adb96ba7dd48a5ace2be1
145
Author: Keir Mierle <mierle@gmail.com>
146
Date: Wed Sep 19 22:25:02 2012 +0000
148
Detect too-small planar tracking patches.
150
The planar tracker did not detect very skinny patches which have
151
effectively zero area and are untrackable. This adds detection and
152
rejection of patterns with zero area. This fixes a crash found by
153
during Mango production.
155
commit 5cf2bae255a5a0f2e36ea0516670782cb88b589d
156
Author: Sergey Sharybin <sergey.vfx@gmail.com>
157
Date: Thu Dec 6 17:33:53 2012 +0600
159
Real fix for previous commit from Keir. He's comment;
161
Cleanup for when trackers fall out of the search window.
163
Sergey originally left a TODO() here, but his fix is the correct
164
one. I removed the TODO and fixed some comment issues.
166
commit a11533918720e5b43dc1e95895db0eb36c8c06aa
167
Author: Sergey Sharybin <sergey.vfx@gmail.com>
168
Date: Thu Dec 6 17:31:16 2012 +0600
170
Fix crash when tracking in planar motion model (and maybe some other)
172
It was an Abort() caused by check for solver result not equal to USER_ABORT.
174
In some cases solver returns USER_ABORT due to BoundaryCheckingCallback
175
detects coordinates does not belong to image.
177
Somehow this callback wasn't called in previous version of Ceres and
178
in the same case marker was jumping. Now when the callback is called
179
it seems we could simply return failure of tracking without aborting
182
Probably this is in fact some issue somewhere else, would double
183
check with Keir about this.
185
commit 4be2306bcc664b259aaf7068b9f32ab60124a509
186
Author: Sergey Sharybin <sergey.vfx@gmail.com>
187
Date: Thu Dec 6 17:29:39 2012 +0600
189
Resolved some compilation warnings (missed prototypes)
191
In some cases it was missed include of header file, in some other
192
cases symbol could be static.
194
commit bef729ba5c12683d13584d2a728b8b6506b7ca90
195
Author: Sergey Sharybin <sergey.vfx@gmail.com>
196
Date: Thu Dec 6 17:27:17 2012 +0600
198
Code cleanup: silence some -Wnarrowing warnings from C++11
200
commit add1415d896818367087c784a3013dd8f1bb2095
201
Author: Sergey Sharybin <sergey.vfx@gmail.com>
202
Date: Thu Dec 6 17:25:18 2012 +0600
204
Changes to SamplePlanarPatch to support mask input and
205
added output for pattern center.
207
commit daa354c0735b954b0cd7725626e9a3d67416d46b
208
Author: Keir Mierle <mierle@gmail.com>
209
Date: Sat Jun 9 19:22:39 2012 +0000
211
Change libmv's bilinear sampling to assume the same
212
pixel conventions as Blender. This fixes the preview
213
widget in Blender, and should make tracking slightly
216
commit 99b6222873fbfbe248316316956720376a58f438
217
Author: Keir Mierle <mierle@gmail.com>
218
Date: Sat Jun 9 18:58:51 2012 +0000
220
Add new warp regularization scheme for planar tracking.
222
This adds a new term to the tracking cost function that
223
restricts how much the optimizer can warp the patch (as
224
opposed to merely adjusting the translation). This should
225
reduce the "jumpiness" that is sometimes seen when doing
228
It is disabled in this commit; a subsequent commit will add
229
controls to the tracking dialog for this.
231
commit a1c5a70badd11cba0470700bad2eac2b2bd30c86
232
Author: Keir Mierle <mierle@gmail.com>
233
Date: Sat Jun 9 06:55:21 2012 +0000
235
Planar tracker polish.
237
- Fixes the correlation checking code that was broken in the
238
previous commit. The bug was a transpose error.
239
- Fixes a memory leak of the warp functor, found by Sameer.
240
- Various cleanups done at Sameer's suggestion.
242
Thanks to Sameer Agarwal for a code review.
244
commit 2cb784caa854a77cdd43620ab133f26b87ed0d83
245
Author: Keir Mierle <mierle@gmail.com>
246
Date: Fri Jun 8 17:42:17 2012 +0000
248
Make planar tracking much faster.
250
- This makes planar tracking around 2-3x or more faster than
251
before, by rearranging how the sampling is done.
252
Previously, the source patch was sampled repeatedly on
253
every optimizer iteration; this was done for
254
implementation speed, but was wasteful in computation.
256
- This also contains some additions to Ceres to help
257
deailing with mixed numeric / automatic differentation. In
258
particular, there is now a "Chain::Rule" operator that
259
facilitates calling a function that takes Jet arguments,
260
yet does numeric derivatives internally. This is used to
261
mix the numeric differentation of the images with the warp
262
parameters, passed as jets by Ceres to the warp functor.
264
There is also a new "JetOps" object for doing operations
265
on types which may or may not be jets, such as scaling
266
the derivative part only, or extracting the scalar part
269
This patche is aimed at Ceres upstream.
271
- A new function for sampling a patch is now part of the
272
track_region.h API; this will get used to make the preview
273
widget properly show what is getting tracked. Currently
274
the preview widget does not handle perspective tracks.
278
This patch introduces a bug such that the "Minimum
279
Correlation" flag does not work; if it is enabled, tracking
280
aborts immediately. The workaround for now is to disable the
281
correlation checking, and examine your tracks carefully. A
282
fix will get added shortly.
284
commit 81d028f13738ebe2304287dfce90e91bc782e2cf
285
Author: Keir Mierle <mierle@gmail.com>
286
Date: Fri May 18 20:04:43 2012 +0000
288
Remove an unnecessary template<> line in libmv. Convert debug logs to LG.
290
commit 238aaba241ef99995d254aadc974db719da04b96
291
Author: Keir Mierle <mierle@gmail.com>
292
Date: Fri May 18 12:05:10 2012 +0000
294
Support normalization in the tracking prepass
296
The last tracker commit added normalized tracking. This makes
297
tracking patches undergoing uniform illumination change easier.
298
However, the prepass which computes a quick translation-only
299
estimate of the warp did not take this into account. This commit
302
This works reasonably well but in some examples the brute
303
initialization fails. I suspect this is due to the warped template
304
estimate in the current frame being too different from the
305
original, so there are multiple peaks in the normalized-SAD
306
correlation function.
308
The solution is to use the previous frame for the brute
309
initialization and the keyframe for refinement, but that requires
310
architecture changes.
312
commit 981ca4f6a679cd9ac3d086eae3cd946ce72ca8a5
313
Author: Keir Mierle <mierle@gmail.com>
314
Date: Fri May 18 02:12:47 2012 +0000
316
Add light-normalized tracking to the planar tracker
318
This commit adds the ability to normalize patterns by their
319
average value while tracking, to make them invariant to global
320
illumination changes.
322
To see this in action, check out the "Lobby" scene from Hollywood
323
VFX. If you track the markers that are shadowed by the actress,
324
previously they would not track. With the scale adaption on, the
325
tracker would shrink the area to compensate for the changed
326
illumination, losing the track. With "Normalize" turned on, the
327
patch is correctly tracked and scale is maintained.
329
A remaining problem is that only the Ceres cost function is
330
updated to handle the normalization. The brute translation search
331
does not take this into account. Perhaps "Prepass" (see below)
332
should get disabled if normalization is enabled until I fix the
333
prepass to normalize as well.
335
There are a few other changes:
337
- Bail out of the sampling loop early if the mask is zero; this
338
saves expensive samples of the image derivatives.
340
- Fix a bug where the mask was ignored when sampling in the cost
343
commit e9384b15fb2a6a5b81346d5758fa136f0911e945
344
Author: Keir Mierle <mierle@gmail.com>
345
Date: Thu May 17 23:53:32 2012 +0000
347
Implement support for affine tracking in the planar tracker; cleanups.
349
commit 021d41eed8b4ce6a4e37786ccd357ed5dc83a13f
350
Author: Keir Mierle <mierle@gmail.com>
351
Date: Thu May 17 21:26:06 2012 +0000
353
For the planar tracker, initialize the warp from the four correspondences
354
after brute force translation search.
356
commit 003d1bf6145cfd30938b35f6e10d43708dbf916c
357
Author: Sergey Sharybin <sergey.vfx@gmail.com>
358
Date: Thu Dec 6 16:56:01 2012 +0600
360
Correction to region tracker options initialization.
362
Based on patch from Keir to Blender:
363
https://svn.blender.org/svnroot/bf-blender/branches/soc-2011-tomato@46743
365
commit 6af47b218cfdf5219f0ebb3cb95459817cf9abf2
366
Author: Keir Mierle <mierle@gmail.com>
367
Date: Thu May 17 02:31:52 2012 +0000
369
Add new planar tracker features and use the new planar API
371
This commit removes the use of the legacy RegionTracker API from
372
Blender, and replaces it with the new TrackRegion API. This also
373
adds several features to the planar tracker in libmv:
375
- Do a brute-force initialization of tracking similar to "Hybrid"
376
mode in the stable release, but using all floats. This is slower
377
but more accurate. It is still necessary to evaluate if the
378
performance loss is worth it. In particular, this change is
379
necessary to support high bit depth imagery.
381
- Add support for masks over the search window. This is a step
382
towards supporting user-defined tracker masks. The tracker masks
383
will make it easy for users to make a mask for e.g. a ball.
385
- Add Pearson product moment correlation coefficient checking (aka
386
"Correlation" in the UI. This causes tracking failure if the
387
tracked patch is not linearly related to the template.
389
- Add support for warping a few points in addition to the supplied
390
points. This is useful because the tracking code deliberately
391
does not expose the underlying warp representation. Instead,
392
warps are specified in an aparametric way via the correspondences.
394
- Remove the "num_samples_xy" concept and replace it with
395
automatic determination of the number of samples. This makes the
396
API easier for users.
398
- Fix various bugs in the parameterizations.
400
There remains a bug with subpixel precision tracking when in
401
"keyframe" mode; this will get fixed shortly.
403
commit 16a46db104468cec80bd31ca9d5f8bffbe3e003e
404
Author: Keir Mierle <mierle@gmail.com>
405
Date: Mon May 14 12:15:38 2012 +0000
407
"Efficient Second-order Minimization" for the planar tracker
409
This implements the "Efficient Second-order Minimization"
410
scheme, as supported by the existing translation tracker.
411
This increases the amount of per-iteration work, but
412
decreases the number of iterations required to converge and
413
also increases the size of the basin of attraction for the
416
commit 23243b1b1f3e1ab3ef862b47bca06ee876ac2cf4
417
Author: Keir Mierle <mierle@gmail.com>
418
Date: Sun May 13 23:08:56 2012 +0000
420
Add a planar tracking implementation to libmv
422
This adds a new planar tracking implementation to libmv. The
423
tracker is based on Ceres[1], the new nonlinear minimizer that
424
myself and Sameer released from Google as open source. Since
425
the motion model is more involved, the interface is
426
different than the RegionTracker interface used previously
429
The ESM tracker, also known as the KLT tracker in the UI, is
430
temporarily changed to use the new Ceres-based planar
431
tracker in translation-only mode. Currently it is a bit
432
slower than ESM and also doesn't have all the bells and
433
whistles implemented. Those will come soon. Longer term,
434
both trackers will remain since Ceres is unlikely to be as
435
fast as ESM for pure translation solving, due to its
438
The next step is to implement a new tracking UI. The current
439
UI assumes a translational motion model; the new one must
440
support arbitrary perspective transforms of the pattern
443
[1] http://code.google.com/p/ceres-solver
445
commit 52be92b53eb4decb1a316690b162196f227cc441
446
Author: Sergey Sharybin <sergey.vfx@gmail.com>
447
Date: Thu Dec 6 16:06:08 2012 +0600
449
Initial Ceres integration
451
Currently only put sources to src/third_party/ceres and made sure they're
452
not giving compilation issues.
454
Used Ceres upstream version 1.3.0.
456
Needed to make some modifications to it's CMakeLists.txt also to glog and
457
fglags. They're described in README.libmv of this libraries.
461
- Added -fPIC to glog/gflags, so shared ceres library could be linked
462
statically against this libraries.
464
- Tweaked Ceres's build rules to use needed libraries from libmv's
467
commit b13f9d13122e091cb85855c2094386ccdef6e5a4
468
Author: Sergey Sharybin <sergey.vfx@gmail.com>
469
Date: Wed Dec 5 19:05:34 2012 +0600
471
Update Eigen to version 3.1.2
473
Mainly because of lots of warnings generating by gcc-4.7 which are
474
resolved in newer eigen version.
476
commit 1f0dd94e8e37d3fe2df89282ec16a6a685fdde0b
477
Author: Sergey Sharybin <sergey.vfx@gmail.com>
478
Date: Fri May 25 16:36:44 2012 +0600
480
- Added avutil to qt-tracker linking when building with FFmpeg support.
481
On some platforms it seems to be required
482
- Synchronized QT Creator project for qt-tracker with changes in sources,
483
so no it might be compiled from QT Creator.
485
commit b813dbe3f46bbbc7e73ac791d4665622e4fc7ba5
486
Author: Sergey Sharybin <sergey.vfx@gmail.com>
487
Date: Wed May 9 19:01:10 2012 +0600
489
Modal solver: Detect rigid transformation between initial frame and current
490
instead of detecting it between two neighbour frames.
492
This prevents accumulation of error and seems to be working better in footages i've tested.
494
commit 9254621c76daaf239ec1f535e197ca792eea97b6
495
Author: Sergey Sharybin <sergey.vfx@gmail.com>
496
Date: Wed May 9 18:57:00 2012 +0600
498
Backport changes made by Keir in Blender:
500
- Enhance logging in libmv's trackers.
501
- Cleanups in brute_region_tracker.cc.
503
commit d9c56b9d3c63f886d83129ca0ebed1e76d9c93d7
504
Author: Sergey Sharybin <sergey.vfx@gmail.com>
505
Date: Fri Apr 27 16:20:41 2012 +0600
507
Fixes for MinGW64 support by Caleb Joseph with slight modifications by Antony Riakiotakis
509
- Functions snprintf and sincos shouldn't be redefined for MinGW64
510
- Type pid_t shouldn't be re-defined for MinGW64
512
commit e1902b6938676011607ac99986b8b140bdbf090e
513
Author: Sergey Sharybin <sergey.vfx@gmail.com>
514
Date: Fri Apr 27 16:04:19 2012 +0600
516
Fixes for Qt calibration tool
518
- Passing directory with images via command line argument now isn't
519
required -- it there's no such directory specified standard open
520
dialog might be used for this (before application used to abort
521
due to accessing to non-existing list element).
522
- Conversion of source images to grayscale now happens correct.
523
It was needed to build grayscale palette for 8bit indexed buffer.
525
commit 05f1a0a78ad8ff6646d1e8da97e6f7575b891536
526
Author: Sergey Sharybin <sergey.vfx@gmail.com>
527
Date: Sat Apr 14 17:21:29 2012 +0600
529
Make QtTracker compilable again porting it to recent API change and code cleanup:
531
- It was using SAD tracker with own API, now it's using standard RegionTracker API
532
which should make it easier to switch between different trackers.
533
- Restored LaplaceFilter from old SAD module which convolves images with the
534
discrete laplacian operator.
536
commit a44312a7beb2963b8e3bf8015c516d2eff40cc3d
537
Author: Sergey Sharybin <sergey.vfx@gmail.com>
538
Date: Thu Apr 12 13:56:02 2012 +0600
540
Added solver for modal camera motion, currently supports only tripod solving
542
This solver is intended to deal with such camera motions as tripod and panning,
543
where it's impossible to reconstruct exact position of markers in 3d view.
545
It projects markers onto sphere and uses rigid registration of rotation to
546
find rotation angles which makes bundles from previous and current frame be
547
as closest as it's possible.
1
549
commit fa3842e472e3b9c789e47bf6d8f592aa40a84f16
2
550
Author: Sergey Sharybin <sergey.vfx@gmail.com>
3
551
Date: Thu Apr 12 12:32:48 2012 +0600
174
722
SAD tracker now can deal with pattern size any size,
175
723
Very quick implementation came from Blender before Hybrid tracker was added.
176
724
Better to be replaced with brute tracker.
178
commit d547c9cfe37d5d3397d33c8b0e58471e1e1c1634
179
Author: Sergey Sharybin <sergey.vfx@gmail.com>
180
Date: Fri Feb 17 20:03:52 2012 +0600
182
Just convert end of lines to unix style.
184
commit eb73ddbaec5b9e1ad30331bbf858a6ebc266c4aa
185
Author: Sergey Sharybin <sergey.vfx@gmail.com>
186
Date: Fri Feb 17 20:02:20 2012 +0600
188
Made some function static. Resolves possible linking issues when building with MinGW.
190
commit 2930681fafd86e4f4a958054b1db8bfff29623d1
191
Author: Sergey Sharybin <sergey.vfx@gmail.com>
192
Date: Fri Feb 17 19:59:45 2012 +0600
194
Missed this in commit with improvements in camera intrinsics.
196
commit 8d31bc767019b05c5bf8c9f309f9545b3428afa1
197
Author: Sergey Sharybin <sergey.vfx@gmail.com>
198
Date: Fri Feb 17 19:57:51 2012 +0600
200
Another step of syncing codebase with Blender.
201
Mainly fixes for freebsd/osx compilation and aligned memory allocation.
203
commit 3214a2df5bfd98021f25d0f1a626a86318bb245f
204
Author: Sergey Sharybin <sergey.vfx@gmail.com>
205
Date: Fri Feb 17 19:48:02 2012 +0600
207
Support compilation on FreeBSD platform
209
commit 0e5abe96f543687ccfb3a923ec639cb8f45d54f8
210
Author: Sergey Sharybin <sergey.vfx@gmail.com>
211
Date: Fri Feb 17 19:44:18 2012 +0600
213
Implementation of basic system for progress reporting into callee stuff
215
Implemented by using simple callbacks classes which are getting invoked from
216
places where lots of calculation happens, so applications which are using
217
libmv may display nice progress bar.
219
commit c5e18fe35464618055e0e9761be8d22fae56db49
220
Author: Keir Mierle <mierle@gmail.com>
221
Date: Fri Feb 17 19:25:45 2012 +0600
223
Add support for detecting tracking failure in the ESM tracker component of
224
libmv. Since both KLT and Hybrid rely on ESM underneath, KLT and Hybrid now
225
have a minimum correlation setting to match. With this fix, track failures
226
should get detected quicker, with the issue that sometimes the tracker will
227
give up too easily. That is fixable by reducing the required correlation (in
228
the track properties).
230
commit ea0fed736ecdcc8c020227aeef8ef4cd3be5e63d
231
Author: Keir Mierle <mierle@gmail.com>
232
Date: Fri Feb 17 19:23:50 2012 +0600
234
Add a new hybrid region tracker for motion tracking to libmv, and
235
add it as an option (under "Hybrid") in the tracking settings. The
236
region tracker is a combination of brute force tracking for coarse
237
alignment, then refinement with the ESM/KLT algorithm already in
238
libmv that gives excellent subpixel precision (typically 1/50'th
241
This also adds a new "brute force" region tracker which does a
242
brute force search through every pixel position in the destination
243
for the pattern in the first frame. It leverages SSE if available,
244
similar to the SAD tracker, to do this quickly. Currently it does
245
some unnecessary conversions to/from floating point that will get
248
The hybrid tracker glues the two trackers (brute & ESM) together
249
to get an overall better tracker. The algorithm is simple:
251
1. Track from frame 1 to frame 2 with the brute force tracker.
252
This tries every possible pixel position for the pattern from
253
frame 1 in frame 2. The position with the smallest
254
sum-of-absolute-differences is chosen. By definition, this
255
position is only accurate up to 1 pixel or so.
256
2. Using the result from 1, initialize a track with ESM. This does
257
a least-squares fit with subpixel precision.
258
3. If the ESM shift was more than 2 pixels, report failure.
259
4. If the ESM track shifted less than 2 pixels, then the track is
260
good and we're done. The rationale here is that if the
261
refinement stage shifts more than 1 pixel, then the brute force
262
result likely found some random position that's not a good fit.
264
commit a07fff8431621c01d81ae52595d8dd91a295a776
265
Author: Keir Mierle <mierle@gmail.com>
266
Date: Fri Feb 17 19:19:58 2012 +0600
268
Assorted camera tracker improvements
270
- Add support for refining the camera's intrinsic parameters
271
during a solve. Currently, refining supports only the following
272
combinations of intrinsic parameters:
280
This is not the same as autocalibration, since the user must
281
still make a reasonable initial guess about the focal length and
282
other parameters, whereas true autocalibration would eliminate
283
the need for the user specify intrinsic parameters at all.
285
However, the solver works well with only rough guesses for the
286
focal length, so perhaps full autocalibation is not that
289
Adding support for the last two combinations, (f, k1) and (f,
290
k1, k2) required changes to the library libmv depends on for
291
bundle adjustment, SSBA. These changes should get ported
292
upstream not just to libmv but to SSBA as well.
294
- Improved the region of convergence for bundle adjustment by
295
increasing the number of Levenberg-Marquardt iterations from 50
296
to 500. This way, the solver is able to crawl out of the bad
297
local minima it gets stuck in when changing from, for example,
298
bundling k1 and k2 to just k1 and resetting k2 to 0.
300
- Add several new region tracker implementations. A region tracker
301
is a libmv concept, which refers to tracking a template image
302
pattern through frames. The impact to end users is that tracking
303
should "just work better". I am reserving a more detailed
304
writeup, and maybe a paper, for later.
306
- Other libmv tweaks, such as detecting that a tracker is headed
307
outside of the image bounds.
309
This includes several changes made directly to the libmv extern
310
code rather expecting to get those changes through normal libmv
311
channels, because I, the libmv BDFL, decided it was faster to work
312
on libmv directly in Blender, then later reverse-port the libmv
313
changes from Blender back into libmv trunk. The interesting part
314
is that I added a full Levenberg-Marquardt loop to the region
315
tracking code, which should lead to a more stable solutions. I
316
also added a hacky implementation of "Efficient Second-Order
317
Minimization" for tracking, which works nicely. A more detailed
318
quantitative evaluation will follow.
320
commit 0bf66c009d5022eacfc473d247884a73ffeefa8f
321
Author: Sergey Sharybin <sergey.vfx@gmail.com>
322
Date: Fri Feb 17 19:13:49 2012 +0600
324
Rest of compilation fix with FAST library.
326
commit 71b578ca2ba34c528363c514cd1fcc85791d01f3
327
Author: Keir Mierle <mierle@gmail.com>
328
Date: Fri Feb 17 19:00:28 2012 +0600
330
Improve the KLT tracking behaviour and UI
332
- Remove the overly-conservative use of libmv's re-track tracker. The re-track
333
tracker would take a normal tracker such as TRKLT or KLT or pyramid KLT, and
334
track from frame 1 to 2, then back from the position found in 2 back to 1.
335
Then, when the reverse-track doesn't match the original track with high
336
precision, the track is considered "failed". This is a good approach for
337
fully automatic reconstruction, but is too conservative for supervised
340
The retrack-tracker will return when fully automatic tracking is added.
342
- Always solve for (dx, dy) in the TRKLT loop even if the linear system is
343
ill-conditioned. The client (Blender in this case) can still use the solved
344
position, even though it is less reliable.
346
commit 7d8a8762f2bc2e36f95b0b6f4fb4ca996f9f0db7
347
Author: Sergey Sharybin <sergey.vfx@gmail.com>
348
Date: Fri Feb 17 18:46:24 2012 +0600
350
Changes in camera intrinsics distortion/undistortion:
352
- Distortion/undistortion of scaled images wasn't happening right,
353
because camera intrinsics are calibrated on an original frame which
354
has got some particular resolution and trying to apply this model on
355
an image with another resolution gives totally wrong result.
356
This is needed to be able to do post-prccessing of render, running
357
distortion on a scene which might be rendered with higher resolution
358
than footage itself and then be scaled down.
359
- Fixed incorrect calculation/applying of precomputed grid when
360
distortion is high high enough and produces pixel offset higher
361
than 127 pixels. This might be still not very distorted image,
362
but if it's a 4K footage "normal" camera will easily give such
364
- Added support of overscan distortion/undistortion.
366
commit ed080785d63bb8e3a13dde51a2dc94fe59b059bb
367
Author: Sergey Sharybin <sergey.vfx@gmail.com>
368
Date: Fri Feb 17 18:38:51 2012 +0600
370
Fast headers now can be included from C++ sources.
371
Was needed to make it working fine when bundling in Blender but might also
372
be needed to bundle into another applications.
374
commit 5f5a7aa46a2d87b96c8098dfc8682f4d01b5cd40
375
Author: Sergey Sharybin <sergey.vfx@gmail.com>
376
Date: Fri Feb 17 18:36:16 2012 +0600
378
Bring back FAST detector which seems to be working much nicer than Morravec.
379
Both of them are available in API.
381
commit 2cab13c18216fb684b270cec077f7300262584af
382
Author: Sergey Sharybin <sergey.vfx@gmail.com>
383
Date: Fri Feb 17 18:27:36 2012 +0600
385
Revert "Make CameraIntrinsics (and thus Qt tracker) compilable without linking libmv."
387
This reverts commit 81613ee0cc94b315f333c9632b18b95d426aad05.
389
That commit made inverting intrinsics totally unworkable, so reverted this and
390
made needed tweaks to qt-tracker project file to make it compilable (was needed
391
to make it linking together with glog).
395
src/ui/tracker/tracker.cc
396
src/ui/tracker/tracker.pro
398
commit ec46cae041401b17afb4fe4d9c9343d10797090f
399
Author: Sergey Sharybin <sergey.vfx@gmail.com>
400
Date: Fri Feb 17 17:59:55 2012 +0600
402
Fix compilation error using official MinGW
404
commit 6fbc370e922c47cfa35381662b6c439f4891ed74
405
Author: Sergey Sharybin <sergey.vfx@gmail.com>
406
Date: Fri Feb 17 17:38:20 2012 +0600
408
Fix compilation error with MSVC 2010 which is more picky for "missed" STL headers
410
commit be9e6b63691d83b551a085f0766878bd84220767
411
Author: Sergey Sharybin <sergey.vfx@gmail.com>
412
Date: Fri Feb 17 17:36:18 2012 +0600
414
Fix compilation with MSVC where snprintf function is declared as unsafe and _snprintf should be used instead.
416
Better to switch to own implementation will ensure string is correctly NULL-terminated.
418
commit 1847d9e414ed763cd80668775d7d9f79575fc8ca
419
Author: Sergey Sharybin <sergey.vfx@gmail.com>
420
Date: Fri Feb 17 17:34:45 2012 +0600
422
Fix compilation error on OSX caused by incorrect access to ucontext
424
commit 90579b6ffad07672172a1c240499615b30b25549
425
Merge: b9aac30 531c79b
426
Author: Sergey Sharybin <sergey.vfx@gmail.com>
427
Date: Fri Feb 17 18:32:52 2012 +0600
429
Merge remote-tracking branch 'Matthias-Fauconneau/master' into devel
432
src/libmv/tracking/CMakeLists.txt
434
commit b9aac30a9ca6bc8362c09a0e191040964f7c6de2
435
Merge: 198894e 6969e1a
436
Author: Keir Mierle <mierle@gmail.com>
437
Date: Sat Nov 5 17:38:30 2011 -0700
439
Merge pull request #3 from nathanwiegand/master
441
Just a few tiny cleanups
443
commit 6969e1a9534291a982749baa5a3672c97bfa506d
444
Author: Nathan Wiegand <nathanwiegand@gmail.com>
445
Date: Sat Nov 5 14:26:54 2011 -0700
447
I've added cleaned up a few style issues here an there. Also, I've updated the CMakeLists.txt file so that it can build the image_io library. Note, it's only been tested on OSX 10.6
449
commit 4763f851299050140757bfaa069107a0cf639e56
450
Author: Nathan Wiegand <nathanwiegand@gmail.com>
451
Date: Fri Nov 4 23:59:08 2011 -0700
453
Removed a superfulous comment
455
commit a44577c0162e273681e4a9a3cc5f5b37d4315b67
456
Author: Nathan Wiegand <nathanwiegand@gmail.com>
457
Date: Fri Nov 4 23:55:52 2011 -0700
459
Removed a duplicate entry for an author.
461
commit 198894e4c4f51c2c1784ad7c02eb45d2d1ada9bc
462
Merge: c4c67db 6e797d6
463
Author: Keir Mierle <mierle@gmail.com>
464
Date: Fri Nov 4 21:47:05 2011 -0700
466
Merge pull request #2 from nathanwiegand/master
468
CMake changes for OSX
470
commit 6e797d678c4c19f6a9e21657d66183f412cc995b
471
Author: Nathan Wiegand <nathanwiegand@gmail.com>
472
Date: Fri Nov 4 21:43:28 2011 -0700
474
Uncomment the GUI part of the CMake file
476
commit 33ef88a33860345d8906f3c9dd22d8dbce3df53e
477
Author: Nathan Wiegand <nathanwiegand@gmail.com>
478
Date: Fri Nov 4 21:31:22 2011 -0700
480
Fixed build error on OSX by adding 'glog' to the dependencies in the tracker CMake
482
commit 531c79bf95fddaaa70707d1abcd4fdafda16bbf0
483
Author: Matthias Fauconneau <matthias.fauconneau@gmail.com>
484
Date: Sat Aug 20 00:00:42 2011 +0200
486
Display warped pattern in marker preview.
488
commit bb5c27e671b6f8eb56ddf490f0795d59bede591b
489
Author: Matthias Fauconneau <matthias.fauconneau@gmail.com>
490
Date: Fri Aug 19 18:37:48 2011 +0200
494
commit 2ac7281ff6b9545b425dd84fb03bf9c5c98b4de2
495
Author: Matthias Fauconneau <matthias.fauconneau@gmail.com>
496
Date: Fri Aug 19 17:34:45 2011 +0200
498
Avoid symbol shadowing.
500
commit 2a7c3de4acc60e0433b4952f69e30528dbafe0d2
501
Author: Matthias Fauconneau <matthias.fauconneau@gmail.com>
502
Date: Fri Aug 19 17:22:47 2011 +0200
504
Better dragging behavior when hitting borders.
506
commit a14eb3953c9521b2e08ff9ddd45b33ff1f8aeafb
507
Author: Matthias Fauconneau <matthias.fauconneau@gmail.com>
508
Date: Fri Aug 19 17:12:12 2011 +0200
510
Update marker preview to new affine tracking.
512
commit 5299ea67043459eda147950e589c2d327a8fbced
513
Author: Matthias Fauconneau <matthias.fauconneau@gmail.com>
514
Date: Fri Aug 19 16:05:54 2011 +0200
516
sqrt takes double precision.
518
commit 9f9221ce151d788c49b48f6f293ab2e2f8813978
519
Author: Matthias Fauconneau <matthias.fauconneau@gmail.com>
520
Date: Fri Aug 19 16:04:37 2011 +0200
522
MSVC compatibility: heap allocate pattern, explicit float cast.
524
commit 702658d2f8616964a6eeb3743fd85e97ac7ff09d
525
Author: Matthias Fauconneau <matthias.fauconneau@gmail.com>
526
Date: Fri Aug 19 14:59:24 2011 +0200
528
Expose regularization parameters (areaPenalty and conditionPenalty) in API.
added
removed
extern/libmv/ChangeLog
