Introduction ------------ This document describes how the gestures are extracted from multi-finger actions. The process is divided into gestural transformations, gesture recognition, and gesture instantiation. Gestural Transformations ------------------------ All two-finger transformations are extracted. These are all exact, in the sense that continuously transforming the original finger positions, frame per frame, will exactly follow the actual finger positions. In addition to two-finger transformations, a global gesture is also extracted. The rotation and scaling is taken from the contact pair with the longest distance between contacts. This approximates the behavior of a region, such that complex transformations could, in principle, be happening inside the region, but at a distance, the transformation will look like it was performed with two fingers. At each time step, a gestural transformation is goverened by rotation, scaling and translation. The point around which rotation and scaling is performed is called the pivot. To form as natural gestures as possible, this point always lies within the area formed by the contacts themselves. It is placed at the point which, after rotation and scaling, leaves the transformed contacts as close to the actual positions as possible. The translation, or drag, is always a fraction of the movement of the center point. The fraction is called moveness, and is related to the distance between the pivot and the center point. When the pivot is at the center point, the moveness is one and the drag is the same as the center movement. When the pivot is at one of the contacts, as in rotation around a finger, the moveness is zero, and consequently the drag is zero. The relations between the pivot, the moveness and the drag are detailed in the document pivot.txt. Gesture Recognition ------------------- Each gestural transformation can give rise to one or several gesture primitives. Based on the rotation, scaling, moveness and drag values, the gesture primitives drag, pinch and rotate are recognized. The onset of a gesture primitive is governed by a threshold and a timeout. Basically, if performed distinctly enough, the gesture will be triggered, and will remain active until a finger is lifted or added. Gesture Instantiation --------------------- Given a set of detected gesture primitives, only some will trigger actual gesture events. First, the set of available primitives are matched against available listeners. Primitives not listened for are dropped. The remaining set is arranged according to priority. A pointer gesture has lower priority than a two-finger gesture, which has lower priority than a tap, for instance. As long as the gesture primitives of higher priority are expected but not activated, all gestures are held back. Once one of the highest priority primitives are activated, all primitives of lower priority are cancelled, and events are emitted.