This section provides you with some background knowledge about the internal functionality of wiigee and may help you to understand it's implementation and, consequently, some of it's behaviours. To some extent, it can be regarded as a stripped-down version of the paper Gesture Recognition with a Wii Controller and is not intended to serve as a programming guide. For this purpose, consult the basic tutorial as well as the JavaDoc.
What is a gesture?
In order to be able to train and recognize arbitrary gestures, we have to define what (from a technical point of view) a gesture actually is. The term is typically associated with a characteristic pattern of incoming signal data which, in the case of the wiimote, refers to the data the built-in accelerometer is constantly sending via Bluetooth. An accelerometer is a small device detecting the acceleration in all three dimensions an external force exerts on it. With these comments in mind, gesture describes an ordered, finite series of three-dimensional acceleration vectors with an explicit start and an explicit end. In wiigee, start and end of a gesture are by default triggered by pressing one of the wiimote's buttons although this behaviour can be altered. (See Recognition.)
Gesture examples. Although these are all 2D,
wiigee is capable of detecting 3D-gestures
just as well.
System components
wiigee is built upon a classic recognition
pipeline which was specifically optimized for the wiimote's
hardware. The following image shows the pipeline and thus
the way the vector data takes during processing.
wiigee system components
The first component is a Filter, removing vectors
which are a) below a given threshold of approx.
1.2g, g being the acceleration of
gravity, and b) roughly equivalent to their predecessor,
thus eliminating suspected duplicates. The second component
is a Quantizer clustering the incoming vector data
in order to generate a discrete code identifying each
gesture. Here, a standard k-mean algorithm is
utilized, k being the number of clusters or
letters in our code-alphabet. In the case of the wiimote,
we identified k = 14 to be a sufficient size for
the code-alphabet, the cluster centers carefully placed on
a three-dimensional sphere with a dynamic radius. From the
theory of speech recognition we picked a discrete,
left-to-right hidden markov model as the third component,
the Model, and further optimized it for the
wiimote. It has proven to deliver reliable results for
patterns with spatial as well as temporal variation which
fits the needs of our accelerometer-based gestures almost
ideally. After this third component, a gesture (or rather:
a series of codes) is mapped to a model probability. This
model probabilty is then classified by the fourth
component, a traditional Bayes-classifier, which
identifies the most probable gesture during a recognition
task.
Workflow
Since the wiigee library aims at not
merely recognizing gestures but allowing you to define your
own, it distinguishes two modes: training as well
as recognition.
wiigee workflow
During training you (or your users) record the
gestures your application later takes as input commands.
For this purpose, each intended gesture is to be performed
several times. This allows wiigee to learn
the gesture and internally generate a code for it
(represented by SQUARE in the above
illustration). During recognition
wiigee tries to identify the gesture which
has just occured by computing which of the trained gestures
fits the performed one most probably and reports this. You,
as a developer, may then trigger a desired function
accordingly. The following two subsections further describe
the two modes.
Training
Let's assume you design an application in which you want
the user to draw a circle into the air. Since different
users perform such a circle in different ways (take
n users and you get n circles), you have
to make sure that your system recognizes circles in a
reliable as well as robust way. Thus, telling your system
only once what a circle is will not be sufficient. You have
to train it repeatedly to get reliable results for
a great range of users.
In wiigee, training is a recording process
triggered by a specific TrainButton. This
TrainButton must be held down during recording.
Releasing it marks the end of the recording process.
Repeating this whole procedure further trains the system
and, by that, makes it more likely that a gesture is
correctly identified during the later phase of recognition.
We found that 5 to 10 training sessions are a necessary
minimum to get feasible results but recommend 10 to 15.
Once you are satisfied with the number of training
sessions, pressing an explicit CloseGestureButton
concludes the training phase.
Note that at this moment wiigee does not
provide it’s own methods to store training data into a file
or to load such stored data. This is a feature which will
come with a later release.
Recognition
Now that your desired gestures have been recorded,
wiigee internally sets up it's parameters
for the recognition process. Your are now able to perform
your newly recorded gestures. By default, a specific
RecognitionButton is to be pressed and held down
during your or your user's gesture performance. After you
released this RecognitionButton
wiigee tries to identify the gesture and
fires a GestureEvent containing information
about the detected gesture along with the calculated
probability.
Alternatively, you may bind start and end of a gesture not
to a button but to the wiimote’s motion. You may do so by
setting the RecognitionButton to the “virtual
button” Wiimote.MOTION and then tune it’s
sensibility via the method setMotionChangeTime(int
milliseconds). However, this approach is still
experimental and we recommend the default method.
Starting the engine
To make your wiimote(s) discoverable during the execution
of your application you have to press an initial button
combination on each device: press the wiimote buttons
and 
at once. This way, connections to your
wiimote(s) are established and you or your users may
start performing a training or a recognition movement.
Your application is now able to perform gesture
recognition.