Streaming model transformations over Jnect
The related sources are available at these locations:
In the case study, a human body is observed by optical sensors. The stream of data from the sensors (Microsoft Kinect in our case) carries the spatial position of the hands, wrists, knees, etc. This stream is continuously processed and its data is stored in a live model, technically, an EMF model maintained via a Java based API jnect. Every time the optical sensors capture a new frame, the model is updated with the appropriate spatial data.
The sensors process 25 frames per second, resulting in 25 model update transactions each second. The complexity of the scenario arises from the frequent changes the model undergoes. Executing model transformations on such a model poses several problems, since it would become obsolete quickly after being loaded into the memory. Moreover, model update transactions affect multiple model elements.
We aim at recognizing a gesture in order to control a PowerPoint presentation with it. On the recognized gesture, the presentation advances to the next slide, therefore the gesture is referred to as the forward gesture. In our previous presentation there is also a backward gesture to move back to the previous slide.
As illustrated above, the forward gesture consists of two postures: the forward start and the forward end. To recognize the gesture, the series of these two postures needs to be identified. Postures are considered as certain *states of the body, which are described with a range or interval of spatial data. For example, the forward start posture is defined by the right arm being approximately stretched roughly to the height of the shoulder. Determining whether the arm is stretched is achieved by continuously measuring the angle between the upper and lower arm and smoothing the resulting stream of spatial data by a moving average transformation.