Diploma Thesis

Optimizing Collision Handling for Natural Hand Interaction in Immersive Virtual Environemts

• Student: Michael Stengel
• Period of time: August 2010 - December 2010
• Defence: February 10th, 2011

Motivation

In the product development process an enterprise always attempts making decisions about the product as soon as possible. To do this even before producing real prototypes, virtual prototypes are used. It is necessary that these virtual prototypes can be handled in the same way as the real prototypes. Therefore, an example is the assembly simulation in automotive areas. In this application natural interaction in immersive environments is useful, which is a very demanding and complex task.

In reality lots of steps in the assembly are done by hand. Nowadays modern tracking systems are able to transfer your precise hand and finger positions into the computer. This enables manipulation as complex as needed. Additionally the virtual scene has to react on this input with highly realistic object motion. This can be done only by the use of real-time physics simulation.

Criteria for Natural Interaction

For natural hand interaction, the following criteria are especially important:

Naturalness
The interaction with the scene should be as natural as possible. The user wants to interact using both hands and he wants to interact with an arbitrary number of objects. Additionally all objects should move in a plausible manner.

Robustness
The interaction has to be stable and precise to enable the user to grasp and manipulate objects as wanted.

Real-time Requirement
The system has to work in real-time, so that the user gets feedback on his actions in time.

Object Penetration

There are some approaches for natural interaction already, but most of them are not able to deal with penetrations between the virtual hand of the user and the objects in the scene. Penetrations are no problem as long as they do not disturb the visual impression or interaction. A problem evolves, if it comes to deep penetrations, which lead to unwanted visual artifacts and an unstable simulation.

Deep penetrations emerge out of the fact that the real hands of the user are not constrained in the immersive environment. So the user can move his hands into improper positions, where there are objects in the virtual world. If the simulation systems tries to move the virtual hands to the tracking targets, large forces are generated, pushing the virtual hands into the scene objects. Another reason is the basic process of a real-time physics simulation. The simulation calculates the physics world stepwise in discrete timesteps and so just considers "snapshots" of it. So, it‘s necessary to develop a simulation system which can deal with these issues.

Diploma Project

In this thesis a solution for the object penetration problem is given. The approach contains a combination of two different methods of physical simulation, to create a system, which enables hand interaction, runs in real-time and avoids deep penetrations simultaneously.

An analysis has shown that deep penetrations between objects can be avoided even with available physics libraries, when choosing the simulation parameters properly. These libraries are highly optimized for simulating a lot of objects interacting with each other. The effective forces are therefore relatively weak. That’s why the interaction between virtual objects are simulated by some fast physics library in this approach.

In contrast, forces between the virtual hands and scene object can be very large. With an ordinary physics library his would lead to an unstable simulation as described before. For simulating these kind of contacts a new approach and way of connecting both simulators was developed.

Simulation Approach

How can virtual hands be simulated without penetration? Considering penetrations there is some related work in the fields of Haptic Interaction, where the user is able to feel collisions with the virtual world. One of those approaches is the God-Object method by Ortega et al. A God-Object follows Gauß' "Least Constraints Principle". This Principle states that from all possible motions of an object it is this motion that materializes, which needs the minimal effort, while considering all effective constraints on the object.

As an example, the figure on the right shows an God-Object (blue), which tries to reach the tracking target (red). When not considering constraints emerging from the surface of this obstacle, this gives the unconstrained movement of the God-Object. But, when also considering the non-penetration condition, the minimization problem of Gauß gives this constrained motion as wanted.

In the approach of this thesis, the virtual hand model consists of single elements, which can be seen as God-Objects. But the God-Object approach is not sufficient for creating a system for hand interaction. The original approach was extended by a multibody simulation for realizing the virtual hand, an N-body simulation for be able to interact with multiple objects simultaneously and a friction approach, since grasping isn’t possible without friction. A set of God-Objects in combination with these extension is called "God-Hand".

Testing Environment & Conclusion

The prototype uses a CAVE with three sides for visual output and an A.R.T. Fingertracking for tracking the hands. The simulator was implemented in the VR system Impression, which was developed in the Research & Development Department of the Volkswagen AG. All scene geometry in the scene graph is updated by the main loop. The tracking data comes from a handtracking plugin running in another thread. The physics simulation of the scene objects is done by Bullet Physics also in a separate thread. This simulator calls the God-Hand module. In the God-Hand module several libraries are used, for example LAPACK for solving equation systems, the Non-Negative Least Squares Solver for doing the Gauß minimization and the FAST library for continuous collision detection.

The measurements taken in a simple interaction test prove that this approach enables natural hand interaction independent from the number of objects in the scene. Deep penetrations are avoided completely. The modular approach ensures high efficiency to process the simulation in real-time.

The results show that this approach works for assembly simulation in concept. But the results also show that some things do not work well in this prototype. First, the used friction model is insufficient. The friction can have unwanted artifacts if the collision detection generates jumping contacts on plane surfaces.

Another issue leads to unrealistic hand movements. The constraints of the virtual hand are not handled properly, if the simulation error is too high. This is the reason, why it is necessary to insert an additional error correction step after the God-Hand simulation.

Last Changes 13.04.2011