Intermodal Conflict in Visio-Haptic Environments
SUMMARY: This project investigated human response to artificially created intermodal conflict between visual and touch (haptic) modalities. We studied the relative weight placed on touch and vision in the exploration of three dimensional shapes by humans in cases of conflict in virtual reality environments.
METHODS: Stimuli consisted of 3D models of rigid shapes that transitioned in 5 stages from a sphere to a cube. Haptic stimuli or touch feedback or the models was delivered using the Phantom Desktop joystick (OpenHL and OpenGL libraries utilized for visual and haptic rendering respectively). Ten participants provided feedback on their perception of the object they were interacting with (more like a sphere or more like a cube?) under three conditions – (a) vision only (b) touch only (c) both vision and touch modalities. Conflicts were introduced between modalities by varying the shapes perceived through vision and those felt using the joystick. The conflicts ranged from no conflict (same object presented visually and haptically) to maximum conflict (sphere presented in one modality while the cube was presented in the other modality).
CONCLUSIONS: The results indicated that in cases of zero (or no) conflict the relative weighting of each modality was roughly equal, but as the conflict increased participants increasingly relied more on their haptic sense rather than vision to make the shape judgment. This suggests that the perceptual conflicts in the visio-haptic domain are resolved in a compromise between vision and touch, with vision dominating touch. Translating these findings to practice, in educational settings where we would like to train users to rely on haptic feedback to better learn motor skills (for example laparoscopic surgical skill acquisition), utilizing virtual reality with controlled intermodal conflicts may enhance the learning experience (see applied example).
ROLE: Application Developer
STATUS: Completed (2006)
- [POSTER] Tripathi P, Gray R, Vankipuram M, Panchanathan S, “Humans Increasingly Rely More on Haptics in 3D Shape Perception with Higher Degrees of Visual-Haptic Conflict”, presented at the 9th International Multisensory Research Forum, (2008).