Haptic Perception in Multimodal Virtual Environments - The Haptic Paradigm for Teaching and Learning

Multimodal Virtual Environments (MVEs) interface between humans and computers using multiple human modalities. A multimodal interface conveys information in a more naturalistic fashion. The user manipulates and experiences the environment through multiple sensory channels. Out of the interfaces for our five senses, the visual and auditory ones have been used most extensively however, interfaces already exist that allow us touch (haptic devices), and technologies for taste and smell may be the next step. Such interfaces give birth to new paradigms for human computer interaction (HCI).

Involving students in the learning process has been a challenge for educators for many years. MVEs may contribute to raising interest and motivation in students, improving their learning and retention. In addition, MVEs provide an alternative method of presenting complex concepts during the educational process.

In this paper we review MVEs and their use in K-16 education, with the emphasis on the potential combinations of audio, visual, and haptic modalities. We provide a brief historical perspective of MVEs and review their evolution in parallel with education. We present the results of an investigation into the potential benefits of incorporating haptic feedback into an MVE intended for college and high school physics curricula. Our HaptEK16 application helps students understand the difficult concepts underlying hydraulics and Pascal’s Principle. HaptEK16 has the potential to augment or replace traditional laboratory instruction with an approach offering enhanced motivation, retention and intellectual stimulation.