Virtual Reality Haptic Drones

Virtual Reality (VR)  is evolving at a staggering rate. A consumer VR head-mounted display (HMD) offers to transform any living room into a highly engaging virtual environment of one’s dreams; which is why VR technologies are becoming more and more popular with time. In order to improve user experience and to achieve the current state of technological development and user expectations, academic and commercially-oriented research has especially focused on improving video and audio technologies.  Recent developments show good results: the devices’ high-resolution, near-eye displays combined with surround sound create an immersive audio-visual experience that induces a high level of presence felt by the user.

What is VR Haptics?

VR Haptics technology offers an extra dimension to the VR world by letting users feel the virtual environment via the sense of touch, in addition to visual and aural perception. It makes the VR world truly immersive for a user who can experience their Virtual Reality from the artificial world with much more immersion.

VR Haptics Feedback

Virtual Reality (VR) technologies show a huge lack in giving appropriate haptic feedback. There has been less emphasis on stimulating other sensory modalities, such as the different subcomponents of the haptic system.

Through the past decade, a number of different systems and approaches have been developed that simulate virtual reality by providing appropriate feedback against touch and sound. One such approach is using gloves, belts, helmets or pads that react to the changes occurring in the virtual reality.

Another approach is to provide haptic feedback via devices and objects that are located in the user vicinity. These can be robot arms that provide force-feedback at the location which the user reaches out for, or real props that emulate the virtual objects’ forms. Here, users are not required to wear a multitude of specialized devices but this approach significantly limits the domain of the Virtual Reality experience since the user has to be in close vicinity to the objects that react to virtual changes in the real world.

VRHapticDrones – A New VR Haptics Approach

To provide an optimized, effective approach for improving the feedback system used in VR Haptics, the publication, “VRHapticDrones: Providing Haptics in Virtual Reality through Quadcopters“, was compiled by different researchers from LMU Munich, TU Darmstadt, Wellesley College, University of Duisburg-Essen and University of Regensburg.  Their solution for development of a VR Haptics system was the usage of drones.

They developed VRHapticDrones, a system that provides haptic feedback in VR.

Their proposed system solves the primary drawbacks with already implemented methodologies described above as it does not require any additional wearable devices and provides the user with the freedom to move around even in large spaces.

How Does it Work?

VRHapticDrones utilizes quadcopters that can simulate haptic user interaction with a large range of objects, such as a coffee mug or a wooden chair, in the VR environment. VRHapticDrones adds another dimension to VR by providing a haptic experience. It imitates stationary and moving objects by aligning its touchable surface with the surface of virtual visual objects, thereby allowing the user an active haptic exploration of the virtual object.

When the user reaches out to touch the virtual angler fish, the system allows the user to experience a congruent haptic stimulus. The system controls a quadcopter at the exact location of the virtual fish to provide a synchronized touchable surface.

Each quadcopter is used as a haptic feedback interface. To provide the feedback to touch and stimulate sounds, different extensions are added to different drones as apparent from the figures attached. The quadcopter used for experimentation was the commercially available, “Parrot Rolling Spider” the extra parts of which were removed to reduce unnecessary load and thereby increase battery timing. The quadcopter was connected to the VRHapticDrones program controller with Bluetooth low energy.

VRHapticDrones supports three different feedback modes: (a) Passive: The object is levitating and the user can touch it. (b) Active: The object is proactively bumping into the user. (c) Proxy: The object can be grasp and moved by the user.

Quadcopter prototypes to deliver different types of feedback. (a) passive feedback, (b) active feedback, (c) haptic proxy feedback.

The VRHapticDrones Backend interconnects the VR rendering engine, the quadcopter control, and the motion tracking system.

All components of the VRHapticDrones system

The backend integrates core power or a computer with VR Renderer system as well as a PID controller, forming the control system for the VR setup that continuously monitors the drone performance and optimizes the feedback given by the drones.

Participant exploring the virtual balloon during the first user
study through touching and pushing.

VRHapticDrones supports three different types of haptic feedback; passive, active, and proxy-based feedback. Based on two different user studies, it was verified that the haptic feedback provided with drones, via VRHapticDrones increases the level of immersion offered by the VR environment to the user as compared with traditional feedback systems.

Moreover, the feedback provided by drones is considered more realistic by the users if the feedback represents lightweight objects with a hard surface, or objects with soft surfaces.