RealWalk: Feeling Ground Surfaces While Walking in Virtual Reality

We present RealWalk, a pair of haptic shoes for HMD-based virtual reality, designed to create realistic sensations of ground surface deformation through MR fluid (Magnetorheological fluid) actuators. RealWalk offers a novel interaction scheme through physical interaction between the shoes and the ground surfaces while walking in virtual reality. Our unique approach of using the property of MR fluid creates a variety of ground material deformation such as snow, mud, and dry sand by changing its viscosity when the actuators are pressed by the user’s foot. We build an interactive virtual reality application with four different virtual scenes to explore the design space of RealWalk.

RealWalk Architecture

The main goal of RealWalk is to deliver realistic sensations of the ground surfaces by providing different types of material deformation while walking in virtual reality. We accomplish this by specially designed MR fluid actuators while the shoes with the actuators are physically stepped on the ground surface, delivering a various sensations of ground deformation such as snow, mud, and sand by varying the magnetic field intensity. Our approach is unique and can replicate the sensations of the material deformations of the ground surface as it uses the property of MR fluid that rapidly changes its viscosity.

In RealWalk, a variety of mechanisms have been implemented to deliver various sensations of ground deformation. This integrated system includes MR fluid actuators, insole pressure sensors with position tracking system, and haptic rendering for multimodal interaction.

MR Fluid Actuators

A key component of physical interaction is MR fluid actuators that we designed and implemented for RealWalk. MR fluid is a type of smart fluid and it immediately changes its viscosity when subjected to a magnetic field. We designed the MR fluid actuators to adaptively adjust the viscosity of MR fluid by varying the magnetic field intensity based on the type of materials in virtual ground surfaces while pressing the actuators with human foot.

In MR fluid actuator, MR fluid is contained in a cylinder along with the magnetic coils. The size of actuator is 55 mm (length) x 46 mm (width) x 50 mm (height). A contact cover is placed onto a cylinder with a supporting plate for supporting the force from the foot and creating the resistive force at the same time. A spring is installed between the contact cover and the cylinder for returning the contact cover back to its original position when it is not pressed by the foot. Ultimately, two MR fluid actuators and one external spring are installed between the shoe and a base sole to complete a RealWalk shoe. The locations of two installed actuators (front and rear) are determined based on amount of foot pressure distribution of human walking. An insole-shaped force sensor and a VIVE tracker for tracking precise foot position is also attached in the shoe, which will be discussed in the next subsection.

Insole Pressure Sensor

RealWalk captures the moment when a shoe makes contact with the ground and how much pressure is distributed over the sole using insole-shaped pressure sensor in the shoe. This insole pressure sensor contains 118 sensing nodes of piezoresistive sensels, with a size of 301.9 mm (length) x 105.8 mm (width).

Haptic Rendering

We set a number of haptic feedback signal profiles based on the viscoelastic behaviour when undergoing deformation with a range of amplitude and frequency in a square waveform signal. Predefined haptic feedback signal is delivered based on the type of materials of ground surfaces and its intensity is adaptively adjusted based on the amount of insole pressure distribution.

Four Seasons

To explore the design space of RealWalk, we built a virtual reality application using Unity3D with four different virtual scenes of the four seasons: spring, summer, autumn, and winter. In a spring scene, the grass is covered on the ground surface to provide the characteristics of grass material when the users step on the ground. In a summer scene, users are allowed to walk around the beach and able to feel the deformation of dry sand on their sole. In autumn, users are able to feel the deformation of mud when stepping on the ground surface. Lastly, a winter scene is implemented with snow piled up on the ground. In all four seasons, users are allowed to freely walk around the scenes while wearing both HMD and RealWalk shoes, perceiving different sensations of material deformation of the ground surfaces. The visual footprints are also created along with the appropriate sound effect (i.e. sound of snow crunching) to maximize the realistic walking experiences with multimodal cues in virtual reality.