Refinity: An Interactive Holographic Signage for New Retail Shopping Experience

Refinity is interactive holographic signage for the new retail shopping experience. In this work, we show a concept of futuristic shopping experience with a tangible 3D mid-air interface that allows customers to directly select and explore realistic virtual products using autostereoscopic 3D display combined with mid-air haptics and finger tracker. [Paper] [Link]

Shopping in brick-and-mortar stores is an integral part of our everyday lives, whether for food, clothes, furniture or home supplies. These offline stores continue to play an important role in providing a level of tangibility and physical connection. Top online retailers have ventured into physical stores like Warby Parker showrooms, Amazon’s Wholefoods, and Alibaba’s Hema. It now becomes important to create not only seamless online and offline experiences but also ways to further enhance the physical shopping experience. Through the integration of interactive immersive technology such as VR and AR, an engaging product experience can be instilled to form consumer interest. Physical shopping is entering into an age of new retail as consumers expect seamless experiences with deeper levels of personalization and memorable engagement to complete the purchase cycle.

The integration of VR and AR into digital signages create a more immersive and engaging experience but is still limited by the unnaturalness of wearing headsets or need to look through a limited mobile screen. Since in-store shopping experiences should be seamless and convenient, natural and tangible user interaction with realistic 3D product experience is essential. It is important to explore the opportunities to design a completely new retail digital signage that can provide more realistic and natural shopping experiences without wearing any dedicated equipment.


This demo illustrates an interactive holographic signage for a new retail shopping experience called Refinity. Refinity provides a tangible shopping experience using glasses-free autostereoscopic 3D display combined with eye-tracking, mid-air haptic display, and finger gesture interaction of 3D products. These unified technologies with seamless natural interaction schemes allow customers to have enriched experiences with realistic virtual products in mid-air. The main contribution of our demo is that we show one promising direction towards the next generation of in-store digital signage. By utilizing natural interaction with virtual products, our demo shows how customers can have seamless and personalized shopping experiences. Since there exists no such 3D user interface that adopts natural and tangible interactions in mid-air, there is therefore a need to carefully design what it should be. We believe that our demo aims to fill this gap.