Computer Vision & AR (Augmented Reality) Try-On Solutions in Retail

Most of the sci-fi books we pick up have one thing in common – the natural ability of stationary and portable computers to interact with humans, analyzing the world around them and making sound judgements seemingly with minimum effort. Humans have long dreamed of giving computers the ability to “see” and interpret the world through the prism of the human visual system, but how close have we come to achieving this dream?

Computer Vision (CV) is one of the most promising scientific fields that combines machine learning and artificial intelligence (AI) to obtain information from images, videos, multi-layered data or even “thermal and infrared cameras”. It is the field that enables and shows computers how to recognize and interpret the world.

Computer Vision has accelerated industries and innovations, with Tesla taking CV to “unprecedented levels with their pseudo-lidar approach” that allows to interpret raw visual data on pixel level, announcing the roll out of their “full self-driving” software to customers from the Tesla Early Access Program on October 21st.

Developments of CV and AI based technology allowed to expand the horizons of inclusivity, enabling devices that can improve the quality of life for visually impaired, with OrCam 2020 announcement of product line expansion to OrCam Hear technology for hearing impairment. The combination of CV and AI technology that drives innovation and is now commonly used in healthcare, military, manufacturing, finance (and many more), has made its way into our daily lives with large online and offline retailers implementing CV for better and more convenient product discovery. Amazon, amongst the pioneers of advanced retail algorithms development, that helps its customers with shopping for their desired items, has gone further and announced the 2020 roll out of the new CV based feature that allows viewing multiple AR items at once.

For many consumers, their first interaction with CV happens through one of the Augmented Reality (AR) Try-On experiences that the majority of large brands have taken on board for the simplicity and potential this technology brings to both online and offline customers. This rapid popularity of implementation is making it possible to literally “try-on” anything – from seeing how an item of furniture or piece of art will look and fit into your place to checking how your dog can rock a pair of glasses.

CV is the main defining pillar of the AR experience, it is the instrument and the building block that allows to tailor make specific features of any required Try-On experience.

The technology can be implemented into make-up, FMCG and accessories segments due to its capability to detect faces and recognize particular facial features (eyes and mouth position, face shape and completion) allowing to track and follow the person’s movements in real-time through convolutional neural network (CNN) architecture. Fashion brands can benefit greatly from the CV algorithm capabilities to go beyond face recognition, to human recognition and full body capture, separate body part detection, understanding human intentions and actions with the help of Eigen-Value Decomposition, Density Estimation and Clustering to generate behavior models. A wider implementation of CV allows the algorithm to separate living things from inanimate objects, recognizing items, backgrounds, patterns, position and points where the try-on is to happen through Deep Neural Network (DNN) & Region Based Convolutional Neural Networks (R-CNN), which can be a great instrument for automobile, manufacturing, healthcare, art and many more product based industries.

Additionally, the incorporation of Cloud Computing can help build more elaborate CV based AR experiences, as the main benefit of cloud computing is in its ability to collect and store analytics data. Making iterations of the AR experience data-driven and thus more user-oriented.

It is widely discussed how big will the COVID-19 pandemic impact be in the long-run, the one thing that is clear is that the landscape of online and offline retail has been forever altered. For any retailer regardless of the scale it has become essential to provide a safe and preferably touch-free shopping experience. A large number of customers are leaning towards e-commerce, which is estimated to grow by 20% in 2020 in US only, while reshaping the view of the consumer basket essentials. It has become vital for product-based businesses to quickly adapt to the online distribution channels, building strategies of how to remotely showcase their products and help customers with decision making. Bringing AR Try-On technology to the table will become the defining factor for many companies being able to stay in business while adapting to the online retail marketplace.

As one of our AR Try-On projects we aimed to create a unique immersive rim selection try-on experience for our client using AR technology, a solution that allows to increase efficiency, optimize demonstration time, digitalize, improve end-consumer communication and convenience in real-time, allowing customers to switch between wheel rims instantly. The task was to create rim detection modules, providing a computer vision (CV) module and delivering an augmented reality experience for the customer, a solution that can be integrated and built upon an existing app, allowing users to browse through and select from a wide assortment of the rim product portfolio.