Imagine walking down the street and seeing a poster for your favorite movie or 3D configurator. Now point your Smarthone camera at the poster. A snapshot brings the characters from the film to life and you can immerse yourself in an Augmented Reality (AR) experience directly from your smartphone.
AR-Tracking is the next milestone in the development of Facebook’s AR platform. The AR Target Tracker effects available in Facebook’s AR Studio allow brands, publishers, artists and developers to create content that is linked to images, logos, signs and images in the real world. The technology developed for the Facebook camera allows the camera to track an image and create an immersive experience. Augmented Reality can be anchored on any flat surface so that the effect appears as an overlay in the Facebook camera.
The films “A Wrinkle in Time and Ready Player one” use the Facebook AR Target Tracking technology to create immersive experiences linked to the latest cinema charts, with the aim of inspiring fans in a new way. People can now try the experience on all their movie posters around the world. A toolkit for all developers is available as a feature under the Plane Tracker menu in AR Studio.
Switch from ephemeral to persistent ARs.
In December last year, AR Studio was launched, with which creative people can create effects and place 3D objects in their environment and interact with them in real time. This ephemeral task allows them to place an AR object on any surface to interact and share. Facebook wanted to take it to the next level and investigate use cases around persistent AR – experiences associated with and related to places or things in the real world. AE Target Tracking is the first instance of Facebook for this kind of persistent AR experience.
Facebook’s first excursion into the persistent AR is focused on planar 2D targets because they are ubiquitous. For example, the movie poster exists all over the world, so everyone can experience the same before the poster, as AR is tied to that poster. This allows us to overcome time and space and offer our Facebook community a unified experience.
Facebook’s approach focuses on quality and access.
While others have developed similar technologies, there are two things that are unique about our approach. Our goal is to make the technology available to a variety of devices at the highest quality level and to maximize access to the different mobile devices so that everyone with an iPhone 5s and above or an Android 2012 device and above can use them. Facebook is able to offer this experience for high-end market devices down to older, lower end devices to ensure that people in every market can experience it. Facebook has achieved this coverage by greatly optimizing the tracking algorithm to work really well on the low-end devices.
Facebook also wants to maintain the highest level of quality in the AR experience. To maintain a high level, Facebook tests algorithms under different lighting conditions, at night, at different angles and with partial occlusion.
Create an AR Target Tracker effect.
To create an AR Target Tracker effect for a poster, you must first upload the master poster image to the AR Studio. You would then create and upload the specific AR art experience to embed in the poster. Once it’s extended, a person simply opens their Facebook camera, starts the effect from the effects tray, and can point to the poster. The poster triggers the AR experience for the user as soon as he points at it. The person can also achieve the effect directly through other mechanisms such as a pre-programmed QR code, such as what was done on the Wrinkle in Time and Ready Player One movie posters.
How it works.
The poster or image uploaded into AR Studio forms the template image with which the camera’s video images are compared to recognize the poster. Facebook technology analyzes the template poster to find clearly identifiable low level visual features and then uses an image matching and verification pipeline to compare it under different conditions.
The process takes the template poster and then searches for uniquely identifiable parts such as corners or texture patterns. Patches are then extracted from these locations and encoded into a feature decriptor vector using a pre-trained encoding process. These vectors are then quantized and stored in an efficient index that is quickly looked up and adjusted.
The key to the effectiveness of this algorithm is how well it can match what has been uploaded so that the user is immediately in the experience. Once there, the algorithm switches to tracking the poster. When the user moves his smartphone, we have to make sure that we don’t lose the tracking and that he continues to see the AR, even if the image is partially obstructed by e.g. someone walking in front of it. To achieve this, Facebook uses the Viper tracker, which was used in the past for SLAM tracking for world effects. Facebook’s method does this by detecting and tracking the support structure in the environment outside the poster – with a highly optimized one that is fast enough to run on low-end smartphones.
Opportunities for founders and businesses.
This new target tracking feature offers companies completely new ways to build their brands and connect with customers through immersive AR experiences. It is a novel and dynamic way to reach customers across touch points and around the world. The movie poster (or any other poster in this context) can be used in all types of promotions, from free popcorn to free ticket hunting. Restaurants and shops can create vivid and entertaining AR experiences tied to their shop windows to tempt customers to walk through the door. In addition, the ability to create games will go far beyond the imagination. The possibilities are endless.
Thank you very much for your visit.
Leave A Comment