To date, the development team of Facebook Reality Labs and Oculus have created the only fully articulated hand tracking system for VR that is completely based on monochrome cameras. The system uses neither active depth sensing technology nor additional devices. Facebook will use this technology as a software update for Oculus Quest. This is the first wireless, stand-alone VR headset that has recently become available on the market.

neural networks Oculus Quest.

By using Quest’s four cameras in conjunction with new deep learning and model-based tracking techniques, a greater volume of interaction is achieved for hand tracking than with depth-based solutions, and at a fraction of the size, weight, performance and cost. Processing is done entirely on the device and the system is optimized to support gestures for interaction, such as pointing or pinch to select.

Here’s how it works.

Deep neural networks are used to predict the location of a person’s hands and landmarks such as wrists. From these landmarks, a 26 degree of freedom pose of the person’s hands and fingers is then reconstructed. The result is a 3D model that includes the configuration and surface geometry of the hand. APIs enable developers to use these 3D models to enable new interaction mechanisms in their applications or to control a user interface.

Facebook uses a novel tracking architecture that provides accurate, low-vibration estimates of hand pose in a variety of environments and an efficient, quantized neural network framework that enables real-time hand tracking on a mobile processor without impacting user application resources.

Increase your conversion rate.

We help you to generate more inquiries from your website with Virtual Reality content.

More informations

Why it’s important.

Precise hand tracking will open up a number of new possibilities and reduce barriers to using the Oculus Quest. For example, it might be possible to pause a movie in VR with just one gesture and express yourself more naturally in social games. In business applications, an instructor can lead a VR-based training session without having to maintain a fleet of paired, charged controllers.

More specifically, hand tracking will help make VR feel more natural and intuitive. It will also help developers create new ways for people to interact in virtual worlds. Facebook Reality Labs and Oculus will look for ways to build on this core technology to improve future AR/VR experiences.

Thank you for your visit.