How to prepare virtual and augmented reality content for the web.

With the consistent introduction of various enabling devices, Virtual Reality (VR – ideal to viewing 3D configurators in real time)) is facing the next wave that is becoming popular. As a technical branch of VR, Augmented Reality (AR) is also preparing for its increasing spread on the basis of clearly stabilized computer vision technologies. The widespread use of VR/AR leads to an ecosystem with a scalable content platform as its center. The platform will support efficient production and offer a broad distribution channel.

The Web has been publicly qualified over the last two decades as a scalable content platform. Its design principles, consisting of simplicity, modularity and decentralization, have made it possible to easily create different content by reusing existing external resources, to publish the content simply by inserting it into a web server and to dynamically update the content at runtime even by end users after distribution. And there is no doubt that these capabilities are the result of HTML and its underlying principle, web architecture.

However, many content specifications for VR/AR that try to follow web-friendly approaches have had difficulty reaching consensus because their independent structures were not compatible with the web architecture and it has been difficult to take advantage of the web ecosystem. We believe that in order to support VR/AR as a content platform, web technologies must merge the requirements of VR/AR in their core specifications according to the web architecture.

In the following article, we would like to share insights from the development of a mobile AR web browser with you.

Preparing Virtual and Augmented Reality Content for the Web.

The current model for creating VR/AR applications for the Web is mainly based on a further development of traditional programmer frameworks for WebGL-based computer graphics. In this model are applications:

• Limited functionality – Although 3D content can be displayed in web browsers, it is not seamlessly integrated into the existing web ecosystem due to its separate handling with other HTML elements. Users want to mangle the existing Web API or widgets (e.g. Google Maps, jQuery) that provide versatile features and live data feeds into the virtual world they create, or vice versa.
• No way to refer to the real world: There is no official way to process the recognition data for objects and locations in the real world (e.g. feature descriptors for visual image matching and tracking) for AR web applications.
• Separate rendering context – Like typical HTML content, VR/AR content needs to know the rendering context (e.g. viewport size, scaling) in order to apply rule-based dynamic layout (e.g. media query) or level of detail (LOD). However, the WebGL-based 3D context is isolated within the canvas element along with the 2D context and managed separately from the rendering context of other elements.

The basic principles of VR/AR must be further developed with regard to the preparation of such content for the Web. The basic building blocks and operational mechanisms are organised according to the type of web architecture. In contrast to the traditional computer graphics approach, AR/VR applications are optimized for the Web:

• Mashable – VR/AR content and existing Web elements are seamlessly integrated into a single Web application. This would allow an HTML element to be embedded in a 3D object and influence each other. In addition, various external frameworks (e.g. Web of Things, Payments and jQuery) can be added to AR/VR content.
• References to the Real World – Like images and videos, AR Web applications can simply declare the real entities that authors want to recognize by referring to their URLs, and the recognition data (e.g., feature descriptors) is loaded by the browser and injected into its internal tracking engine.
• Reacts to both the virtual and the real world – the browser’s renderer selectively adapts to the page, virtual and real world. The page model is a traditional browser metaphor based on a 2D layer to display an HTML document on a screen. For VR, the browser’s viewport is seen as the same as a camera in the 3D scene (accessed via the WebVR API). For AR, the tracking data of real objects and locations are applied to the rendering context of the browser.

Requirements and possibilities of standardization.

In order to realize the AR/VR Content Vision optimized for the Web outlined above, there are several requirements and possibilities of standardization as follows:

• Development of HTML for VR/AR – In order to mix VR/AR content with existing web content, it is necessary that VR/AR content becomes a first class experience in HTML. To achieve this, a declarative 3D specification is required that is fully HTML compliant. VR/AR content should be handled in the same way by browsers with typical HTML elements in a single DOM tree.
• “Physical Things” as Resources – To enable AR recognition on the Web, all real-world targets, including physical objects and locations, are considered resources and should be accessible via standard Universal Resource Identifiers (URIs) to provide feature descriptors for identification and tracking. In our implementation, they will be declared as a value for the specified CSS property in the AR web application.
• New media type for VR/AR – The separation of rendering contexts between VR/AR scene and page could be addressed with respect to the Responsive Web. The different renderers can be dynamically assigned by adding New CSS Media types for VR/AR or even Unified Renderers for Mixed Reality (MR). CSS Media Query contains a media type specification to apply different stylesheets to different rendering methods (e.g. language, Braille embossed etc.). In our implementation we have defined the “place” media and the AR rendering mode and vision tracking engine are triggered when declared in an HTML document.

Conclusions.

The VR/AR for Web promises to become one of the most fascinating technological challenges. The scale and the overall impact of this emerging field are staggering. We believe that standardization efforts will be critical to success and the ability to bring them to their full potential. There are many areas for standardization in VR/AR for the Web and we are very interested in participating in this workshop. We believe that one of the critical domains is the definition of a standard development, deployment and management model for VR/AR web applications, including the formal definition of physical things, media types and associated data types.

Thank you for visiting.