The Khronos Group has introduced its new interoperability standard for neural networks. The Neural Network Exchange Format (NNEF) is an open, implementation-independent method for describing neural networks that aims to break the current tangle of framework-specific formats.

What is KeyShot?

The new standard was provisionally published in December 2017 and, after a period of consultation with industry, has now been ratified in its final form and is available for immediate use. In addition to the standard itself, Khronos is also releasing a number of open source tools to enable developers to immediately use the format with the three most popular training frameworks: Tensorflow and Caffe/Caffe 2. All these tools are available on Github in the Khronos Repo.

NNEF belongs to a group of standards promoted as “Khronos Standards for Machine Learning”, the others being OpenVX and SYCL.

Together with other Khronos standards, including Vulkan and OpenCL, which interface with GPU computing power, these three standards form an ecosystem that links a complete workflow for training and deploying neural networks and the vision applications that increasingly depend on them. SYCL provides an alternative to CUDA that works across systems, OpenVX allows hardware vendors to present a common vision API in an embedded system that does not rely on a higher-level OS-specific deployment such as the Android NN API, and NNEF simplifies and standardizes the movement of neural networks from the training phase to deployment on any inference engine.

Khronos moves quickly to make these standards work together as seamlessly as possible. While SYCL is a relatively universal programming language, the OpenVX API naturally provides a number of functions, and its legacy lies in the worlds of video streaming and what we now call “classical” image processing. A recently completed neural network extension expands inferentiality, and AMD demonstrated an exemplary implementation of a Caffe-to-OpenVX workflow using NNEF as the transfer format at this year’s Embedded Vision Summit. With NNEF in flow, trained networks can be used on any framework to output them in NNEF format. Khronos started with bidirectional translators for Caffe, Caffe 2 and both tensorflow representations (Python and Protobuf).

We are strong advocates of open standards and see them as the basis for a dynamic industry that enables competing companies to jointly drive technology forward. However, this view is not universal, and more than one company advocates rapid open source development based on proprietary initiatives and states that the speed and responsiveness of open source teams is required in the evolving world of machine learning.

There is no doubt that Nvidia has done a great job with CUDA by equipping the machine learning industry almost single-handedly with the tools to accelerate training with neural networks. The disadvantage is that CUDA only runs on Nvidia GPUs and the industry now needs to be larger than one company, so the development of SYCL is welcome. The same consideration applies to NNEF, which has a competitor in ONNX (Open Neural Network eXchange format). ONNX is a valid format, but it is owned by a small clique of companies and the infrastructure of the machine learning industry needs to be more open than that.

The arguments for open source standards are broader and much more profound than mere development speed, but it is thanks to Khronos that they recognize the tendency for standards to continue to be worked on while the outside world continues. Their strong urge for open source development, not only of the tools surrounding NNEF, but also of the process of extending it and putting everything on Github, where everyone can freely collaborate, is a masterpiece.

These standards are something that the further development of machine learning will continue to urgently need in the future.

Thank you very much for your visit.