Self-Driving Neural Networks

Hardware, architectural advances key in establishing self-driving neural networks

In many ways, the theory behind using machine learning to improve self-driving vehicle functionality is in place. Industry leaders, such as NVIDIA, have demonstrated the ability to use GPU acceleration and similar technologies to establish neural networks capable of processing images and other environmental data sources to make decisions about how a vehicle should move. These advances serve as an excellent proof of concept, but progress still needs to be made before autonomous vehicles are a reality.
Regulatory and field testing issues aside, hardware and application architectural issues may be the primary factors holding back autonomous vehicles. In this case, it is important to look beyond self-driving cars and consider the entire robotics ecosystem.

For example, consider a small drone used by a military unit to patrol a base. These drones can intelligently fly over terrain in preset patterns and respond to any barriers that arise along the way. However, there are still limitations in these type of devices, particularly when moving through complex environments, such as forests, or in unknown areas. The problem is that drones of this sort are fairly small and, therefore, need processing and memory systems that fit into a minimal form factor. While cars may have more space, the room for machinery and computing systems is still at a premium.
Fitting complex neural networks into a minimal hardware footprint requires major engineering and development architecture advances before self-driving systems based on neural networks can become mainstream.

Hardware advances and the self-driving vehicle

According to the IEEE Spectrum, engineering teams from MIT, NVIDIA and the Korea Advanced Institute of Science and Technology came together at the IEEE International Solid-State Circuits Conference to put some new prototypes on display. These systems focused on the idea of using low-power chips that are specifically designed for neural network demands. These types of solutions could prove invaluable moving forward as autonomous vehicles become more mainstream.

The report explained that, where the market stands right now, the smartphone processor industry is on the brink of being able to handle basic neural network functionality. Qualcomm’s next-generation Snapdragon processors are expected to release with an automatic image labeling software-development kit. This functionality will enable organizations to apply basic neural network design into smartphones. While this technology is promising, it only represents basic neural network functionality, not a system that is capable of learning new things by itself. Essentially, organizations will need a more robust machine learning ecosystems to identify the best way to respond to situations and then apply that logic to the small-scale neural network.

The project with MIT, NVIDIA and KAIST showed the potential solution for small form-factor processing systems, and they focus on solving a straightforward problem – most of the energy consumption within the neural network comes when moving data between the processor and the memory, IEEE Spectrum explained. To resolve this issue, NVIDIA showed off a prototype that strategically locates dedicated memory in close proximity to the processing engine in the GPU. Data is also compressed and initially drawn from a large memory bank, allowing for a reduced need to send data back and forth between processing systems and memory. All told, the prototype chip, called Eyeriss, was able to run the AlexNet algorithm using just one-tenth of the energy that a typical mobile GPU would require to get the job done.

This type of hardware innovation is integral in allowing for convolutional neural network deployment in a small enough form factor to work in a smartphone-sized device. As hardware improvements continue to advance, organizations can also leverage sophisticated programming architectures to improve neural network functionality in self-driving vehicle settings.

Improving programming models to streamline neural network operations

While the most complex neural networks may depend on hardware improvements to work well in small form factors, it is possible to solve some of the efficiency challenges facing neural systems with nuanced programming architectures. Udacity recently ran its second challenge pertaining to building efficient deep neural networks. In this competition, Udacity challenged developers to submit models for how they would use a neural network to steer a car based exclusively on data from camera frames. The winner ended up being Ilya Edrenkin, senior researcher for Yandex.

According to an Udacity blog post, Edrenkin was able to create a robust neural network model by treating each sequence of camera frames as a sequence-to-sequence mapping problem. This allows the model to apply causal logic to the steering sequence.

“The model architecture was composed of a convolutional vision stack followed by a stateful recurrent network,” Edrenkin wrote in a journal detailing the neural network model. “Convolution was performed not only in the image plane but also in time; it allowed the model to build motion detectors and account for the driving dynamics. The recurrent cell tracked the state of the vehicle, which was supported by an auxiliary cost function – the model tried to predict not only the steering angle but also the steering wheel torque and the speed of the vehicle. Well-established methods like residual connections, layer normalization, and an aggressive regularization via dropout were necessary to obtain the best results.”

Bringing neural networks into mainstream uses

Faster, cheaper and smaller. In the computing world, you used to only be able to get two of the three. Neural networks are increasingly tasked with innovating in all three areas to allow for large-scale adoption in self-driving vehicles of all sorts, and both hardware and programming advances are making these advances possible.