The new chip, called "Navion," is only 20 square millimeters in size, about the size of a LEGO minifigure, and consumes only 24 milliwatts of power, or about one-thousandth of the energy required to light a light bulb.
Using this small amount of energy, the chip can process camera images in real time at up to 171 frames per second, as well as inertial measurements, which it uses to determine its location in space. The researchers say the chip can be integrated into "nanodrones" as small as a fingernail to help vehicles navigate, particularly in remote or inaccessible locations where global positioning satellite data is unavailable.
The chip design can also be run on any small robot or device that needs to navigate for long periods of time with a limited power supply.
"I can imagine applying this chip to low-energy robotics, like flapping vehicles the size of your fingernail, or lighter-than-air vehicles like weather balloons, which have to last for months on a single battery," says Karaman, a member of the Information Systems and Decision Lab and the Institute for Data, Systems and Society at MIT. "Or imagine medical devices like a tiny pill you swallow, which can navigate intelligently on very little battery power so it doesn't overheat in your body. The chips we're building can help with all of this.".

Co-authors Sze and Karaman are EECS graduate students Amr Suleiman, who is the lead author; EECS graduate student Zhengdong Zhang; and Luca Carlone, who was a research scientist during the project and is now an assistant professor in the Department of Aeronautics and Astronautics at MIT.

A Flexible Chip:
In recent years, multiple research groups have designed miniature drones small enough to fit in the palm of your hand. Scientists envision such tiny vehicles flying around and taking pictures of their surroundings, like mosquito-sized photographers or surveyors, before landing back in your palm, where they can then be easily stored.
But a hand-sized drone can only carry one battery, most of which is used to power its motors, leaving very little energy for other essential operations, such as navigation and, in particular, state estimation, or a robot's ability to determine its location in space.
"In traditional robotics, we take off-the-shelf computers and implement [state estimation] algorithms on them because we don't typically have to worry about power consumption," says Karaman. "But in every project that requires us to miniaturize low-power applications, we now have to think about the programming challenges in a very different way."
In their earlier work, Sze and Karaman began to address these problems by combining algorithms and hardware on a single chip. Its initial design was implemented on an FPGA, a commercially available hardware platform that can be configured for a specific application. The chip was able to perform state estimation using 2 watts of power, compared to larger, standard drones that typically require 10 to 30 watts to perform the same tasks. Even so, the chip's power consumption was greater than the total amount of power that miniature drones can typically carry, which researchers estimate at around 100 milliwatts.

To further reduce the chip's size and power consumption, the team decided to build a chip from scratch rather than reconfigure an existing design. "This gave us much more flexibility in the chip's design," says Sze.

Racing in the World:
To reduce the chip's power consumption, the group devised a design to minimize the amount of data stored on the chip at any given time, in the form of camera images and inertial measurements. The design also optimizes how this data flows through the chip.
"Some of the images that we would have temporarily stored on the chip, we actually compressed, so it required less memory," says Sze, a member of MIT's Research Laboratory of Electronics. The team also reduces extraneous operations, such as calculating zeros. The researchers found a way to skip those computational steps that involve zeros in the data. "This allowed us to avoid having to process and store all those zeros, so we can eliminate a lot of unnecessary store-and-calculate cycles, which reduces the chip's size and power and increases the chip's processing speed," says Sze.
Through their design, the team was able to reduce the chip's memory from its previous 2 megabytes to approximately 0.8 megabytes. The team tested the chip on previously collected datasets generated by drones flying through various environments, including office spaces and warehouses.
"While we've customized the chip for low power and high-speed processing, we've also made it flexible enough to adapt to these different environments to save additional power," says Sze. "The key is finding the balance between flexibility and efficiency." The chip can also be reconfigured to support different cameras and inertial measurement unit (IMU) sensors.
From these tests, the researchers found they could reduce the chip's power consumption from 2 watts to 24 milliwatts, which was sufficient to power the chip to process images at 171 frames per second—even faster than the projected datasets.
The team plans to showcase their design by implementing the chip in a miniature race car. While a screen displays live video from an onboard camera, the researchers also hope to show the chip determining its location in space in real time, as well as the amount of energy it uses to perform this task. Eventually, the team plans to test the chip on a real drone, and finally on a miniature drone.

This research was supported, in part, by the Air Force Office of Scientific Research and the National Science Foundation.
###
Written by Jennifer Chu, MIT News Office