Part of the problem is the time it takes to produce and interpret the camera data. An autonomous vehicle using a standard camera to monitor its surroundings would need about a fifth of a second to update its location. That's sufficient for normal operating conditions, but not fast enough to handle the unexpected.
Andrea Censi, a research scientist at MIT’s Laboratory for Information and Decision Systems, believes the solution could be to complement cameras with a new type of “event-based” (or “neuromorphic”) sensor, capable of taking measurements a million times per second.
At this year’s International Conference on Robotics and Automation, Censi and Davide Scaramuzza of the University of Zurich are presenting the first state estimation algorithm—the kind robots use to calibrate their position—for processing data from event-based sensors. A robot running their algorithm could update its location every millisecond or so, enabling it to perform more agile maneuvers.
"In a normal camera, you have an array of sensors, and then there's a clock," Censi explains. "If you have a 30 frames-per-second camera, every 33 milliseconds the clock freezes all the values, and then the values are read in order." With an event-based sensor, on the other hand, "each pixel acts as an independent sensor," Censi says. "A change in brightness—whether positive or negative—is greater than a threshold for the pixel, which then says, 'I see something interesting,' and it communicates this information as an event. And then it waits until it perceives another change.".
Key Event:
When a standard state estimation algorithm receives an image from a robot-mounted camera, it first identifies "features": gradations of color or tone needed to determine the boundaries between objects. Then it selects a subset of these features that it considers unlikely to change much with new perspectives.
Thirty milliseconds later, when the camera fires again, the algorithm performs the same type of analysis and begins by trying to match features between the two images. This is a trial-and-error process, which can take anywhere from 50 to 250 milliseconds, depending on how dramatically the scene has changed. Once it has matched features, the algorithm can deduce from the robot's positional changes how far it has moved.
Censi and Scaramuzza's algorithm supplements the camera data with events reported by an event-based sensor, which was designed by their collaborator Tobi Delbrück of the Institute for Neuroinformatics in Zurich. The first advantage of the new algorithm is that it doesn't have to identify features: Every event is inherently a change in brightness, which is what defines a feature. And because events are reported so rapidly—every millionth of a second—the matching problem becomes much simpler. There aren't that many candidate features to consider, because the robot couldn't have moved very far.
Furthermore, the algorithm does not attempt to match all the features of an image at once. For each case, it generates a set of hypotheses about how far the robot has moved, corresponding to several candidate functions. After enough events have accumulated, it simply selects the hypothesis that most frequently matches the image.
In experiments involving a robot with a camera and an event-based sensor mounted on it, its algorithm proved to be as accurate as existing state estimation algorithms.
Arriving on board:
One of the inspirations for the new work, Censi says, was a series of recent experiments by Vijay Kumar of the University of Pennsylvania, which showed that quadcopters—robotic helicopters with four sets of rotors—could perform very agile maneuvers. But in those experiments, Kumar measured the robots' location using an array of external cameras that captured 1,000 exposures per second. Censi believes that his and Scaramuzza's algorithm would allow a quadcopter with onboard sensors to replicate Kumar's results.
Now that he and his colleagues have a reliable state estimation algorithm, Censi says, the next step is to develop a corresponding control algorithm—an algorithm that decides what to do based on the state estimates. That is the subject of an ongoing collaboration with Emilio Frazzoli, a professor of aeronautics and astronautics at MIT.
Written by Larry Hardesty, MIT News Office
