In a real-world application, that likely translates to energy savings of 90 to 99 percent, which could make voice control practical for relatively simple electronic devices. This includes energy-constrained devices that have to harvest power from their surroundings or go months between battery charges. Such devices form the technological backbone of what is called the "Internet of Things," or IoT.

“Voice input will become a natural interface for many wearable applications and smart devices,” says Anantha Chandrakasan, the Vannevar Bush Professor of Electrical and Computer Engineering at MIT, whose group developed the new chip. “The miniaturization of these devices will require an interface other than touch or a keyboard. It will be critical to integrate voice functionality locally to save system power consumption compared to performing this operation in the cloud.”
“I don’t think we really developed this technology for a particular application,” adds Michael Price, who led the chip’s design as a graduate student in electrical and computer engineering at MIT and now works for chipmaker Analog Devices. “We’ve tried to make the infrastructure easier to offer better trade-offs to a systems designer than they would have had with previous technology, whether it’s software or hardware acceleration.”

The Sleepyhead Wakes Up.
Today, the most effective speech recognition systems, like many other cutting-edge artificial intelligence systems, are based on neural networks—virtual networks of simple information processors modeled after the human brain. Much of the circuitry in new chips is dedicated to implementing speech recognition networks as efficiently as possible.
But even the most energy-efficient speech recognition system could quickly drain a device's battery if run continuously. Therefore, the chip also includes a simpler "speech activity detection" circuit that monitors ambient noise to determine if it might be speech. If so, the chip triggers the larger, more complex speech recognition circuit.
In fact, for experimental purposes, the researchers' chip had three different speech activity detection circuits, with varying degrees of complexity and, consequently, different power demands. Which circuit is most efficient depends on the context, but in tests simulating a wide range of conditions, the most complex of the three circuits offered the greatest energy savings for the system as a whole. Although it consumed almost three times more power than the simplest circuit, it generated far fewer false positives; simpler circuits often negate their energy savings by triggering the rest of the chip.
A typical neural network consists of thousands of processing "nodes" capable of performing only simple calculations but densely interconnected. In the type of network commonly used for speech recognition, the nodes are arranged in layers. Speech data is fed into the bottom layer of the network, whose nodes process it and pass it on to the nodes in the next layer, whose nodes process it and pass it on to the next layer, and so on. The output from the top layer indicates the probability that the speech data represents a particular speech sound.
A speech recognition network is too large to fit on a single chip's internal memory, which is a problem because using off-chip data is much more intensive than retrieving it from local storage. So the MIT researchers' design focuses on minimizing the amount of data the chip has to retrieve from off-chip memory.
Bandwidth Management:
A node in the middle of a neural network might receive data from a dozen other nodes and transmit data to another dozen. Each of those two dozen connections has an associated "weight," a number indicating how important the data sent through it is to the receiving node's calculations. The first step in minimizing the new chip's memory bandwidth is to compress the weights associated with each node. The data is decompressed only after it's brought onto the chip.
The chip also exploits the fact that, with speech recognition, wave after wave of data must pass through the network. The incoming audio signal is divided into 10-millisecond increments, each of which must be evaluated separately. The MIT researchers' chip carries only one neural network node at a time, but passes the data from 32 consecutive 10-millisecond increments through it.
If a node has a dozen outputs, then the 32 passes produce 384 output values, which the chip stores locally. Each of these must be coupled with 11 other values ​​when fed to the next layer of nodes, and so on. Therefore, the chip ends up needing a substantial built-in memory circuit for its intermediate calculations. But it only pulls one compressed node at a time from off-chip memory, keeping its power requirements low.
The research was funded through Project Qmulus, a joint venture between MIT and Quanta Computer, and the chip was prototyped through the Taiwan Semiconductor Manufacturing Company's satellite university program.
###
Author: Larry Hardesty, MIT News Office