Other designs use lower-resolution sensors spread across the finger, but these don't capture as much detail, often requiring multiple grasps.

Instead, the MIT team built a robotic finger with a rigid skeleton encased in a soft outer layer that incorporates multiple high-resolution sensors beneath its transparent "skin." The sensors, which use a camera and LEDs to gather visual information about an object's shape, provide continuous detection along the entire length of the finger. Each finger simultaneously captures a wealth of data about many parts of an object.

With this design, the researchers built a three-fingered robotic hand capable of identifying objects after a single grasp, with 85% accuracy. The rigid skeleton makes the fingers strong enough to pick up a heavy object, such as a drill, while the soft skin allows them to safely grasp a flexible object, such as an empty plastic bottle, without crushing it.

These soft-and-rigid fingers could be especially useful in a home assistance robot designed to interact with elderly people. The robot could lift a heavy object from a shelf with the same hand it uses to help the elderly person bathe.

"Having both soft and rigid elements is very important in any hand, but so is the ability to perform extensive sensing over a large area, especially if we want to consider performing very complex manipulation tasks like those our own hands can do. Our goal with this work was to combine all the things that make our human hands so good into a robotic finger capable of performing tasks that other robotic fingers currently cannot," says mechanical engineering student Sandra Liu, co-author of a research paper on the robotic finger.

Liu wrote the paper with lead co-author and mechanical engineering student Leonardo Zamora Yañez and their advisor, Edward Adelson, John and Dorothy Wilson Professor of Vision Science in the Department of Cognitive and Brain Sciences and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL). The research will be presented at the RoboSoft Conference.

A Human-Inspired Finger:

The robotic finger consists of a rigid, 3D-printed endoskeleton that is placed in a mold and wrapped in a transparent silicone "skin." Manufacturing the finger in a mold eliminates the need for fasteners or adhesives to hold the silicone in place.

The researchers designed the mold with a curved shape so that the robotic fingers are slightly curved when at rest, just like human fingers.

"Silicone wrinkles when it's bent, so we thought that if we had the finger molded in this curved position, when you bend it further to grasp an object, it wouldn't induce as many wrinkles. Wrinkles are good in a way—they can help the finger glide across surfaces very smoothly and easily—but we didn't want wrinkles that we couldn't control," says Liu.

Each finger's endoskeleton contains a pair of detailed touch sensors, known as GelSight sensors, embedded in the upper and middle sections, beneath the transparent skin. The sensors are positioned so that the camera ranges slightly overlap, providing continuous detection along the entire length of the finger.

The GelSight sensor, based on pioneering technology from the Adelson group, consists of a camera and three colored LEDs. When the finger grasps an object, the camera captures images while the colored LEDs illuminate the skin from within.

From the illuminated contours that appear on the soft skin, an algorithm performs retrospective calculations to map the contours on the surface of the grasped object. The researchers trained a machine learning model to identify objects from the raw camera image data.

While refining the finger's manufacturing process, the researchers encountered several obstacles.

First, the silicone tends to peel away from surfaces over time. Liu and his colleagues discovered they could limit this peeling by adding small curves along the hinges between the endoskeleton's joints.

When the finger bends, the flexing force of the silicone is distributed along the small curves, reducing stress and preventing detachment. They also added folds at the joints so the silicone wouldn't flatten as much when the finger bends.

While troubleshooting their design, the researchers realized that the silicone's wrinkles prevent the skin from tearing.

"The usefulness of wrinkles was an accidental discovery on our part. When we synthesized them on the surface, we discovered that they made the finger more durable than we expected," he explains.

Get a good grip

Once the design was perfected, the researchers built a robotic hand with two fingers arranged in a Y shape and a third finger as an opposable thumb. The hand captures six images when it grasps an object (two from each finger) and sends them to a machine learning algorithm that uses them to identify the object.

Since the hand has touch sensors on all the fingers, it can collect a lot of touch data from a single grip.

"Although we have many sensors in our fingers, perhaps adding a palm with sensors would help to better distinguish tactile objects," says Liu.

In the future, the researchers also want to improve the hardware to reduce wear and tear on the silicone over time and add more actuators to the thumb so it can perform a wider variety of tasks.

This work has been partly funded by the Toyota Research Institute, the Office of Naval Research, and the SINTEF BIFROST project.

Author: Adam Zewe, MIT News Office