"Having options can help in decision-making. Even just seeing that there is uncertainty in a medical image can influence someone's decisions, so it's important to take this uncertainty into account," says Marianne Rakic, a doctoral candidate in computer science at MIT.
Rakic is the lead author of a paper published with other researchers from MIT, the Broad Institute of MIT and Harvard, and Massachusetts General Hospital, which introduces a new AI tool capable of capturing uncertainty in a medical image.
Known as Tyche (named after the Greek god of chance), the system offers multiple plausible segmentations that highlight slightly distinct areas of a medical image. The user can specify how many options Tyche offers and select the one most appropriate for their purpose.
And more importantly, Tyche can tackle new segmentation tasks without needing to be retrained. Training is a data-intensive process that involves showing a model many examples and requires extensive machine learning expertise.
Because it requires no retraining, Tyche could be easier to use than other methods for physicians and biomedical researchers. It could be applied "out of the box" to a variety of tasks, from identifying lesions on a chest X-ray to locating abnormalities on a brain MRI.
Ultimately, this system could improve diagnoses or aid in biomedical research by drawing attention to potentially crucial information that other AI tools might miss.
"Ambiguity hasn't been studied enough. If your model overlooks a node that three experts say is there and two say it isn't, you should probably pay attention to that," adds Adrian Dalca, lead author, assistant professor at Harvard Medical School and MGH, and research scientist at MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL).
Among its co-authors are Hallee Wong, a graduate student in Electrical and Computer Engineering; José Javier González Ortiz, PhD (23); Beth Cimini, Associate Director of Bioimaging Analysis at the Broad Institute; and John Guttag, Dugald C. Jackson Professor of Computer Science and Electrical Engineering. Rakic will present Tyche at the IEEE Conference on Computer Vision and Pattern Recognition, where Tyche has been selected as a featured project.
Addressing ambiguity
Artificial intelligence systems for medical image segmentation often use neural networks. Based on the human brain, neural networks are machine learning models composed of many interconnected layers of nodes, or neurons, that process data.
After speaking with collaborators at the Broad Institute and MGH who use these systems, the researchers realized that there are two major problems limiting their effectiveness. The models cannot capture uncertainty and must be retrained even for a slightly different segmentation task.
Some methods attempt to overcome one of the obstacles, but tackling both problems with a single solution has proven particularly difficult, says Rakic.
"If you want to account for ambiguity, you often have to use an extremely complicated model. With the method we propose, our goal is to make it easy to use with a relatively small model so that it can make predictions quickly," he says.
The researchers created Tyche by modifying a simple neural network architecture.
First, the user provides Tyche with some examples that illustrate the segmentation task. For example, the examples could include several images of lesions in a cardiac MRI that have been segmented by different human experts, so that the model can learn the task and see that ambiguity exists.
The researchers discovered that just 16 example images, called the "context set," are enough for the model to make good predictions, but there is no limit to the number of examples that can be used. The context set allows Tyche to solve new tasks without needing to be retrained.
To enable Tyche to capture uncertainty, the researchers modified the neural network so that it would produce multiple predictions based on an input medical image and the set of contexts. They adjusted the network's layers so that, as data passed from one layer to the next, the candidate segmentations produced at each step could "talk" to each other and to the examples in the context set.
In this way, the model can ensure that the candidate segmentations are all slightly different, but still solve the task.
"It's like rolling dice. If your model can roll a two, a three, or a four, but doesn't know you already have a two and a four, then either of those can appear again," he says.
They also modified the training process to reward him by maximizing the quality of his best prediction.
If the user requests five predictions, they can ultimately see the five medical image segmentations produced by Tyche, although one may be better than the others.
The researchers have also developed a version of Tyche that can be used with a pre-trained medical image segmentation model. In this case, Tyche allows the model to generate multiple candidates through slight image transformations.
Better and faster predictions
When researchers tested Tyche with annotated medical image datasets, they found that its predictions captured the diversity of human annotators and that its best predictions were better than those of any of the reference models. Furthermore, Tyche was faster than most of the models.
"Producing multiple candidates and ensuring they are different from each other really gives you an advantage," Rakic says.
The researchers also noted that Tyche could outperform more complex models trained on a large, specialized dataset.
Looking ahead, they plan to experiment with a more flexible set of contexts, perhaps including text or various image types. They also want to explore methods to improve Tyche's poor predictions and enhance the system so it can recommend the best targeting candidates.
This research is funded, in part, by the National Institutes of Health, the Eric and Wendy Schmidt Center at the Broad Institute of MIT and Harvard, and Quanta Computer.
###
Written by Adam Zewe, MIT News
