Researchers in Israel have developed an AI system capable of recreating images from patterns of brain activity with substantially less individual calibration than earlier methods, opening potential avenues for communication in paralysis while intensifying questions about the future privacy of human thoughts.

By yourNEWS Media Newsroom

An artificial intelligence system developed at Israel’s Weizmann Institute of Science can reconstruct images a person is viewing by analyzing brain activity recorded through functional magnetic resonance imaging, achieving substantially greater visual detail while requiring only a fraction of the individual calibration demanded by earlier brain-decoding technologies.

The system, developed by computer scientist Michal Irani and colleagues, uses high-resolution fMRI recordings to infer both what an individual is looking at and how the objects, colors and structures in that scene are arranged. The research was reported by MIT Technology Review as scientists continue exploring whether artificial intelligence can translate patterns of neural activity into recognizable visual information.

The technology is sometimes described as “mind-reading,” although its current capability is considerably narrower: Participants are shown images while inside an fMRI scanner, and the AI attempts to reconstruct those visible images from the resulting brain activity. It does not currently extract arbitrary thoughts, dreams or imagined scenes from a person’s mind.

Previous visual brain decoders could often determine the broad meaning of what someone was seeing but struggled with spatial relationships, colors and other details. A model might correctly determine that someone was viewing a banana, for example, while generating a banana that looked markedly different from the original image.

“It wouldn’t have the same structure, the same position,” Irani said.

The new system attacks that problem by separating visual reconstruction into complementary tasks. One component predicts lower-level information such as the structure of a scene and where colors and objects appear. A second interprets higher-level semantic information — essentially what the image depicts.

Those predictions are then used to guide a generative diffusion model, which creates the final reconstruction. The combination gives the AI information about both the identity of what someone is viewing and its physical arrangement, making the resulting images more faithful to the originals.

Training a system capable of making those connections presented a major obstacle because high-quality brain imaging data are scarce and expensive to collect.

Researchers relied on an existing dataset involving eight people who had each spent extensive time inside an fMRI scanner while viewing approximately 9,000 images. Those scans provided examples linking a known visual stimulus with the patterns of brain activity that appeared while a participant viewed it.

Unlike ordinary AI development, researchers cannot simply collect millions of labeled fMRI scans from the internet. Producing each example requires a participant to enter an MRI machine, view controlled images and remain sufficiently still while researchers collect high-resolution measurements.

Irani’s team addressed that shortage by building not only a decoder capable of translating brain activity into an image, but also an encoder designed to work in the opposite direction.

The encoder learned to predict the type of brain activity an image would be expected to produce. Researchers could therefore take an ordinary picture that had never been shown to someone during an actual fMRI session, generate a simulated neural response and use that synthetic pairing to help train the decoder.

That approach dramatically enlarged the training material available to the AI without requiring thousands of additional hours inside MRI machines. About 70% of the images used during training had not originally been paired with real fMRI scans, according to the report.

The result is a system capable of recognizing visual-processing patterns that carry across different people rather than requiring an entirely separate AI model to be painstakingly trained for each individual.

That ability significantly reduced the amount of brain data researchers needed from a new user.

Earlier approaches could require about 40 hours of fMRI measurements from a person before the system was sufficiently calibrated to begin decoding that individual’s visual activity. Irani’s system reportedly achieves comparable functionality using roughly one hour.

For neuroscience research, that difference could dramatically reduce both cost and logistical difficulty.

“None of us can afford 40 hours of imaging for a new subject,” University of California, Santa Barbara neuroscientist Tommy Sprague told the Review. “It’s something like $600 to $1,000 an hour.”

The system remains far from perfect, however.

“Of course we have failures,” Irani said.

Some errors demonstrate the distinction between recognizing the broad characteristics of a scene and accurately recreating every object within it. In one test, an image of a cake was reconstructed as three sandwiches. In another, a dog sitting in a bathtub became a goat of similar coloring sitting in a bathtub.

Those examples suggest the AI successfully retained parts of the scene — including general composition, colors and the presence of an animal — while incorrectly determining the precise object or subject represented by the neural activity.

Despite those failures, Irani said comparisons with previous brain-to-image technology showed a significant improvement.

“All in all, really we outperformed the others by a significant margin,” she said.

The research has implications extending beyond the reconstruction of still photographs.

One potential application is communication technology for people who are completely paralyzed or suffer from locked-in syndrome, conditions in which an individual can remain conscious and cognitively aware while losing most or all voluntary muscle control.

Scientists have already developed brain-computer interfaces capable of translating some neural signals into speech or text. A sufficiently advanced visual or conceptual decoder could potentially provide another channel through which people unable to speak or move could express themselves.

Irani hopes future versions of the system might eventually reconstruct imagery that a person is imagining rather than physically viewing. Researchers are also interested in extending similar techniques to moving video, sound and dreams.

“That’s something we don’t have yet,” Irani said. “But we’re striving to achieve it.”

Reaching that point would require scientists to determine whether internally generated mental imagery can be decoded with the same reliability as brain responses produced by an external visual stimulus. The neural activity involved in seeing an object can overlap with activity associated with imagining it, but translating that activity into a reliable reconstruction remains a considerably more difficult problem.

University of British Columbia neuroethicist Judy Illes called the research “magnificent” and described its potential therapeutic uses as “tremendously exciting.”

For people who have lost the ability to communicate normally, increasingly capable neural decoders could eventually provide ways to convey information that today remains trapped behind severe neurological injury or paralysis.

The same advances, however, are creating a parallel debate over cognitive liberty and mental privacy.

Current fMRI technology places substantial practical limits on covert brain decoding. A person must enter a large, expensive magnetic scanner and remain inside it while the machine records activity across the brain. The system also requires individual calibration data before it can perform meaningful reconstruction.

Those constraints make the present technology fundamentally different from a device capable of remotely extracting thoughts from an unsuspecting person.

Scientists nevertheless are watching the development of other brain-monitoring technologies, particularly electroencephalography, or EEG. Unlike fMRI, which requires a large scanner, EEG measures electrical activity using electrodes placed on or near the scalp and can be incorporated into comparatively portable equipment.

If increasingly sophisticated AI models eventually extract richer information from such signals, some of the physical barriers protecting mental privacy could begin to diminish.

“The results seem very impressive,” Sprague said. “But if there’s a way to surreptitiously extract information about what you’re thinking about, then …150 years of sci-fi can come true anytime, and that’s worrisome in a lot of ways.”

That concern remains ahead of what Irani’s system can currently do. The technology reconstructs images deliberately shown to participants while their brains are being scanned; it does not secretly reveal unrestricted private thoughts.

Still, the rapid reduction in calibration time illustrates how quickly the technical barriers are changing. Moving from roughly 40 hours of individualized brain scanning to approximately one hour could make the approach considerably more practical for neuroscience experiments and future clinical research.

It also means future ethical questions may extend beyond whether a brain decoder works to who controls the resulting neural information, how consent is obtained, whether brain data can be retained or repurposed and what protections should apply if neural decoding technologies eventually become portable.

For now, the immediate advance is visual rather than telepathic: AI can use fMRI measurements to reconstruct with increasing accuracy an image that a person is actively viewing.

But researchers are already working toward the next threshold — decoding imagined images, video, sound and potentially dreams — a progression that could give people with devastating neurological conditions new ways to communicate while simultaneously forcing society to determine whether the contents of the human mind require a new category of privacy protection.

Original article