Researchers from Cornell University, Cornell Tech, and the Cornell main campus in Ithaca are using artificial intelligence (AI) to explore and generate images for studying brain visual processing. Functional Magnetic Resonance Imaging (fMRI) captured increased brain activity in response to […]
Researchers from Cornell University, Cornell Tech, and the Cornell main campus in Ithaca are using artificial intelligence (AI) to explore and generate images for studying brain visual processing. Functional Magnetic Resonance Imaging (fMRI) captured increased brain activity in response to these images, surpassing control images. This approach allows for the customization of visual models based on individual responses, enhancing the study of the brain’s reaction to visual stimulus materials. This method, providing an unbiased and systematic view of visual processing, could revolutionize neurology and therapeutic approaches.
– AI-selected and generated images were used to systematically study brain visual processing and resulted in more significant activation of targeted areas compared to control images.
– Personalized AI models proved effective in improving the brain’s response to visual stimulus materials, opening up possibilities for personalized neurological research.
– These studies open up new avenues for studying other sensory systems and exploring therapeutic applications such as modifying brain connectivity for the treatment of mental disorders.
Researchers from Cornell University, together with Cornell Tech and the Weill Cornell Medical College, demonstrated the use of AI-selected natural images and AI-generated synthetic images as neurological tools for studying brain visual processing areas. The goal is to apply a data-driven method to understand how vision is organized while removing possible biases that may arise when studying responses to a limited set of images selected by researchers.
In a study published on October 23rd in the journal Communications Biology, researchers asked volunteers to view images selected or generated based on AI models of the human visual system. The images were intended to maximize the activation of several visual processing areas. Using fMRI to record the volunteers’ brain activity, researchers found that the images significantly activated the targeted areas compared to control images.
The researchers also demonstrated that they could use the response data to adjust their vision model for each individual volunteer, so that images generated to maximally activate a specific person were more effective than images generated based on a general model.
“We believe this is a promising new approach to studying the neurology of vision,” said senior study author Dr. Amy Kuceyeski, a professor of mathematics in radiology and mathematics in neurology at the Feil Family Brain and Mind Research Institute at Weill Cornell Medical College.
The research was a collaboration with the laboratory of Dr. Mert Sabuncu, a professor of electrical and computer engineering at Cornell Engineering and Cornell Tech, as well as electrical engineering in radiology at Weill Cornell Medical College. The first author of the study was Dr. Zijin Gu, a doctoral student who was co-advised by Dr. Sabuncu and Dr. Kuceyeski during the research.
Creating an accurate model of the human visual system, partially by mapping the brain’s responses to specific images, is an ambitious goal of modern neuroscience. For example, researchers have discovered that one visual processing area can strongly activate in response to a face image, while another may respond to landscapes.
Given the risk and difficulty of directly recording brain activity with implanted electrodes, scientists must mostly rely on non-invasive methods in their efforts to achieve this goal.
The preferred non-invasive method is fMRI, which essentially measures changes in blood flow in small blood vessels in the brain – an indirect measure of brain activity – while subjects are exposed to sensory stimuli or perform cognitive or physical tasks. An fMRI machine can detect these subtle changes in three dimensions throughout the brain, with a resolution at the level of a cubic millimeter.
For their research, Dr. Kuceyeski and Dr. Sabuncu and their teams used an existing dataset that includes tens of thousands of natural images, along with corresponding fMRI responses from human subjects, to train an AI system known as Artificial Neural Networks (ANN) to model the human visual processing system.
They then used that model to predict which images in the dataset would maximize the activation of several targeted visual areas of the brain. They also combined the model with a VI-based image generator system to achieve the same goal.
“Our general idea here was to map and model the visual system in a systematic, unbiased way, essentially even using images that a person wouldn’t normally be exposed to,” said Dr. Kuceyeski.
The researchers recruited six volunteers and recorded their fMRI brain responses to these images, focusing on responses in several visual processing areas.
The results showed that both natural images and synthetic images that were predicted to be maximum activators, on average, significantly activated the targeted brain areas more than a set of control images that were either selected or generated as average activators.
This confirms the general validity of the team’s ANN-based model and suggests that even synthetic images can be useful as tools for testing and improving such models.
In the next experiment, the team used the image and fMRI response data from the initial scan to create special ANN models of the visual system for each of the six subjects. They then used these individualized models to select or generate images that would maximize each subject’s activation.
The fMRI responses to these images showed that, at least regarding synthetic images, there was greater activation of the target visual area, a region involved in face processing called FFA1, compared to responses to images based on the group model.
This result suggests that AI and fMRI can be useful for individualized modeling of the visual system, such as studying differences in visual system organization among different populations.
Scientists are now conducting similar experiments using an enhanced version of the image generator called Stable Diffusion.
The same general approach could be useful for studying other senses, such as hearing, the researchers noted.
Dr. Kuceyeski also hopes to explore the therapeutic potential of this approach.
“In principle, we could change the connectivity between two brain regions using designed stimulus materials, for example, to weaken a connection that causes excessive anxiety,” she said.
1. What is fMRI?
fMRI (Functional Magnetic Resonance Imaging) is an imaging method that uses a magnetic field to measure changes in blood flow in the brain. This allows for the investigation of brain activity and identification of associated areas and functions.
2. How does artificial intelligence work in this research?
Artificial Intelligence (AI) uses a model of the human visual system to select or generate images that will enhance the activation of targeted visual areas of the brain. This allows for the study of the brain’s response to visual stimulus materials and improves understanding of visual processing.
3. What are the therapeutic applications of this research?
This research has the potential to explore therapeutic applications, such as modifying brain connectivity, for the treatment of mental disorders. By designing stimulus materials that target specific brain regions, it may be possible to modulate brain connections to alleviate symptoms like anxiety.