What does the inside of a mouse's head actually look like while it watches a movie? Researchers at University College London say they now have a rough answer, having built a system that reconstructs 10-second video clips using nothing but electrical signals recorded from neurons in a mouse's visual cortex.
The study, led by Dr. Joel Bauer alongside Troy Margrie and Claudia Clopath, was published in eLife and relies on what the team calls a dynamic neural encoding model, which maps how individual neurons respond as a mouse watches specific movie frames. Crucially, the model also factors in what the mouse's body is doing at the time, including its movements and pupil dilation, since those physical cues shape how the animal's brain is processing what it sees. "Using this approach, we were able to achieve high-quality reconstructions of 10-second video clips," Bauer said, according to a summary of the findings published by ScienceDaily.
The researchers tested the system on footage the model had never encountered during training, and it was still able to infer a reasonable approximation of what the mouse was looking at, suggesting it had learned something generalizable about how the visual cortex encodes the world rather than simply memorizing familiar clips.
Vision is not simply a camera-like recording process.
Dr. Joel Bauer, University College London
That distinction is the real point of the research. The reconstructed videos do not perfectly match the footage the mice were actually shown — and that gap, the researchers argue, is a feature rather than a flaw. The brain appears to continuously reinterpret and reshape raw visual input on the way to conscious perception, and studying exactly where and how that reshaping happens could offer clues about the basic principles behind how any brain, mouse or human, turns light hitting the retina into a coherent sense of the world.
The work builds on earlier reconstructions of mouse-viewed footage from UCL's Margrie lab, refined here with single-cell recordings that the team says offer a sharper picture of the brain's internal representations than earlier, cruder methods. Broader applications remain a long way off, but the researchers say the approach could eventually help identify how specific visual cues distort neural representations, a step toward understanding perception disorders in which the brain's version of reality drifts further from what is actually out there.