A recent study by UCL researchers published in Nature Human Behavior and funded by Wellcome explores how recent advances in generative AI are helping to explain how memories allow us to learn about the world, relive old experiences, and construct entirely new experiences for imagination and planning.
The study uses a generative neural network model to simulate how neural networks in the brain learn from and recall a series of events represented by simple scenes.
By representing networks applicable to the hippocampus and the neocortex in the model, the study investigates how both parts of the brain collaborate in memory, imagination, and planning.
Eleanor Spens, the lead author, explained, „Recent advances in generative networks used in AI demonstrate how information can be extracted from experiences, allowing us to both remember a specific experience and flexibly imagine what might happen.“ She added, „Remembering means picturing the past based on concepts and combining some stored details with our expectations about what might have happened.“
According to the study, the generative neural networks suggest how our brains help us recognize patterns from past experiences that can be used to make predictions by replaying memories in a resting state.
The researchers exposed the model to 10,000 images of simple scenes. The hippocampus network quickly encoded each experienced scene. Later, the scenes were replayed to train the generative neural network residing in the neocortex.
The neocortical network learned to communicate the activities of thousands of input neurons (neurons receiving visual information) representing each scene through smaller layers of neurons (the smallest containing only 20 neurons) to simulate scenes as activity patterns in thousands of output neurons (neurons predicting visual information).
This allowed the neocortical network to learn efficient „conceptual“ representations of the scenes capturing their meaning (e.g., the arrangement of walls and objects) – enabling it to recall old scenes and generate entirely new ones.
Consequently, the hippocampus was able to encode the meaning of new scenes presented to it without having to encode every single detail, focusing its resources instead on encoding unique features that the neocortex could not reproduce—such as new types of objects.
The model illustrates how the neocortex gradually acquires conceptual knowledge and together with the hippocampus, enables events to be „relived“ by reconstructing them in our minds.
The model also explains how new events can be generated during imagination and planning for the future and why existing memories often contain „essential“ distortions—where unique features are generalized and remembered more like the features of previous events.
„The way memories are reconstructed, rather than being truthful records of the past, shows us how the meaning or core of an experience is recombined with unique details and how this can lead to biases in the way we remember things.“
– Neil Burgess, lead author, Professor, UCL Institute of Cognitive Neuroscience and UCL Queen Square Institute of Neurology
University College London
Spens, E. & Burgess, N. (2024). A generative model of memory construction and consolidation. Nature Human Behavior. doi.org/10.1038/s41562-023-01799-z.