Source虎嗅•Stalecollected in 53m
AI Key-Value Memory Mirrors Human Brain Logic
Discover how the brain's memory logic validates RAG and Transformer architectures in this 2025 Neuron study.
30-Second TL;DR
What Changed
AI key-value memory uses 'keys' for indexing and 'values' for storage.
Why It Matters
This research provides a unified framework for understanding memory organization in both biological and artificial intelligence systems.
What To Do Next
Optimize your RAG retrieval pipeline by treating the embedding vector as a 'key' and the document chunk as a 'value' to improve indexing precision.
Who should care:Researchers & Academics
Key Points
- •AI key-value memory uses 'keys' for indexing and 'values' for storage.
- •The hippocampus acts as an indexer (key), while the neocortex acts as a storage warehouse (value).
- •Forgetting is often a failure of the retrieval index (key) rather than the loss of stored information (value).
Deep Insight
AI-generated analysis for this event — not the original article.
Enhanced Key Takeaways
- •The key-value memory framework allows for separate optimization of keys (for discriminability in retrieval) and values (for fidelity in storage), addressing distinct computational demands not possible with classical similarity-based retrieval models. [10, 22]
- •In AI, particularly Large Language Models (LLMs) based on the Transformer architecture, key-value memory is implemented as a 'KV-cache' to store intermediate attention outputs, significantly speeding up inference by avoiding recomputing past tokens for each new token generated. [5, 15, 18, 19, 26]
- •Beyond simple storage, brain-inspired AI memory systems are exploring dual-memory architectures with varying learning rates, mimicking the fast hippocampal learning and slow cortical integration to manage the stability-plasticity dilemma in AI. [2]
- •Research using generative AI models has demonstrated how the neocortex, in conjunction with the hippocampus, forms efficient conceptual representations, enabling the AI to not only recall past experiences but also generate novel scenarios for imagination and planning. [6, 13]
Technical Deep Dive
- Key-Value Memory Networks (KV-MemNNs): In machine learning, these networks store explicit (key, value) pairs where keys capture contextual clues for addressing and values are precise answer fragments. Addressing typically uses learned projections and softmax attention, potentially in multiple 'hops' to update a query state. [1]
- Transformer Architecture's QKV Mechanism: At the core of the attention mechanism in Transformers are Query (Q), Key (K), and Value (V) vectors. These are linear projections of the input data (embeddings). The Query vector represents the element of interest, Key vectors are used to compute relevance to the query (often via dot product), and Value vectors store the actual information. Attention scores, derived from Query-Key interactions, are then used as weights to combine the Value vectors, allowing the model to dynamically focus on relevant parts of the input. [3, 7, 8, 16, 17]
- KV-Cache in LLMs: For generative AI, especially LLMs, the KV-cache stores the Key and Value tensors from previous steps in each layer. This prevents redundant computation during the 'decode' stage of text generation, where each new token needs to attend to all preceding tokens. The cache grows linearly with sequence length and layers, posing memory challenges. [5, 15, 19, 26]
- KV-Cache Management Strategies: To address memory pressure from growing KV-caches, strategies include per-session KV cache, sliding window attention (keeping only recent tokens), KV quantization (compressing tensors), and disk/host-backed caching (storing parts on CPU or disk). [15]
- Memory³ Architecture: A proposed system that externalizes 'concrete knowledge' (specific facts) from a model's parameters into explicit, sparse attention key-value pairs. A 'memory generator' encodes, filters, and sparsifies key tokens. Retrieval involves vector search (e.g., using Faiss) to find relevant memory units, which are then directly concatenated into the model's attention layer, bypassing the need for the model to 'read' raw text. [14]
Future ImplicationsAI analysis grounded in cited sources
AI systems will achieve more robust and human-like long-term memory capabilities.
By adopting brain-inspired dual-memory architectures and explicit key-value memory systems, AI can better balance rapid learning of new information with the stable integration of existing knowledge, reducing issues like catastrophic forgetting. [2, 24]
The efficiency and scalability of large language models will significantly improve through advanced KV-cache management and externalized memory architectures.
Optimizing KV-cache strategies (e.g., quantization, sliding windows) and externalizing concrete knowledge into sparse key-value stores will reduce memory footprint and computational overhead, enabling larger context windows and more cost-effective inference. [5, 14, 15, 26]
AI will exhibit enhanced imaginative and planning abilities by leveraging conceptual representations derived from brain-inspired memory consolidation.
Models that simulate hippocampus-neocortex interaction to form conceptual representations can not only reconstruct past events but also generate novel scenarios, leading to more sophisticated planning and creative AI applications. [6, 13]
Timeline
2017
Transformer architecture introduced, utilizing attention mechanisms with Query, Key, and Value components. [8, 16]
2023-12
A study reveals striking similarities between the memory processing of AI models (specifically Transformer's gatekeeping) and the hippocampus's NMDA receptor in memory consolidation. [9, 12]
2024-01
UCL researchers publish a study using generative AI to model hippocampus-neocortex interaction, providing insights into memory, imagination, and planning. [6, 13]
2025-01
The paper 'Key-value memory in the brain' by Gershman et al. is accepted to Neuron, formalizing the concept of key-value memory and its biological plausibility. [10, 22]
2025-03
University of Chicago neuroscientists publish a study demonstrating that adapting brain-inspired algorithms can dramatically improve AI's ability to multitask and avoid catastrophic forgetting. [24]
2025-08
The 'Memory³' architecture is proposed, focusing on externalizing concrete knowledge as sparse key-value pairs for large language models to enhance efficiency. [14]
- 2017Transformer architecture introduced, utilizing attention mechanisms with Query, Key, and Value components. [8, 16]
- 2023-12A study reveals striking similarities between the memory processing of AI models (specifically Transformer's gatekeeping) and the hippocampus's NMDA receptor in memory consolidation. [9, 12]
- 2024-01UCL researchers publish a study using generative AI to model hippocampus-neocortex interaction, providing insights into memory, imagination, and planning. [6, 13]
- 2025-01The paper 'Key-value memory in the brain' by Gershman et al. is accepted to Neuron, formalizing the concept of key-value memory and its biological plausibility. [10, 22]
- 2025-03University of Chicago neuroscientists publish a study demonstrating that adapting brain-inspired algorithms can dramatically improve AI's ability to multitask and avoid catastrophic forgetting. [24]
- 2025-08The 'Memory³' architecture is proposed, focusing on externalizing concrete knowledge as sparse key-value pairs for large language models to enhance efficiency. [14]
Weekly AI Recap
Read this week's curated digest of top AI events →
AI-curated news aggregator. All content rights belong to original publishers.
Original source: 虎嗅 ↗
This is a summary, not the original. Read the source, or get the weekly briefing.
The weekly digest
One email a week. Unsubscribe anytime.