LightMem 大幅降低大模型記憶成本

🔑 增強重點摘要

• •LightMem is inspired by the Atkinson-Shiffrin model of human memory, organizing into sensory, short-term, and long-term stages with sleep-time consolidation[1][3][4].
• •On LongMemEval and LoCoMo benchmarks with GPT and Qwen backbones, it improves QA accuracy by up to 7.7% and 29.3% over baselines while reducing token usage by 38x/20.9x and API calls by 30x/55.5x[3].
• •Uses LLMLingua-2 for token pre-compression in sensory memory and hybrid attention-similarity segmentation for topic grouping[2].

🛠️ 技術深入

• •Three modules: Light1 (Sensory Memory) with pre-compression using LLMLingua-2 and hybrid topic segmentation based on attention and similarity when buffer capacity is reached[1][2].
• •Light2 (Short-term Memory): Summarizes topic-based groups into compact entries[1][2].
• •Light3 (Long-term Memory): Supports soft online inserts and offline parallel 'sleep-time' updates to decouple consolidation from inference, with configurable indexing ('embedding', 'context', 'hybrid')[1][2][6].
• •GitHub configs include options for online/offline updates, KV cache persistence, and graph memory organization for relation queries[6].

🔮 前景展望AI analysis grounded in cited sources

⏳ 時間線