Retrieval Boosts LLM Agent Generalization

🔑 Enhanced Key Takeaways

• •The methodology introduces 'Negative Trajectory Mining,' where the model is explicitly trained to identify and ignore failed execution paths retrieved from the memory bank, reducing error propagation.
• •The researchers utilize 'Rank-Stabilized LoRA' (rsLoRA) with a rank of 128, which prevents the catastrophic forgetting of general reasoning capabilities often seen in narrow agentic fine-tuning.
• •The pipeline demonstrates a 40% higher success rate on tasks involving 'API Drift' (unseen tool updates) by retrieving updated documentation at inference time and mapping it to fine-tuned procedural logic.
• •A 'Dual-Memory' architecture is employed, separating short-term task context (working memory) from a long-term vector database of successful multi-step trajectories (procedural memory).

🛠️ Technical Deep Dive

• •Base Models: Evaluated on Llama-3-70B and Mistral-Large-v2 architectures.
• •LoRA Configuration: Rank (r)=128, Alpha=256, targeting all linear layers (q, k, v, o, gate, up, down) to maximize expressive power for complex logic.
• •Retrieval Mechanism: HNSW (Hierarchical Navigable Small World) index using BGE-M3 embeddings for high-density semantic matching of agent states.
• •Trajectory Selection: Employs a reward-weighted similarity metric that prioritizes historical paths with the highest 'Success Score' rather than just semantic similarity to the prompt.
• •Optimization: AdamW optimizer with a 1e-5 learning rate and a linear warmup over the first 10% of training steps.

🔮 Future ImplicationsAI analysis grounded in cited sources

⏳ Timeline