ICRL: Training LLMs to Internalize Self-Critique
๐กLearn how to train LLMs to self-correct without external critique, boosting performance while reducing token costs.
โก 30-Second TL;DR
What Changed
Jointly trains a solver and critic from a shared backbone to enable autonomous self-improvement.
Why It Matters
This research offers a path to reduce reliance on expensive external critique loops in agentic workflows. It enables more efficient, self-correcting LLM agents that improve over time through internalizing feedback.
What To Do Next
Clone the ICRL repository and test the training framework on your specific domain tasks to see if it reduces the need for multi-turn critique prompts.
Key Points
- โขJointly trains a solver and critic from a shared backbone to enable autonomous self-improvement.
- โขUses a distribution-calibration re-weighting ratio to prevent dependency on critique-conditioned behavior.
- โขAchieved significant performance gains on agentic and mathematical tasks using Qwen3 backbones.
- โขThe 8B critic model demonstrates performance comparable to 32B models with lower token overhead.
Weekly AI Recap
Read this week's curated digest of top AI events โ
๐Related Updates
AI-curated news aggregator. All content rights belong to original publishers.
Original source: ArXiv AI โ