MiRA Supercharges Open LLM Agents Past GPT-4

🔑 Enhanced Key Takeaways

• •MiRA utilizes a novel 'Reward-Weighted Regression' (RWR) variant that specifically addresses the high variance typically associated with RL fine-tuning in web-based environments.
• •The framework incorporates a 'Dynamic Subgoal Re-planning' mechanism that triggers automatically when the agent detects a deviation from the expected DOM (Document Object Model) state, reducing cumulative error.
• •Unlike previous WebRL approaches that rely heavily on offline trajectory datasets, MiRA demonstrates significant sample efficiency by leveraging a hybrid training loop that combines synthetic trajectory generation with real-time environment feedback.

🛠️ Technical Deep Dive

• •Architecture: Employs a dual-tower structure where a lightweight 'Planner' module generates subgoals, and a 'Policy' module (Gemma3-12B) executes actions.
• •Reward Function: Uses a dense reward signal derived from DOM-tree distance metrics and successful completion of intermediate HTML-element interactions.
• •Execution Drift Mitigation: Implements a 'State-Consistency Check' that compares the current browser state against the predicted state from the subgoal planner; if the divergence exceeds a threshold, the agent forces a re-plan.
• •Training Methodology: Utilizes a two-stage process: (1) Supervised fine-tuning on successful trajectories, followed by (2) Reward-Weighted Regression (RWR) to optimize for milestone completion.

🔮 Future ImplicationsAI analysis grounded in cited sources

⏳ Timeline