📄ArXiv AI•較早收集於 23h
Sci-PRM：用於科學推理的工具感知獎勵模型
💡科學推理領域的突破，利用工具感知獎勵來解決複雜領域中的幻覺問題。
⚡ 30-Second TL;DR
有什麼變化
引入了包含 Chain-of-Tool 軌跡的大規模資料集 SCIPRM70K。
為什麼重要
這項研究為將外部工具整合至 LLM 以執行科學任務提供了強大的框架，有望減少生物、化學與物理應用中的錯誤。它為突破專業推理領域的性能瓶頸提供了一條路徑。
下一步行動
如果您正在開發科學 AI 代理，請根據 Sci-PRM 的方法評估您目前的推理流程，看看工具感知監督是否能降低模型的幻覺率。
誰應關注：Researchers & Academics
關鍵要點
- •引入了包含 Chain-of-Tool 軌跡的大規模資料集 SCIPRM70K。
- •針對工具選擇、執行準確性與結果解釋提供細粒度監督。
- •透過 Best-of-N 選擇實現有效的測試時擴展（test-time scaling）。
- •緩解強化學習中的優勢消失問題，突破現有性能瓶頸。
🧠 深度解析
Web-grounded analysis with 11 cited sources.
🔑 增強重點摘要
- •Sci-PRM extends the application of Process Reward Models (PRMs) to complex scientific domains such as biology, chemistry, and physics, areas largely unexplored by previous PRMs that primarily focused on mathematical reasoning.
- •The SCIPRM70K dataset, integral to Sci-PRM, is a novel, large-scale collection of 70,000 'Chain-of-Tool' trajectories that explicitly interleave reasoning steps with the execution and outputs of scientific tools, providing a rich source of fine-grained supervision.
- •Sci-PRM demonstrates superior performance over state-of-the-art proprietary models like GPT-5-Mini, particularly in steps involving tool-calling, where GPT-5-Mini shows substantial degradation in accuracy.
- •The model's training incorporates an automated two-stage step-level labeling pipeline that combines execution-based technical validation with Monte Carlo Tree Search (MCTS)-based logical verification to assign reliable binary labels to each reasoning and tool-use step.
- •Sci-PRM is publicly available on GitHub, fostering further research and innovation in tool-aware scientific reasoning.
📊 競品分析▸ Show
| Feature / Model | Sci-PRM | OpenAI's 2023 PRM (Math) | Process Q-value Model (PQM) | DeepSeek's PRM (Math) |
|---|---|---|---|---|
| Primary Domain | Scientific reasoning (biology, chemistry, physics) | Mathematical reasoning | Complex reasoning & decision-making | Mathematical problems |
| Key Innovation | Tool-aware process reward, SCIPRM70K dataset, mitigates advantage disappearance in RL, high accuracy on tool-calling steps. | Demonstrated substantial outperformance over outcome-based models in competition math. | Redefines PRM as Markov Decision Process, optimizes Q-value rankings with comparative loss. | Trained with Math-Shepherd method for instruction-following models. |
| Supervision Granularity | Fine-grained supervision on tool selection, execution, interpretation at each step. | Step-level scoring for sound logic. | Captures interdependencies among reasoning states. | Step-by-step solutions for math problems. |
| Performance Highlight | Significantly enhances foundation models, outperforms GPT-5-Mini on tool-calling steps. | Substantially outperformed outcome-based models. | Outperforms classification-based PRMs. | Used to train models with high benchmark results. |
| Availability | Publicly available (GitHub). | Not explicitly stated as publicly available as a standalone model. | Framework proposed in research paper. | Integrated into DeepSeek's open-weight models. |
🛠️ 技術深入
- Sci-PRM is an efficient reward model that assigns a scalar quality score to each step of a partially generated reasoning trajectory, conditioned on the full history up to that step.
- It is trained on the SCIPRM70K dataset, which consists of 'Chain-of-Tool' trajectories that interleave natural language reasoning with explicit scientific tool calls, tool outputs, and grounded interpretations.
- The dataset's step-level labels are generated via an automated two-stage pipeline: first, execution-based technical validation assesses the feasibility of tool calls and provides observations; second, MCTS-based logical verification assigns binary labels (correct/incorrect) to each step.
- Sci-PRM is designed to enhance foundation models by enabling test-time scaling through Best-of-N trajectory selection, where it scores individual steps and aggregates these into a trajectory score to select the most reliable path.
- It functions as a dense, learned reward signal for Reinforcement Learning (RL) training of tool-using Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs), providing process-sensitive feedback to mitigate advantage disappearance.
🔮 前景展望AI analysis grounded in cited sources
Sci-PRM will accelerate the development of more reliable and verifiable AI systems in scientific research.
By providing fine-grained, tool-aware supervision and mitigating advantage disappearance in RL, it directly addresses key limitations in current AI reasoning for complex scientific problems.
The SCIPRM70K dataset will become a benchmark for developing and evaluating tool-augmented scientific reasoning models.
As a large-scale, publicly available dataset with 'Chain-of-Tool' trajectories, it provides a crucial resource for training and comparing future models in this specialized domain.
The methodology of integrating execution-based and logical verification for step-level labeling will be adopted by other domain-specific reward model developments.
This automated pipeline offers a scalable way to create high-quality supervision for complex, multi-step reasoning processes involving external tools.
⏳ 時間線
2023-11
DeepSeek-LLM series released, utilizing a Process Reward Model for math problems.
2023
Process Reward Models (PRMs) gained prominence following OpenAI's research on solving competition mathematics problems.
2025-01
Process Q-value Model (PQM) framework introduced for complex reasoning.
2025-03
Chain-of-Tools (CoTools) framework for LLM tool use discussed.
2026-02
CoLT (Chain of Latent Tool Calls) framework proposed for accelerated latent reasoning.
2026-06
Sci-PRM paper submitted to arXiv and accepted by KDD 2026 AI4Science Track.
📎 來源 (11)
Factual claims are grounded in the sources below. Forward-looking analysis is AI-generated interpretation.
📰
AI 週報
閱讀本週精選 AI 大事摘要 →
👉相關動態
AI 策展新聞聚合。所有內容版權歸原始發布者所有。
原始來源： ArXiv AI ↗