ICML Poll: Policy B Higher Scores

Post LinkedIn

🤖Read original on Reddit r/MachineLearning

#conference-review #poll-results #review-confidenceicml-2026

💡100+ responses reveal Policy B score edge in ICML reviews—check bias impact

⚡ 30-Second TL;DR

What Changed

Policy B: 41 responses, mean score 3.43 (SD 0.63)

Why It Matters

Highlights potential review biases by policy, informing ICML submission strategies. Inverse score-confidence trend suggests LLM-assisted reviews may lower confidence.

What To Do Next

Fill out the ICML policy poll to contribute before rebuttal deadline.

Who should care:Researchers & Academics

🧠 Deep Insight

AI-generated analysis for this event.

🔑 Enhanced Key Takeaways

•The 'Policy A' vs 'Policy B' terminology refers to the 2026 ICML review process, where the conference implemented a split-blind review policy experiment to test the impact of different reviewer anonymity and disclosure guidelines on scoring distributions.
•Statistical analysis of the Reddit dataset suggests that the higher reviewer confidence in Policy A may be correlated with the disclosure of author identities, which some community members argue introduces implicit bias compared to the double-blind structure of Policy B.
•ICML organizers have officially stated that these community-sourced polls are not representative of the final acceptance rates, as they lack access to the full metadata, including reviewer expertise calibration and area chair adjustments.

🔮 Future ImplicationsAI analysis grounded in cited sources

ICML will adopt a unified review policy for 2027 based on the 2026 experiment results.

The conference organizers have committed to using the 2026 split-policy data to determine the long-term standard for reviewer anonymity and disclosure.

Reviewer confidence metrics will be weighted differently in future ICML acceptance algorithms.

The discrepancy between score averages and confidence levels in the 2026 experiment has highlighted a need to better calibrate reviewer expertise against score variance.