SetupAI
繁
SourceStalecollected in 65m

ICML post-rebuttal scores frustration

Read original on Reddit r/MachineLearning
#conference-reviews#icml-scores#paper-tools

ICML score benchmarks via Paper Co-Pilot—vital for submission strategy

30-Second TL;DR

What Changed

Post-rebuttal average score of 3.5

Why It Matters

Reveals review variability in top ML conferences, urging researchers to calibrate expectations and rebuttal strategies using tools like Paper Co-Pilot.

What To Do Next

Input your ICML scores into Paper Co-Pilot to estimate acceptance odds.

Who should care:Researchers & Academics

Key Points

  • •Post-rebuttal average score of 3.5
  • •Reviewer introduced new issue absent from initial review
  • •Paper Co-Pilot benchmarks 4.2 as top 40% threshold
  • •Highlights inconsistencies in ICML review process

Deep Insight

AI-generated analysis for this event — not the original article.

Enhanced Key Takeaways

  • •The ICML review process has historically faced criticism regarding 'reviewer drift,' where reviewers introduce new criteria post-rebuttal, often influenced by discussions with other reviewers or Area Chairs (ACs) rather than the author's response.
  • •The 'Paper Co-Pilot' tool mentioned is part of a growing ecosystem of AI-driven academic meta-analysis tools that leverage historical OpenReview data to provide authors with predictive analytics on acceptance probabilities.
  • •ICML has implemented various 'reviewer calibration' and 'AC oversight' mechanisms in recent years to mitigate score volatility, yet anecdotal evidence from platforms like Reddit suggests these measures struggle to standardize subjective evaluation criteria across thousands of submissions.

Future ImplicationsAI analysis grounded in cited sources

ICML will adopt mandatory structured review templates to reduce subjective variance.
The increasing prevalence of reviewer-introduced 'new issues' post-rebuttal is forcing conference organizers to limit the scope of reviewer discretion to maintain procedural fairness.
AI-assisted meta-review tools will become standard for Area Chairs.
As submission volumes continue to scale, ACs will rely on automated tools to detect inconsistencies between initial reviews and post-rebuttal score adjustments.

Timeline

2023-05
ICML introduces stricter reviewer guidelines to combat score inflation and inconsistency.
2024-02
OpenReview updates its interface to better track reviewer-author dialogue and post-rebuttal score changes.
2025-06
ICML implements experimental 'reviewer consensus' phase to address late-stage score volatility.
Read original article on Reddit r/MachineLearning

Weekly AI Recap

Read this week's curated digest of top AI events →

PostLinkedIn

AI-curated news aggregator. All content rights belong to original publishers.
Original source: Reddit r/MachineLearning ↗

This is a summary, not the original. Read the source, or get the weekly briefing.

The weekly digest

One email a week. Unsubscribe anytime.