Should ML research limit submissions per author?
💡Learn how the ML community is struggling with submission volume and potential policy changes for future conferences.
⚡ 30-Second TL;DR
What Changed
Current ML submission volumes are negatively impacting peer review quality.
Why It Matters
Implementing submission limits could significantly improve the quality of peer reviews and reduce burnout among reviewers in the ML community.
What To Do Next
Review the submission policies of your target conferences and consider prioritizing quality over quantity in your upcoming research cycles.
Key Points
- •Current ML submission volumes are negatively impacting peer review quality.
- •Other fields like Security and Computer Architecture successfully use submission caps.
- •The community is debating if cultural barriers prevent similar policy adoption in ML.
- •ARR cycles are cited as evidence of the current system's strain.
🧠 Deep Insight
AI-generated analysis for this event — not the original article.
🔑 Enhanced Key Takeaways
- •Major AI conferences like NeurIPS and ICML have experimented with 'author-level' constraints, such as the 'NeurIPS 2024 submission limit' which restricted authors to a maximum of 10 submissions to curb the 'paper mill' phenomenon.
- •The Association for Computational Linguistics (ACL) implemented a 'Rolling Review' (ARR) system specifically to decouple the review process from conference deadlines, though it has faced criticism for creating a 'perpetual review' cycle that increases reviewer burnout.
- •Data from OpenReview indicates that the number of submissions to top-tier AI conferences has grown at a compound annual growth rate (CAGR) exceeding 20% over the last five years, far outpacing the growth of the qualified reviewer pool.
- •The 'reviewer-to-paper ratio' has dropped significantly, leading to a reliance on 'emergency reviewers' and junior researchers, which has been statistically correlated with higher variance in review scores and lower inter-rater reliability.
- •Some research organizations have proposed 'reviewer credit' systems or 'mandatory service quotas' as alternatives to submission caps, aiming to tie the ability to submit papers directly to the amount of high-quality reviewing performed by the author's lab.
🔮 Future ImplicationsAI analysis grounded in cited sources
⏳ Timeline
Weekly AI Recap
Read this week's curated digest of top AI events →
👉Related Updates
AI-curated news aggregator. All content rights belong to original publishers.
Original source: Reddit r/MachineLearning ↗
This is a summary, not the original. Read the source, or get the weekly briefing.
The weekly digest
One email a week. Unsubscribe anytime.