🤖Reddit r/MachineLearning•Freshcollected in 49m
OpenReview Comments Disappear, Raising Review Transparency Concerns
💡A missing AC comment and rebuttal highlight why AI researchers need auditable peer-review records.
⚡ 30-Second TL;DR
What Changed
An Area Chair’s comment summarizing reviewer questions and weaknesses reportedly disappeared.
Why It Matters
For AI researchers, unexplained changes to public review records can complicate rebuttal tracking and post-publication accountability. If such behavior is widespread, conferences may need clearer policies for editing, hiding, or retaining review discussions.
What To Do Next
Save screenshots and OpenReview activity records for every rebuttal, then contact the conference chairs or OpenReview support if a comment unexpectedly disappears.
Who should care:Researchers & Academics
Key Points
- •An Area Chair’s comment summarizing reviewer questions and weaknesses reportedly disappeared.
- •The authors’ response addressing those questions disappeared at the same time.
- •The incident raises concerns about review transparency and whether deleted comments affect how rejection decisions are perceived.
- •The post does not establish whether the disappearance resulted from platform behavior, moderation, or a conference workflow change.
🧠 Deep Insight
AI-generated analysis for this event.
🔑 Enhanced Key Takeaways
- •OpenReview utilizes a 'visibility' and 'readers' permission system where Area Chairs (ACs) and Program Chairs (PCs) can restrict comment visibility, sometimes leading to confusion when content is hidden rather than deleted.
- •The platform has historically faced criticism regarding the 'meta-review' process, where AC summaries can be edited or retracted by PCs without a public audit trail, complicating accountability.
- •OpenReview's architecture allows for 'private' comments that are only visible to specific roles (e.g., ACs and PCs), which can be mistaken for public comments disappearing if the user's permission level changes.
- •Conference organizers often have the authority to redact or hide comments that violate codes of conduct or contain sensitive information, a process that is frequently opaque to the authors and reviewers involved.
- •Recent updates to the OpenReview API have introduced more granular control over comment threads, which has occasionally resulted in UI bugs where threads appear to vanish before re-syncing with the database.
📊 Competitor Analysis▸ Show
| Feature | OpenReview | CMT (Microsoft) | SoftConf (START) |
|---|---|---|---|
| Transparency | High (Public by default) | Low (Closed) | Low (Closed) |
| Reviewer Anonymity | Double-blind supported | Double-blind supported | Double-blind supported |
| Public Discussion | Yes | No | No |
| Customization | High (API-driven) | Moderate | Low |
🛠️ Technical Deep Dive
- OpenReview is built on a custom platform utilizing a document-based database model where every comment, review, and meta-review is stored as a JSON object with specific 'readers' and 'writers' fields.
- The 'readers' field controls access control lists (ACLs); if an AC changes the reader group of a comment to exclude authors, the comment effectively disappears from the authors' view.
- The platform uses a versioning system for comments, but historical versions are often not exposed to the end-user, making it difficult to track if a comment was deleted or merely updated to a restricted state.
- The system relies on a centralized event log for moderation actions, but this log is typically restricted to Program Chairs and is not accessible to the general research community.
🔮 Future ImplicationsAI analysis grounded in cited sources
OpenReview will implement a public 'Edit History' feature for meta-reviews.
Increasing pressure from the research community regarding transparency will likely force the platform to expose versioning metadata to maintain trust.
Conference organizers will adopt standardized 'Redaction Logs' for transparency.
To mitigate concerns about arbitrary censorship, major AI conferences will likely mandate public logs explaining why specific comments were removed or hidden.
⏳ Timeline
2013-01
OpenReview is launched by Andrew McCallum and the UMass Amherst IESL lab.
2016-05
ICLR adopts OpenReview for the first time, pioneering open peer review in AI.
2020-09
OpenReview introduces enhanced API support for conference-specific workflows.
2023-11
Platform updates include more granular permission settings for ACs and PCs.
📰
Weekly AI Recap
Read this week's curated digest of top AI events →
👉Related Updates
Same topic
Explore #peer-review
Same product
More on openreview
Same source
Latest from Reddit r/MachineLearning
🤖
Do Honest Limitations Hurt ML Papers?
Reddit r/MachineLearning•Aug 14
🤖
Does Theory Still Guide Machine Learning?
Reddit r/MachineLearning•Aug 14
🤖
Designing an Adaptive Question Recommendation Engine
Reddit r/MachineLearning•Aug 14
🤖
OncoThresh Brings Clinical-Threshold Evaluation to Oncology AI
Reddit r/MachineLearning•Aug 14
AI-curated news aggregator. All content rights belong to original publishers.
Original source: Reddit r/MachineLearning ↗