π€Reddit r/MachineLearningβ’Stalecollected in 2h
ICLR 2025 Oral Paper Flaws SQL Eval
π‘Exposes eval flaw in top ICLR paperβcritical for code LLM researchers
β‘ 30-Second TL;DR
What Changed
Paper uses NL metrics for SQL eval, not execution-based.
Why It Matters
Highlights risks of flawed evals in ML conferences, urging better benchmarks for code gen tasks.
What To Do Next
Review the paper at openreview.net/forum?id=GGlpykXDCa and replicate SQL eval tests.
Who should care:Researchers & Academics
Key Points
- β’Paper uses NL metrics for SQL eval, not execution-based.
- β’Tests show 20% false positive rate in evaluation.
- β’Questions acceptance as ICLR 2025 Oral despite flaw.
π°
Weekly AI Recap
Read this week's curated digest of top AI events β
πRelated Updates
Same topic
Explore #sql-generation
Same product
More on sql-code-generation-llm
Same source
Latest from Reddit r/MachineLearning
π€
Running Qwen 35B MoE on Samsung S26 Ultra
Reddit r/MachineLearningβ’Jul 18
π€
Stereo2Spatial: Convert Stereo Music to Spatialized Binaural Mixes
Reddit r/MachineLearningβ’Jul 17
π€
Optimizing market data features for sports prediction models
Reddit r/MachineLearningβ’Jul 17
π€
Navigating acceptance rates for ACL/EMNLP/EACL short papers
Reddit r/MachineLearningβ’Jul 17
AI-curated news aggregator. All content rights belong to original publishers.
Original source: Reddit r/MachineLearning β