Search

Few direct matches — filled in with the latest updates.

Tag: #rater-effects1 results

Fixing Rater Bias in AI Evals with IRT

Fixing Rater Bias in AI Evals with IRT

This paper integrates psychometric rater models into AI evaluations to correct systematic errors from human raters. It employs Item Response Theory, particularly the multi-faceted Rasch model, to disentangle true output quality from rater effects like severity and centrality. Applied to OpenAI's summarization dataset, it delivers adjusted quality scores and rater diagnostics for more reliable AI assessments.

Anthropic Targets a Record-Breaking IPO

Anthropic Targets a Record-Breaking IPO

Anthropic reportedly expects its IPO to match or exceed SpaceX's record $75 billion raise and could file publicly as soon as the end of this month. The company reportedly recorded an almost $42 billion net loss in 2025, about five times the previous year's figure.

The Next Web (TNW)Media1h ago#ipo#frontier-ai#fundraising