Few direct matches — filled in with the latest updates.
Tag: #vlm-evaluation3 results
CRYSTAL is a new benchmark with 6,372 visual questions featuring verified step-by-step reasoning to evaluate multimodal models beyond final answers. Tests on 20 models show high accuracy but poor reasoning recovery, like GPT-5 at 58% accuracy vs 48% steps. CPR Curriculum training boosts reasoning by up to 93% on select VLMs.
CUAAudit introduces Vision-Language Models (VLMs) as autonomous auditors for evaluating Computer-Use Agents (CUAs) via observable interactions. A meta-evaluation of five VLMs across three CUA benchmarks on macOS, Windows, and Linux reveals strong accuracy and calibration, but degradation in complex environments and significant inter-model disagreements.
Researcher reviews video benchmarks like VideoMME, MLVU, MVBench, LVBench for VLMs. Seeks ideas on gaps, especially for physical, open-world evaluation. Posted on r/MachineLearning.
Astro 7 reportedly rewrites its compiler and Markdown processing pipeline in Rust. The change targets the framework’s core build and content-processing infrastructure.
FAW Hongqi has launched the refreshed HS5 SUV at a promotional price starting at 134,800 yuan. The vehicle adds Lingxi Smart Cabin 5.0, Hongqi Sinuo intelligent driving, upgraded displays, and a 2.0T powertrain.
Jingxin Microelectronics has successfully powered on its domestically developed RapidIO 3.2 switch chip SR1820 and RapidIO 3.2-to-PCIe 4.0 bridge chip ST0420. The chips completed all primary functional use cases in under one hour after returning from fabrication.
Shanghai is advancing a fourth-generation semiconductor ecosystem in Lingang, building on early research efforts launched in March 2024. The initiative focuses on gallium oxide and diamond materials, with incubators such as Sinan helping move laboratory results toward commercialization.
As industrial systems, infrastructure, robots, and AI agents gain the ability to change physical conditions, cybersecurity must protect actions and outcomes—not only data and access. The article argues that future defenses need to evaluate whether an authorized action is appropriate for the current environment, state, and safety boundaries.
Counterpoint Research predicts that Samsung Electronics will regain the top position in the global smartphone market this year. The forecast highlights Samsung’s relative resilience amid rising memory-chip prices.
CXMT Corp.’s breakout listing has boosted optimism across China’s domestic AI hardware sector. The momentum supports China’s broader ambition to compete with the US in AI development.