Models that see, hear and speak are replacing text-only systems. Coverage of vision-language models, native audio and video understanding.
354 articles
PRISM introduces a framework that couples VLM perception and LLM decision-making via a dynamic question-answer pipeline. The LLM critiques VLM descriptions, probes with goal-oriented questions, and synthesizes task-driven scene understanding.
ArXiv AI · 134d ago
Pull request #22493 adds support for Xiaomi's Mimo v2.5 Sparse MoE model in llama.cpp. The 310B total parameter model activates 15B params, supports 1M token context, and handles text, image, video, audio modalities.
Reddit r/LocalLLaMA · 135d ago
ByteDance upgraded Doubao-Seed-2.0-lite, the first full-modality model in Doubao family, enabling native unified understanding of video, images, audio, and text. It achieves SOTA in visual/audio benchmarks, surpassing Gemini-3.1-Pro, with boosts in Agent, Coding, and GUI capabilities.
IT之家 · 136d ago
Apple introduces SFI-Bench, a video-based benchmark with over 1700 questions from diverse egocentric indoor video scans. It evaluates higher-order spatial-functional intelligence in multimodal LLMs, advancing beyond basic geometric perception like VSI-Bench.
Apple Machine Learning · 136d ago
CVPR 2026 multimodal papers redefine vision intelligence: VideoAuto-R1 enables on-demand reasoning, LIVR performs latent visual reasoning sans language, and ARC reframed as vision task. Trends emphasize efficient, implicit inference over always-on chains.
雷峰网 · 142d ago
Chinese AI startup DeepSeek has added multimodal capabilities to its flagship chatbot, allowing it to process images and videos alongside text. This first-time feature is in limited release to select users, shortly after launching V4 model with price cuts.
SCMP Technology · 143d ago
DeepSeek V4, released just 5 days ago, now features image recognition in gray-degree testing on its web platform. Users can upload screenshots for direct analysis, simplifying problem-solving over manual descriptions.
cnBeta (Full RSS) · 143d ago
SenseTime has launched and open-sourced SenseNova U1, a unified multimodal model. Built on the NEO-unify architecture, it merges understanding and generation into a single framework.
Pandaily · 143d ago
A novel open-source model employs a low-parameter convolutional architecture to unify image understanding and generation tasks. It is fully available across the web for immediate use.
量子位 · 143d ago
Facewall Intelligence released a technical report for MiniCPM-o 4.5, emphasizing its quick setup and run on consumer-grade GPUs. The model has surpassed 250,000 downloads, showing high popularity.
量子位 · 144d ago
Apple presents StereoFoley, a video-to-audio framework producing semantically aligned, temporally synchronized, and spatially accurate stereo sound at 48 kHz. It overcomes mono limitations and lack of stereo datasets in prior models.
Apple Machine Learning · 144d ago
Kimi K2.6 now tops benchmarks in 3D design tasks. Described as one of the best benchmarks for model evaluation.
Reddit r/LocalLLaMA · 149d ago
X Square Robot launched WALL-B, a next-gen embodied AI model for real-world home deployment. It features multimodal learning and physical reasoning for practical household interactions.
Pandaily · 149d ago
OpenAI has launched Images 2.0 across ChatGPT, Codex, and its API. It introduces two modes, advanced text support, and 4K output resolution.
TestingCatalog · 149d ago
Koharu, a Rust-based local manga/image translator, has been updated for reliability and ease of use. Integrates llama.cpp with Gemma/Qwen models, object detection, OCR, and inpainting.
Reddit r/LocalLLaMA · 150d ago
OpenAI announced ChatGPT Images 2.0, an advanced image generation model where AI reasons step-by-step before creating visuals. It significantly improves accuracy for Japanese text rendering compared to previous versions.
ITmedia AI+ (日本) · 150d ago
Zibian Robotics launched WALL-B, a next-gen embodied AI model for home deployment. It integrates multimodal learning and physical reasoning for real-world interactions.
Pandaily · 150d ago
OpenAI launches ChatGPT Images 2.0 free worldwide, advancing multimodal AI utility. Meta invests $1B in data centers; Nvidia releases DLSS 4.5 SDK and Jetson optimizations; Intel unveils agent PC concept.
钛媒体 · 150d ago
OpenAI rolls out ChatGPT Images 2.0, enabling web searches for creating sophisticated images from single prompts. Powered by GPT Image 2 with thinking capabilities, it improves instruction adherence, detail preservation, and text generation.
The Verge · 151d ago
OpenAI's ChatGPT Images 2.0 is the newest image-generation model. It surprisingly excels at generating text, highlighting rapid AI evolution in recent years.
TechCrunch AI · 151d ago
Daimon Infinit, the world's largest multimodal physical world dataset including tactile data, has been jointly released. It attracted participation from Google and numerous universities.
量子位 · 152d ago
GIST transforms consumer-grade mobile point clouds into semantically annotated navigation topologies via 2D occupancy maps, topological layouts, and lightweight semantic overlays. It powers tasks like intent-driven semantic search, one-shot localization (1.04m error), zone classification, and visually-grounded instruction generation.
ArXiv AI · 152d ago
A team's experience fine-tuning and deploying Gemma-4 reveals key bugs: PEFT incompatibility with custom layers, SFTTrainer silently failing due to KV-sharing, DeepSpeed saving empty adapters, and lack of runtime LoRA support in serving tools. Fixes include unwrapping layers, updating transformers, avoiding DeepSpeed, and manual weight merging.
Reddit r/MachineLearning · 153d ago
Canva and Anthropic launched Claude Design, an Anthropic Labs product powered by Claude Opus 4.7 and Canva’s Design Engine. It generates fully editable, on-brand visuals from text descriptions.
The Next Web (TNW) · 155d ago
A user tasked Qwen3.6-35B with building a tower defense game using screenshots from MCP, and it successfully implemented and tested features like upgrades. The model self-detected and fixed bugs in canvas rendering and wave completions.
Reddit r/LocalLLaMA · 155d ago
OpenAI updated its AI programming agent Codex for Mac, adding desktop app control, image generation, and memory features. Codex can now use its cursor to view screens, click, and input for task automation.
cnBeta (Full RSS) · 156d ago
Anthropic released Claude Opus 4.7, its most powerful generally available model. It excels in advanced software engineering, complex coding, image analysis, instruction following, and creative document generation.
The Verge · 156d ago
Cloudflare is evolving AI Gateway into a unified inference layer for AI agents. Developers can now call models from 14+ providers.
Cloudflare Blog · 156d ago
LLM-HYPER uses LLMs as hypernetworks to generate training-free CTR model parameters for cold-start ads via few-shot CoT prompting on multimodal content. It retrieves similar campaigns with CLIP embeddings for demonstrations.
ArXiv AI · 156d ago
Cloudflare's experimental voice pipeline for Agents SDK enables real-time voice over WebSockets. It supports continuous STT and TTS.
Cloudflare Blog · 157d ago