๐ผPandailyโขStalecollected in 28m
Meituan Open-Sources LongCat-Video-Avatar 1.5 for Digital Humans
๐กHigh-efficiency, open-source digital human framework with state-of-the-art lip-sync performance.
โก 30-Second TL;DR
What Changed
Achieves state-of-the-art lip-sync accuracy in digital human generation.
Why It Matters
This release lowers the barrier for developers to create high-quality, real-time digital avatars. It significantly improves the efficiency of lip-syncing in generative video pipelines.
What To Do Next
Clone the LongCat-Video-Avatar repository and benchmark the 8-step inference latency against your current lip-sync solution.
Who should care:Developers & AI Engineers
Key Points
- โขAchieves state-of-the-art lip-sync accuracy in digital human generation.
- โขOptimized for efficiency with only 8 inference steps required.
- โขOpen-source release allows developers to integrate photorealistic avatars.
๐ง Deep Insight
Web-grounded analysis with 13 cited sources.
๐ Enhanced Key Takeaways
- โขLongCat-Video-Avatar 1.5 integrates the Whisper-Large audio encoder, enabling enhanced lip-sync accuracy and style generalization across 99 languages, a significant upgrade from its predecessor's Wav2Vec2.
- โขThe framework achieves a substantial efficiency gain, boasting approximately 15 times faster inference speed by reducing generation steps from 50 to 8 through Distribution Matching Distillation (DMD).
- โขThe model demonstrates robust generalization capabilities, supporting the generation of digital humans in diverse styles, including realistic human portraits, anime characters, virtual idols, and even animals, while also handling multi-person interactions.
- โขBuilt upon the LongCat-Video foundation, version 1.5 is designed for long-duration video generation, capable of producing coherent content up to 15 minutes, addressing a key limitation in temporal consistency for AI video.
- โขIt operates with a 'one model for multiple tasks' design, natively supporting Audio-Text-to-Video (AT2V), Audio-Text-Image-to-Video (ATI2V), and video continuation, offering versatile application scenarios.
๐ Competitor Analysisโธ Show
While specific pricing details are not consistently available for all platforms, especially open-source ones, a comparison of features and benchmarks can be made:
| Feature / Platform | Meituan LongCat-Video-Avatar 1.5 | HeyGen | Kling AI / Avatar 2.0 | ByteDance OmniHuman-1 | Dubly.AI |
|---|---|---|---|---|---|
| Lip-Sync Accuracy | State-of-the-art, Whisper-Large encoder, 99 languages | Strong, for personalized/translated videos | Impressive, strong for dialogue | Accurate | Highest benchmark (96.4), handles complex footage |
| Efficiency | 8 inference steps, ~15x speedup via DMD | Fast generation times | Fast generation times | Not specified | Not specified |
| Long-Video Stability | Up to 15 minutes, temporal consistency, identity preservation | Up to 3 minutes | Improves across long-form videos | Not specified | Not specified |
| Subject Generalization | Real humans, anime, virtual idols, animals, multi-person | Talking digital twins | Photorealistic humans | Portraits, full body, cartoons, stylized avatars | Real video footage |
| Licensing | Open-source (MIT License for academic use, commercial licensing available) | Commercial platform | Commercial platform | Research model | Commercial platform |
| Benchmarks | Leads EvalTalker evaluations for anthropomorphism | 76.8 (independent benchmark) | Best-in-class for realistic human generation | Not specified | 96.4 (independent benchmark) |
| Unique Features | Multi-segment rolling inference, GRPO alignment, Cross-Chunk Latent Stitching | Personalized & translated videos, overview outline before generation | Best-in-class for realistic human faces/movements | Single image + motion signals | GDPR-compliant servers, data not used for AI training |
๐ ๏ธ Technical Deep Dive
- Base Model: Built on the LongCat-Video Diffusion Transformer (DiT) architecture, which serves as a foundational video generation model.
- Audio Encoder: Upgraded from Wav2Vec2 to Whisper-Large (specifically Whisper-large-v3) for superior audio feature extraction, leading to more precise lip synchronization and multilingual support.
- Inference Efficiency: Utilizes Distribution Matching Distillation (DMD) to compress the generation process from 50 steps to just 8, resulting in an approximate 15x speedup.
- Memory Optimization: Employs a shared base model combined with multiple LoRA (Low-Rank Adaptation) adapters to significantly reduce VRAM usage compared to traditional three-model parallel deployment schemes. INT8 quantization is also supported for further VRAM reduction.
- Long Video Stability: Incorporates a hierarchical attention mechanism (fine-grained, medium-scale, coarse attention), Cross-Chunk Latent Stitching, and Reference Skip Attention to maintain visual consistency, narrative coherence, and identity over extended video durations, capable of generating up to 15 minutes of coherent content.
- Quality and Alignment: Leverages Group Relative Policy Optimization (GRPO) with frame-level human preference rewards to ensure accurate coordination of speech, lip motion, facial expressions, head pose, and body movements.
- Silent Segment Naturalness: Features Disentangled Unconditional Guidance to generate natural micro-expressions and body dynamics during silent periods, preventing unnatural stillness.
- Supported Tasks: Natively supports Audio-Text-to-Video (AT2V), Audio-Text-Image-to-Video (ATI2V), and video continuation, accommodating both single-stream and multi-stream audio inputs.
- Data Pipeline: Utilizes a high-quality, multi-stage data processing workflow involving offline annotation (extracting face keypoints, person count, body composition, audio-visual sync) and online validation (filtering low-quality segments).
๐ฎ Future ImplicationsAI analysis grounded in cited sources
Increased adoption of AI-generated content in commercial production.
The model's 'commercial-grade' quality, enhanced efficiency, and open-source nature with commercial licensing significantly lower the barriers for businesses to integrate advanced digital human technology into their production workflows.
Acceleration of multimodal AI agent development.
Meituan's broader LongCat family strategy aims to build intelligent agents by combining vision (LongCat-Video) and reasoning (LongCat-Flash), suggesting future integration for more sophisticated AI applications that can both understand and generate complex multimodal content.
Democratization of advanced digital human creation.
The open-source release of a state-of-the-art framework allows a wider range of developers, researchers, and creators to access, modify, and build upon cutting-edge technology, fostering innovation beyond large corporate entities.
โณ Timeline
2010-03
Meituan.com founded by Wang Xing, focusing on group-buying services.
2015-10
Meituan merges with Dianping, a major competitor in local services.
2020-09
Company officially rebrands from 'Meituan Dianping' to 'Meituan'.
2023
Meituan acquires AI startup Light Year for US$281 million, marking a significant entry into the AI sector.
2025-09
Meituan releases its first open-source Large Language Model (LLM), LongCat-Flash-Chat.
2025-10
Meituan releases LongCat-Video, a foundational video generation model.
2025-12
Meituan releases LongCat-Video-Avatar (v1.0), an audio-driven character animation model.
2026-05
Meituan open-sources LongCat-Video-Avatar 1.5, an upgraded framework for audio-driven human video generation.
๐ Sources (13)
Factual claims are grounded in the sources below. Forward-looking analysis is AI-generated interpretation.
๐ฐ
Weekly AI Recap
Read this week's curated digest of top AI events โ
๐Related Updates
AI-curated news aggregator. All content rights belong to original publishers.
Original source: Pandaily โ