🏕️极客公园•Freshcollected in 9m
Xiaohongshu Open-Sources Its AI Foundation
#mixture-of-experts#multimodal-model#512k-context#model-infrastructuredots3-note-preview小红书dots3-note previewhugging facehuawei ascendgithub
💡A major content platform is open-sourcing a 280B MoE model and benchmarks tailored to real-life search.
⚡ 30-Second TL;DR
What Changed
dots3-note preview has 280B total parameters, 16B activated parameters, and a 512K context window.
Why It Matters
This release suggests that content platforms increasingly view model ownership as infrastructure rather than a standalone AI feature. Platform-specific models could improve cost control, data feedback loops, and understanding of contextual lifestyle queries, while expanding the open-source model ecosystem.
What To Do Next
Download dots3-note preview from Hugging Face, run its 512K-context multimodal inference on your target hardware, and benchmark it against your current search or agent model.
Who should care:Developers & AI Engineers
Key Points
- •dots3-note preview has 280B total parameters, 16B activated parameters, and a 512K context window.
- •The model supports text, visual, and speech understanding and is released under the Apache 2.0 license on Hugging Face and GitHub.
- •Xiaohongshu also released VibeSearchBench and VibeLifeBench for multi-turn search and long-horizon life-task evaluation.
- •Huawei Ascend completed same-day adaptation, while early Hugging Face downloads exceeded 100.
- •The model remains weaker in terminal operation, complex programming, hallucination control, and stability than leading systems.
🧠 Deep Insight
Background and context from public sources — not the original article. 19 sources cited.
🔑 Enhanced Key Takeaways
- •dots3-note preview integrates a novel TEMPO reinforcement learning method specifically designed for training agents on long-horizon tasks that can span tens of hours.
- •The model's architecture includes a separate perception stack, with a Mixture-of-Experts vision transformer (7B total, 1.2B active parameters) for images and video frames, and a dense 800M parameter encoder for audio.
- •The dots3-note series has demonstrated advanced capabilities, with a related model achieving a perfect score of 42 points in the International Mathematical Olympiad (IMO), showcasing strong mathematical reasoning.
- •Huawei Ascend's same-day adaptation involved deep optimization for multimodal processing and incremental decoding, including native adaptation of MTP speculative decoding to reduce latency.
- •Xiaohongshu elevated AI to a tier-1 strategic department named "Dots" in April 2026, reporting directly to the CEO, signaling a significant organizational commitment to AI development.
📊 Competitor Analysis▸ Show
| Model | Features | Licensing | Context Window | Benchmarks (Selected) |
|---|---|---|---|---|
| Xiaohongshu dots3-note preview | Multimodal (text, vision, audio, video), MoE (280B total, 16B active) | Apache 2.0 | 512K tokens | Terminal-Bench 2.1: 75.1, IMO: 42 (series) |
| Alibaba Qwen 3.6 | Multimodal (text, image, video), MoE (35B-A3B, 3B active), agentic coding | Apache 2.0 | Not specified (Qwen 3.5 had 1M) | SWE-bench Verified: 73.4%, Outperforms previous 397B flagship on coding |
| Meta Llama 4 Maverick | Multimodal (text, images, video), MoE (400B total, 17B active) | Llama 4 Community License (free commercial use < 700M MAU) | 1M tokens | MMMLU: 84.6% |
| NVIDIA Nemotron 3 Ultra | MoE (550B), hybrid Mamba-Transformer, long-running agents | Not specified (open-weight) | 1M tokens | Not specified |
| Google Gemma 4 | Multimodal reasoning, alternates sliding window/global attention | Open-weight | Up to 256K tokens (cloud) | MMMU-Pro: 73.8%, GPQA Diamond: 82.3% |
🛠️ Technical Deep Dive
- Architecture: Mixture-of-Experts (MoE) with one dense layer and 45 MoE layers.
- Expert Routing: Routes to 8 of 256 routed experts plus one shared expert per token.
- Hidden Size: 5,120.
- Feed Forward Width: Dense feed forward width of 13,824 and 1,536 per expert.
- Attention Mechanism: Mixes 13 DSA blocks with 33 sliding window blocks (approx. 1:3 ratio), with DSA operating over the top 2,048 positions.
- Multi-token Prediction Layer: Adds an additional 1.13 billion parameters.
- Perception Stack: Separate from the language model.
- Vision: Handled by a Mixture-of-Experts vision transformer (7B total parameters, 1.2B active).
- Audio: Processed by a dense 800 million parameter encoder.
- Vocabulary Size: 152,000 tokens.
- Precision: Weights are available in both BF16 and FP8, with FP8 recommended for practical deployment on a single 8-GPU node.
- Training Method: Incorporates TEMPO reinforcement learning for long-horizon agent training.
- Huawei Ascend Adaptation: Includes deep optimization for multimodal processing and incremental decoding, natively adapting MTP speculative decoding capability.
- vLLM Ascend Support: Supports Dots3 Note MoE + MLA (Multi-head Latent Attention) architecture, multimodal encoders (audio + vision, enabled per form), MTP speculative decoding (for text-only/audio), FlashComm1 for TP communication optimization, FusedMC2 for MoE operators, and prefix caching.
🔮 Future ImplicationsAI analysis grounded in cited sources
Xiaohongshu will release larger and more capable models within the
dots3 family in the near future.dots3-note preview is explicitly described as the "most lightweight member of the family," implying that larger, more expensive-to-run dots3 variants are planned or already in development.Xiaohongshu's open-source AI strategy will significantly increase its influence within the global AI developer community, particularly in multimodal agentic AI.
By open-sourcing advanced multimodal MoE models designed for long-horizon agentic tasks and providing benchmarks, Xiaohongshu aims to demonstrate technological capabilities, build developer communities, and spread influence globally.
The integration of
dots3-note preview and its successors will lead to more sophisticated and personalized AI-powered content creation and discovery tools on the Xiaohongshu platform.The model's focus on multimodal understanding, long-horizon tasks, and agentic capabilities directly supports enhancing search, recommendations, and creation services, aligning with Xiaohongshu's strategy to boost production efficiency and user experience.
⏳ Timeline
2023
Xiaohongshu began investing in large language model development and formed an independent large model team.
2024-08
Xiaohongshu launched Diandian, an AI-powered search application, as an independent product.
2025-06
Xiaohongshu open-sourced its first large language model, `dots.llm1`, a 142 billion parameter MoE model.
2026-04
Xiaohongshu established "Dots" as a first-level AI department, elevating AI to a tier-1 strategic priority.
2026-05
Xiaohongshu's AI lab open-sourced OpenStoryline, a multimodal model for short-video and image-text narrative generation.
2026-08
Xiaohongshu AI Lab dots.studio open-sourced `dots3-note preview`, a multimodal MoE model.
📎 Sources (19)
Factual claims are grounded in the sources below. Forward-looking analysis is AI-generated interpretation.
📰
Weekly AI Recap
Read this week's curated digest of top AI events →
👉Related Updates
AI-curated news aggregator. All content rights belong to original publishers.
Original source: 极客公园 ↗
This is a summary, not the original. Read the source, or get the weekly briefing.
Weekly AI briefing
One email a week. Unsubscribe anytime.