Tencent Open-Sources 770B Hy4 Preview

Post LinkedIn

📋Read original on TestingCatalog

#770b-parameters#coding#long-contexthy4tencenthy4

💡Explore Tencent’s open-source 770B model and its unusually large 1M-token context window.

⚡ 30-Second TL;DR

What Changed

Hy4 is released as an open-source preview model by Tencent.

Why It Matters

The combination of open-source availability and a 1 million-token context window could enable experimentation with long-context workflows at substantial model scale. Developers and researchers can evaluate whether Hy4 fits coding, document-heavy, or research-oriented applications.

What To Do Next

Download the Hy4 preview from Tencent’s official release channel and test its long-context performance on a representative coding or document-retrieval workload.

Who should care:Researchers & Academics

Key Points

•Hy4 is released as an open-source preview model by Tencent.
•The model has 770 billion parameters.
•It supports a 1 million-token context window.
•Target capabilities include coding, office tasks, gaming, and research.

🧠 Deep Insight

Background and context from public sources — not the original article. 10 sources cited.

🔑 Enhanced Key Takeaways

•Hy4 utilizes a Mixture-of-Experts (MoE) architecture with 49 billion active parameters per token out of its 770 billion total.
•The model architecture features 78 layers, employing 256 routed experts and one shared expert, with a top-8 expert selection mechanism.
•It incorporates a native 10 billion parameter Multi-Token Prediction (MTP) layer to facilitate speculative decoding for enhanced inference speed.
•Tencent released the model under the Apache 2.0 license, making it available on major repositories including Hugging Face, ModelScope, and GitCode.
•The model was developed using a co-design strategy, integrating domain-specific training data from internal Tencent teams in finance, gaming, and security.

📊 Competitor Analysis▸ Show

Feature	Hy4 Preview	GLM-5.3	Kimi K3
Architecture	770B MoE	Proprietary	Proprietary
Internal Benchmark Score	2.99/4.00	2.92/4.00	2.94/4.00
License	Apache 2.0	Proprietary	Proprietary

🛠️ Technical Deep Dive

Architecture: Mixture-of-Experts (MoE) with 78 layers.
Expert Configuration: 256 routed experts plus one shared expert; top-8 experts activated per token.
Speculative Decoding: Native 10B parameter Multi-Token Prediction (MTP) layer with 0.7B active parameters.
Inference Optimization: Official support for vLLM and SGLang frameworks.
Parameter Density: 49B active parameters per token.

🔮 Future ImplicationsAI analysis grounded in cited sources

Hy4 will see rapid adoption in enterprise coding environments.

The inclusion of native speculative decoding and official vLLM/SGLang support significantly lowers the barrier for high-throughput deployment in production.

Tencent's open-source strategy will pressure proprietary model providers in the Chinese market.

By outperforming established models like GLM-5.3 and Kimi K3 on internal benchmarks while offering an Apache 2.0 license, Tencent is aggressively commoditizing high-end model capabilities.