Tencent Hy4 Preview: The 770B Open-Source MoE Model Reshaping the Productivity AI Race
Posted on 31st Aug 2026 06:04:41 in Artificial Intelligence, Machine Learning
Tagged as: Tencent, Hunyuan, Hy4, open source AI, LLM, MoE
On August 27, 2026, Tencent quietly shipped one of the most consequential open-source releases of the year. Tencent Hy4 preview — the fourth generation of the Hunyuan flagship family — is a 770-billion-parameter mixture-of-experts (MoE) large language model with 49 billion active parameters per token and a context window exceeding one million tokens. It is not just the scale that matters. What makes this release notable is that Tencent has positioned Hy4 preview squarely at real-world productivity: coding, office work, game development, and scientific research — and it has released the weights under an Apache 2.0 license on Hugging Face, ModelScope, GitCode, and CNB on day one.
The launch follows the pattern Tencent established with Hy3 earlier this year: release a preview first, gather real-world feedback, then push the official release. That preview-first strategy has now produced what Tencent calls its largest generation-over-generation capability gain to date, with the company claiming a spot at the open-source frontier.
Inside the Architecture: 770B Parameters, 256 Experts, Sparse Attention
Hy4 preview is an engineering statement as much as a model release. The backbone spans 78 layers: the first layer uses a standard dense feed-forward network, while the remaining 77 layers replace it with mixture-of-experts blocks, each containing 256 routed experts plus one shared expert. Every token activates only the top-8 routed experts alongside the shared expert, which is how a 770B-parameter model keeps inference cost to roughly 49B active parameters per token.
The architecture borrows ideas from the leading edge of open-source research. The attention module uses Gated DeepSeek Sparse Attention (Gated DSA) combined with IndexCache for cross-layer sparse index reuse, an approach that makes a one-million-token context window computationally tractable. The residual pathway employs iHC (identity Hyper-Connections) to widen inter-layer information flow across four residual streams. A native multi-token prediction layer — 10B total parameters with 0.7B active — is built in for speculative decoding, accelerating inference without changing output quality.
Other headline specifications: a hidden size of 6,144, 64 attention heads, a query compression dimension of 2,048, and a vocabulary of 120,832 tokens. Tencent also released Hy4 preview-FP8, a quantized variant aimed at deployments where memory is tighter.
Built for Productivity — and Benchmarked Against the Frontier
Most frontier releases chase benchmark leaderboards. Hy4 preview chases a different metric: whether domain experts prefer its output on the work they actually ship. Tencent built the training data in partnership with its own in-house specialists — software engineers, game developers, finance analysts, and security researchers — and co-designed the model with products like CodeBuddy and WorkBuddy.
The results are directionally strong. In a blind side-by-side evaluation, 163 internal experts rated model outputs on 203 engineering tasks. Hy4 preview scored an average of 2.99 out of 4.00, slightly ahead of GLM-5.3 at 2.92 and Kimi K3 at 2.94, with a win rate of 46.8% against the former and 51.2% against the latter.
On public benchmarks, early community evaluations tell a similar story: 92.3 on GPQA Diamond (graduate-level science reasoning), 64.3 on DeepSWE, 65.7 on SWE-bench Pro, and 82.9 on SWE-bench Multilingual. On software engineering specifically, Tencent reports stronger understanding, planning, debugging, and verification on long-horizon development tasks, plus visibly better front-end work.
Across other domains the pitch is equally practical:
- Office and analysis: converts messy context spread across many files into documents, spreadsheets, and presentations, with sharper handling of financial models and equations.
- Game development: generates a playable prototype from a single natural-language prompt and iterates fluently with game engines like Unity over multiple turns.
- Scientific research: improved reasoning across AI research and development, molecular dynamics simulation, condensed-matter physics, and pure mathematics.
The Model That Helped Build Itself
The most unusual claim in Tencent's announcement is recursive self-improvement. For the first time, Hy4 preview participated in its own development pipeline: proposing approaches, running experiments, and iterating on training methods, data strategies, evaluation frameworks, and low-level operators. The code, logs, and feedback it generated fed into subsequent rounds of exploration, forming what Tencent describes as an early-stage self-improvement loop.
That capability extended to infrastructure. Hy4 preview autonomously analyzed bottlenecks in its own inference system and ran multiple rounds of optimization on operator fusion and communication. The result was a 31.8% improvement in end-to-end throughput against baseline, with consistent gains across different context lengths and concurrency levels. In one research exercise, the model coordinated several Codex sessions in parallel, judged research direction as results came in, and beat Codex working alone across all eight benchmarks in a small-model post-training task.
Tencent is equally candid about the limitations. The preview ships with known issues — reasoning longer than necessary on complex tasks and a tendency to over-verify its own work — and the company explicitly says there is headroom left in both pre-training and post-training.
Access, Pricing, and What Comes Next
Hy4 preview is available today through multiple channels. The weights are on Hugging Face, ModelScope, GitCode, and CNB under Apache 2.0, with vLLM and SGLang deployment recipes and an OpenAI-compatible API. Developers can set reasoning effort to "high" for deep chain-of-thought or "no_think" for direct responses. Tencent recommends a temperature of 0.9 for inference.
For hosted access, the model is live on Tencent Cloud TokenHub and OpenRouter. API pricing is US$0.834 per million input tokens, US$2.501 per million output tokens, and US$0.042 per million cached tokens. Tencent has also made the model free for two weeks on WorkBuddy and CodeBuddy, and extended free access to Hy3 on both platforms until September 30, 2026.
The preview-first cadence means this is not the final word on Hy4. Tencent has confirmed the next batch of Hy4-series models will roll out soon, incorporating feedback from this release. For teams evaluating open-source models for production workloads, that cadence matters: Hy4 preview is a snapshot of a model that will keep improving. The question for the broader industry is whether Tencent's productivity-first positioning — training on the tasks real engineers, analysts, and researchers do every day — becomes the new playbook for frontier open-source releases.