Tencent Releases Hy3: Open 295B MoE Model

Tencent's Hy team released Hy3, a 295 billion parameter Mixture-of-Experts (MoE) model, on an unspecified date. This model activates only 21 billion parameters per token and is released under the Apache License 2.0, making it available for broader use. Hy3 is designed for tasks involving reasoning, agentic workflows, and long-context understanding.

The architecture of Hy3 features a sparse MoE with 192 experts, utilizing top-8 routing, which means only 8 experts are engaged per token to maintain computational efficiency. It also incorporates a Multi-Token Prediction (MTP) layer, capable of predicting multiple tokens simultaneously to accelerate decoding. This MTP layer is supported by vLLM and SGLang through speculative decoding. The model's specifications include 80 layers (excluding the MTP layer), 64 attention heads, a hidden size of 4096, an intermediate size of 13312, a context length of 256K, and a vocabulary size of 120832. A Hy3-FP8 checkpoint is also available, which reduces the memory footprint for more cost-effective serving.

Performance benchmarks released by the research team show Hy3 achieving strong results across various domains. In coding tasks, it scored 78.0 on SWE-Bench Verified, 57.9 on SWE-Bench Pro, and 75.8 on SWE-Bench Multilingual. For Terminal-Bench 2.1, it achieved 71.7, and for DeepSWE, it reached 28.0. The model demonstrates even higher capabilities in STEM and reasoning, with scores of 90.4 on GPQA Diamond and 72.0 on USAMO 2026. It also achieved 90.0 on IMOAnswerBench and 53.2 on HLE (with tools).

In a blind test involving 270 experts, Hy3 was compared against GLM-5.1. The test collected 312 valid comparisons on real-world workflows, where Hy3 scored 2.67 out of 4, surpassing GLM-5.1's score of 2.51. This advantage was particularly evident in areas such as frontend development, CI/CD, and data and storage management. The research team also emphasized production reliability, addressing specific failure modes related to tool calling and output formatting.