DeepSeek has open-sourced DeepInfer, the inference framework that powers its production services and the DeepSeek-V4.1-Flash model. This release, available under the Apache 2.0 license, provides developers with a rare view into a frontier lab's real serving stack.
Key points
- DeepSeek open-sourced its DeepInfer inference framework.
- DeepInfer powers the DeepSeek-V4.1-Flash model.
- The framework supports a 1 million token context window.
- It runs on NVIDIA Blackwell GPUs and Huawei Ascend NPUs.
- A single cluster serves 155 trillion tokens daily.
The company also released deepseek-recipe, an MIT-licensed token encoding and decoding layer. DeepInfer is described as a high-performance inference engine built primarily on vLLM, incorporating ideas from SGLang and TensorRT-LLM.
DeepInfer supports one model, DeepSeek-V4.1-Flash, with a 1 million token context window. It runs on NVIDIA SM100 (Blackwell) GPUs and Huawei Ascend 950 NPUs. Benchmarks utilized 4 GPUs for prefill and 64 for decode.
By the numbers
- token context window
- 1M
- tokens served per day per cluster
- 155 trillion
- output tokens/s/GPU
- 25,000+
DeepSeek claims the framework achieves production-grade throughput, with a single cluster serving 155 trillion tokens per day. The system offers over 25,000 output tokens/s/GPU performance at 256 requests per rank with a 131,072 context.
The engine incorporates DeepSeek's kernels, including FlashMLA, DeepGEMM, DeepEP, TileKernels, DeepSelect, and custom TileLang kernels. System-level features include Attention DP plus Wide-EP Attention for parallelism, and expert parallelism with an expert-parallel load balancer for MoE layers.
Sources and references
Original report
DeepSeek Open-Sources DeepInfer, the Inference Framework Behind DeepSeek-V4.1-Flash
MarkTechPost
Read on MarkTechPost (opens in a new tab)Linked in the original report
- Codegithub.com/deepseek-ai/DeepInfer (opens in a new tab)
- Modelhuggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017 (opens in a new tab)
- Codegithub.com/deepseek-ai/DeepInfer/blob/main/LICENSE (opens in a new tab)
- Codegithub.com/deepseek-ai/deepseek-recipe (opens in a new tab)
- Codegithub.com/vllm-project/vllm (opens in a new tab)
- Codegithub.com/deepseek-ai/DeepInfer/blob/main/doc/inf… (opens in a new tab)
Ref 003DSMLV
Cite this
APA
Interestana AI Editorial. (2026, October 20). DeepSeek Open-Sources DeepInfer Inference Framework. Interestana. https://interestana.com/articles/deepseek-open-sources-deepinfer-the-inference-framework-behind-deepseek-v41-flash-003dsmlv
MLA
“DeepSeek Open-Sources DeepInfer Inference Framework.” Interestana, 20 Oct. 2026, interestana.com/articles/deepseek-open-sources-deepinfer-the-inference-framework-behind-deepseek-v41-flash-003dsmlv.
Chicago
Interestana AI Editorial. “DeepSeek Open-Sources DeepInfer Inference Framework.” Interestana, October 20, 2026. https://interestana.com/articles/deepseek-open-sources-deepinfer-the-inference-framework-behind-deepseek-v41-flash-003dsmlv.
BibTeX
@misc{interestana_003dsmlv,
author = {{Interestana AI Editorial}},
title = {{DeepSeek Open-Sources DeepInfer Inference Framework}},
howpublished = {Interestana},
year = {2026},
month = oct,
url = {https://interestana.com/articles/deepseek-open-sources-deepinfer-the-inference-framework-behind-deepseek-v41-flash-003dsmlv},
note = {Summary of reporting by MarkTechPost, https://www.marktechpost.com/2026/10/20/deepseek-deepinfer-open-source-inference-framework/}
}Credit the source
Original reporting: MarkTechPost, “DeepSeek Open-Sources DeepInfer, the Inference Framework Behind DeepSeek-V4.1-Flash”, https://www.marktechpost.com/2026/10/20/deepseek-deepinfer-open-source-inference-framework/(opens in a new tab)
Follow this story
Get Artificial Intelligence in your inbox — new stories in 40 words or fewer, once a week.
Subscribe