DeepSeek has fully open-sourced its internal inference framework, DeepInfer, which already powers the high-throughput, low-latency large-scale inference service behind DeepSeek V4.1 Flash, the Tencent Research Institute's AI brief reported on October 21.
DeepInfer supports distributed inference, speculative decoding, SWA Bounded Replay and structured output, and is compatible with both Nvidia and Huawei Ascend platforms — a notable step toward compute-independence for domestic model serving. DeepSeek also open-sourced the Token codec component deepseek-recipe, fully aligned with the official API processing logic, with new-model inference code to be continuously updated there.
The open-sourcing lowers the barrier for developers and enterprises to deploy high-performance inference on diverse hardware, and reinforces a broader Chinese open-source momentum that industry observers say is a catalyst for global progress.
Together with other recent domestic releases, the move strengthens the case that China's AI stack — from models to inference frameworks — is increasingly self-contained across chips, software and services.
Source: Tencent Research Institute AI Brief
