🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!

Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.

🔗 github.com/deepseek-ai/DeepInfer

1/4

⚡ Inside DeepInfer

🔹 High-performance kernels & optimized inference engine
🔹 Distributed inference & speculative decoding
🔹 SWA Bounded Replay & structured outputs

Inference code for future DeepSeek models and ongoing optimizations will land here too.

2/4

🛠️ Also open-sourcing deepseek-recipe

Our production token encoding & decoding component, aligned with the processing logic behind the official DeepSeek API to bring the same API behavior to community deployments.

🔗 github.com/deepseek-ai/deepseek-recipe

3/4

🌍 Open models. Open kernels. Open inference.

From private deployments to public clouds, we’ll keep working with the community to make large-scale inference more efficient.

Explore the code, contribute, and build with us!

4/4