SGLang now supports DSpark, enabling confidence-driven, variable-length verification for speculative decoding 🎉
DSpark addresses a key bottleneck under load: instead of verifying every draft token, it verifies only where the draft model is confident, so the gains hold even as batch size scales.
We heavily optimized variable-length verification in SGLang. Across batch sizes 1 to 256, DSpark gives the best throughput/latency tradeoff on DeepSeek-V4-Flash, ahead of both MTP and non-spec.
At high concurrency, dynamic scheduling provides up to ~20% higher throughput compared to a fixed budget, while maintaining high verification quality across workloads.
With fused kernels and zero-overhead scheduling, DeepSeek-V4-Pro reaches 383.7 tok/s at B=1 on B300.
DSpark is now available in SGLang with support for Qwen3 and DeepSeek-V4. Thanks @deepseek_ai for open-sourcing!
Blog with full technical details and commands to run below 👇
00:00
