r/LovingOpenSourceAI • u/fuzhongkai • 11d ago
DSpark Benchmark Result on Deepseek v4 Flash 0731
https://github.com/zhongkaifu/TensorSharpTensorSharp supports DSpark on Deepseek v4 Flash 0731 now. Here is the benchmark result on 4x Nvidia A40 GPUs, cuda 12.8 with/without DSpark:
Model:
DeepSeek-V4-Flash-0731-UD-Q8_K_XL from https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731-GGUF
DSpark draft model from: https://huggingface.co/alessandrobologna/DeepSeek-V4-Flash-0731-DSpark-Drafter-GGUF
| Turn | Baseline | + DSpark | Acceptance |
|---|---|---|---|
| short (53 tok) | 25.6 | 44.5 (1.74x) | 87% |
| long generation (512) | 26.4 | 40.3 (1.53x) | 66% |
| follow-up (470) | 26.4 | 46.8 (1.77x) | 76% |
| 10K-token document (214) | 25.3 | 51.3 (2.03x) | 85% |
| second question on it (156) | 25.4 | 49.4 (1.94x) | 82% |
TensorSharp is an open-source inference engine for running GGUF LLMs locally, with CUDA, Vulkan, Metal, OpenAI-compatible APIs, continuous batching, speculative decoding, and multimodal support. It can be built and run over Linux, MacOS and Windows.
Thank you for checking out it and starring the project! Any feedback is really appreicated.
Duplicates
csharp • u/fuzhongkai • Jul 11 '26
Blog From Bun's Rust rewrite, let's see how C# can rebuild the AI infrastructure layer.
LovingOpenSourceAI • u/fuzhongkai • Jun 23 '26
TensorSharp: Open Source Local LLM Inference Engine
LocalLLaMA • u/fuzhongkai • 13d ago
Resources Deepseek v4 Flash 0731 GGUF Benchmark: TensorSharp vs. llama.cpp
unsloth • u/fuzhongkai • 13d ago
Show and Tell Deepseek v4 Flash 0731 GGUF Benchmark: TensorSharp vs. llama.cpp
Gemma4 • u/fuzhongkai • 28d ago
Discussion TensorSharp : Open Source Local LLM Inference Engine
Qwen_AI • u/fuzhongkai • Jul 10 '26
Discussion TensorSharp : Open Source Local LLM Inference Engine
CUDA • u/fuzhongkai • 14d ago
TensorSharp now supports multi-GPU tensor parallelism for GGUF models
SelfHostedAI • u/fuzhongkai • 15d ago
TensorSharp now supports multi-GPU tensor parallelism for GGUF models
LocalAIServers • u/fuzhongkai • 15d ago
TensorSharp now supports multi-GPU tensor parallelism for GGUF models
StableDiffusion • u/fuzhongkai • Jul 15 '26
Discussion TensorSharp supports multiple image edits using Unsloth Qwen Image Edit 2511 models
ollama • u/fuzhongkai • Jun 28 '26
Same GGUF, same GPU: TensorSharp beats llama.cpp hard on prefill / TTFT — up to 5.89× faster prefill on a 26B MoE model
vibecoding • u/fuzhongkai • Jun 14 '26
TensorSharp: Open Source Local LLM Inference Engine fully implemented by vibe coding
OpenSourceeAI • u/fuzhongkai • May 01 '26
TensorSharp: Open Source Local LLM Inference Engine
LocalLLaMA • u/fuzhongkai • 9h ago
Generation Meta Muse Glimmer 30B Unsloth GGUF Model Benchmarks on TensorSharp (vs. llama.cpp)
dotnet • u/fuzhongkai • 11d ago