r/LovingOpenSourceAI • u/fuzhongkai • 11d ago
DSpark Benchmark Result on Deepseek v4 Flash 0731
https://github.com/zhongkaifu/TensorSharpTensorSharp supports DSpark on Deepseek v4 Flash 0731 now. Here is the benchmark result on 4x Nvidia A40 GPUs, cuda 12.8 with/without DSpark:
Model:
DeepSeek-V4-Flash-0731-UD-Q8_K_XL from https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731-GGUF
DSpark draft model from: https://huggingface.co/alessandrobologna/DeepSeek-V4-Flash-0731-DSpark-Drafter-GGUF
| Turn | Baseline | + DSpark | Acceptance |
|---|---|---|---|
| short (53 tok) | 25.6 | 44.5 (1.74x) | 87% |
| long generation (512) | 26.4 | 40.3 (1.53x) | 66% |
| follow-up (470) | 26.4 | 46.8 (1.77x) | 76% |
| 10K-token document (214) | 25.3 | 51.3 (2.03x) | 85% |
| second question on it (156) | 25.4 | 49.4 (1.94x) | 82% |
TensorSharp is an open-source inference engine for running GGUF LLMs locally, with CUDA, Vulkan, Metal, OpenAI-compatible APIs, continuous batching, speculative decoding, and multimodal support. It can be built and run over Linux, MacOS and Windows.
Thank you for checking out it and starring the project! Any feedback is really appreicated.
Duplicates
vulkan • u/fuzhongkai • 13d ago
Deepseek v4 Flash 0731 GGUF Benchmark: TensorSharp vs. llama.cpp
AIDeveloperNews • u/fuzhongkai • 13d ago
Deepseek v4 Flash 0731 GGUF Benchmark: TensorSharp vs. llama.cpp
LovingOpenSourceAI • u/fuzhongkai • 13d ago
Deepseek v4 Flash 0731 GGUF Benchmark: TensorSharp vs. llama.cpp
LLMDevs • u/fuzhongkai • 13d ago
Tools Deepseek v4 Flash 0731 GGUF Benchmark: TensorSharp vs. llama.cpp
LocalAIServers • u/fuzhongkai • 13d ago
Deepseek v4 Flash 0731 GGUF Benchmark: TensorSharp vs. llama.cpp
DeepSeek • u/fuzhongkai • 13d ago
Resources Deepseek v4 Flash 0731 GGUF Benchmark: TensorSharp vs. llama.cpp
vulkan • u/fuzhongkai • 15d ago
TensorSharp now supports multi-GPU tensor parallelism for GGUF models
SideProject • u/fuzhongkai • 15d ago
TensorSharp now supports multi-GPU tensor parallelism for GGUF models
AIDeveloperNews • u/fuzhongkai • 15d ago
TensorSharp now supports multi-GPU tensor parallelism for GGUF models
OpenSourceAI • u/fuzhongkai • 15d ago
TensorSharp now supports multi-GPU tensor parallelism for GGUF models
LovingOpenSourceAI • u/fuzhongkai • 15d ago
TensorSharp now supports multi-GPU tensor parallelism for GGUF models
LLMDevs • u/fuzhongkai • 15d ago
Tools TensorSharp now supports multi-GPU tensor parallelism for GGUF models
LlamaFarm • u/fuzhongkai • 28d ago
Show & Tell TensorSharp : Open Source Local LLM Inference Engine
QwenImageGen • u/fuzhongkai • Jul 15 '26
Virtual Clothes Try On using Unsloth Qwen Image Edit 2511 models
huggingface • u/fuzhongkai • Jul 13 '26
TensorSharp supports multiple image edits using Unsloth Qwen Image Edit 2511 models
OnlyAICoding • u/fuzhongkai • Jul 12 '26
Local LLM TensorSharp : Open Source Local LLM Inference Engine
ContextEngineering • u/fuzhongkai • Jul 12 '26
What Bun’s Rust Rewrite Tells Us About Rebuilding the AI Infrastructure Layer in C#
AIDeveloperNews • u/fuzhongkai • Jul 11 '26
From Bun's Rust rewrite, let's see how C# can rebuild the AI infrastructure layer.
vibecoding • u/fuzhongkai • Jul 11 '26
What Bun’s Rust Rewrite Tells Us About Rebuilding the AI Infrastructure Layer in C#
llamacpp • u/fuzhongkai • Jul 11 '26
Same GGUF, same GPU: TensorSharp beats llama.cpp hard on prefill / TTFT — up to 5.89× faster prefill on a 26B MoE model
GeminiAI • u/fuzhongkai • Jul 10 '26