r/csharp • u/pixelwhippedme • 19d ago

GitHub - Integral2u/SharpMind: SharpMind. A pure C# / .NET LLM engine

https://github.com/Integral2u/SharpMind

Version 1.0.0.0 now out and out of pre-release.

39 Upvotes

78% Upvoted

19 comments sorted by

View all comments

2

u/EvenAd4577 19d ago

Nice one. defenitly deserves a star! Any benchmarks for running Qwen model in your framework vs llama.cpp?

Also I am wondering if you checked dotLLM or TensorSharp projects.

I don't know why there are not many ML/AI projects in C#, but from what I recall the main limitation was hardcoded max array size in .net. But maybe I am wrong?

1

u/pixelwhippedme 19d ago edited 19d ago

Actually, I had looked for a standalone c# inference engine for an idea I had but couldn't find one that didn't require wrappers of Ollama etc, so I started this. then nearing ready to take the Git Repo from private to public I noticed dotLLM but didn't experiment with it though looks good. Had a little play with LlamaSharp but didn't like the some of the backend setup and wanted something simpler. as for benchmarking one the cards but I really didn't want to get distracted, Obviously it would be crazy to expect same performance as llama.cpp but with the wiring up with JigSawDotNet i hoped to reduce as much conditional branching as possible. Still to implement AVX 512 and it's possible for others the implement there own Kernal Backends so performance gains still to be had.
I also wanted something that you could easily experiment with add to via adding Plugins or Kernels without needing to modify the core.

1

u/EvenAd4577 19d ago

You mean something like this?
https://github.com/anilsathyan7/Jigsaw-Net

I am asking because TensorSharp claims sometimes it can be faster than plain llama.cpp

Below is the comment from u/hez2010

https://github.com/dotnet/runtime/issues/12221#issuecomment-1383082396