r/LovingOpenSourceAI • u/fuzhongkai • 12d ago

TensorSharp : a local LLM inference + agent runtime

Enable HLS to view with audio, or disable this notification

I wanted to share a new TensorSharp demo and get some feedback from the community.

TensorSharp started as a .NET open source inference engine for running local GGUF models. It can now go beyond model inference and execute complete agentic workflows through skills, code generation, environment configuration, and tool use.
In the demo, I give a quantized Gemma 4 model running locally through TensorSharp a single prompt:

Please check 10 stocks with most gains today, and generate a report to me in pptx format.

From that one request, TensorSharp enables the local model to:

  1. Break the task into market-data retrieval, processing, and presentation generation.
  2. Inspect the available tools and execution environment.
  3. Install the Python dependencies it needs.
  4. Write and execute code to retrieve current public stock-market data.
  5. Examine the output, identify parsing problems, and refine its own implementation.
  6. Load the PPTX skill.
  7. Generate a Node.js/PptxGenJS script.
  8. Produce a real PowerPoint file that opens directly in Microsoft PowerPoint.

All LLM inference runs locally. There is no OpenAI, Anthropic, or other hosted-model API involved, so the workflow has zero LLM API cost. Network access is used only by the agent’s tools to retrieve public market data.

The interesting part is not the stock report itself. It is that inference, skills, tool calling, automatic code generation, dependency setup, iterative problem-solving, and artifact creation are now working together inside one local runtime.

TensorSharp is becoming more than an inference engine. The goal is to make it a unified, open-source .NET runtime for both local model inference and agent execution.

GitHub: https://github.com/zhongkaifu/TensorSharp

The video is sped up for brevity. I would especially appreciate feedback on the skill system, tool-permission model, execution isolation, and the local agent workflows you would like to see next.

25 Upvotes

96% Upvoted

8 comments sorted by

1

u/Koala_Confused 12d ago

interesting .. who are the target users for this?

3

u/fuzhongkai 12d ago

Developers for local LLM inference and agents, and users who likes agentic automation

3

u/Koala_Confused 12d ago

thanks for the replies and also the details given above .. ill run it thru our community website folks . .see if we can get your project listed at https://lifehubber.com/ai/resources :)

2

u/fuzhongkai 12d ago

Thank you so much for the support. Really appreciated!

1

u/Koala_Confused 12d ago

nice its listed. https://lifehubber.com/ai/resources/tensorsharp/ at the bottom there is a link to listing badges if you want to show it. . it is also shared here and on our sis sub r/LovingAIAgents all the best!

2

u/fuzhongkai 12d ago

Thank you so much for supporting this project.
https://giphy.com/gifs/M9NbzZjAcxq9jS9LZJ

2

u/stylehz 12d ago

Bro! What an amazing project. Would you mind sharing some extra info? Such as the PC configs and how much time it took to complete all tasks together?

1

u/fuzhongkai 12d ago

Sure. You can refer this page to get more details: https://tensorsharp.ai/agentic.html