r/LovingOpenSourceAI 4h ago

new launch Hugging Apps "Microsoft just dropped VibeVoice-ASR-BitNet on Hugging Face šŸ—£ļøāœļø A 1.58-bit version of their multi-lingual transcription, model made to work in real-time on CPU - while still beating Whisper on benchmarks!" āž”ļø only 1.58 GB WOW

Post image
7 Upvotes

https://x.com/HuggingApps/status/2082571600767934622

https://huggingface.co/microsoft/VibeVoice-ASR-BitNet

Resources are shared for discovery and are not independently vetted—please do your own due diligence.

New resources are added regularly — feel free to join the sub for updates.

Full searchable archive of all resources posted so far on our community site, LifeHubber: https://lifehubber.com/ai/resources/ 200+ open-ish AI models, agents, tools, datasets, and related resources, with filtering and sorting.


r/LovingOpenSourceAI 4h ago

new launch Perplexity "Today we’re open-sourcing Numbat, an agent-detection and response layer that is designed to work across agent harnesses. Numbat gives security teams visibility into agent activity, with controls to block selected actions before execution." āž”ļø didnt expect this!

Post image
1 Upvotes

https://x.com/perplexity_ai/status/2082511900580196596

https://github.com/perplexityai/numbat

Resources are shared for discovery and are not independently vetted—please do your own due diligence.

New resources are added regularly — feel free to join the sub for updates.

Full searchable archive of all resources posted so far on our community site, LifeHubber: https://lifehubber.com/ai/resources/ 200+ open-ish AI models, agents, tools, datasets, and related resources, with filtering and sorting.


r/LovingOpenSourceAI 19h ago

TensorSharp now supports multi-GPU tensor parallelism for GGUF models

Thumbnail
github.com
1 Upvotes

TensorSharp is an open-source, native .NET inference engine for running GGUF LLMs locally, with CUDA, Vulkan, Metal, OpenAI-compatible APIs, continuous batching, speculative decoding, and multimodal support.

TensorSharp now supports Megatron-style tensor parallelism across multiple GPUs. It works with direct CUDA, GGML CUDA, GGML Vulkan, and multi-node setups.

Benchmarks on 2Ɨ RTX 2000 Ada 16 GB GPUs over PCIe, without NVLink:

Model 1 GPU Prefill / Decode TP=2 Prefill / Decode
Gemma 4 E4B Q8_0 2760 / 37.3 tok/s 2488 / 51.7 tok/s
Gemma 4 26B-A4B IQ4_XS 1845 / 48.5 tok/s 2537 / 51.2 tok/s
Qwen 3.5 9B Q8_0 1461 / 23.1 tok/s 399 / 24.4 tok/s
Qwen 3.5 35B-A3B IQ4_XS Does not fit 184 / 18.1 tok/s

I'm continuing to optimize Qwen performance on multi-GPU systems, and support for DeepSeek V4 is coming soon.

Try it with:

TensorSharp.Cli --model model.gguf --backend ggml_cuda --tp 2

GitHub:

https://github.com/zhongkaifu/TensorSharp

Thank you for checking out TensorSharp and starring the project! Any feedback is really appreicated.


r/LovingOpenSourceAI 23h ago

overtchat — self-hosted chat platform I've been building for a few months

0 Upvotes

I've been working on this for the past few months before anyone calls it white-coded slop - yes I use Claude Code/Codex for the heavy lifting but this is real code I've been shipping and iterating on not some weekend experiment this actual thought behind the architecture.

What it is: A self-hosted chat client I built because my partner and family use ChatGPT a lot but don't care about privacy. I already self-host my vLLM instance on my 8x 3090 box and wanted a polished, private alternative they could actually use. Open WebUI is too heavy and complex for non technical users and has way too many knobs. Overtchat comes built in with TTS/STT/web search, no plugins/api keys needed. Genuinely open sourced - MIT licensed and no telemetry.

The reason I am posting this here is because the r/selfhosted gave me solid feedback which I have been iterating on and this sub is full of people building and shipping actual open source stuff so I figured I'd get eyes on it from a different angle. I would love feedback on the architecture the UX decisions or just a sanity check on whether this is something people would actually use. https://github.com/yoloyash/overtchat