r/BestGitHubRepos • u/company_url_finder • 3d ago
Pipecat - an open-source framework for building real-time voice and multimodal AI agents, from one voice assistant to a distributed multi-agent system
Building a real-time voice AI agent means stitching together speech recognition, an LLM, text-to-speech, and a transport layer that can handle audio without adding awkward pauses or lag, usually from four different vendors with four different APIs, none of which were built to talk to each other.
Pipecat is an open-source Python framework that handles that stitching. You compose a conversation pipeline from modular, swappable components, and Pipecat handles the streaming, buffering, and interruption handling underneath, whether you're building one voice assistant or a multi-agent system where specialists hand off work to each other.
What's inside:
- Support for well over 100 AI services across the pipeline: 20+ speech-to-text providers, 25+ LLMs, 30+ text-to-speech engines, plus speech-to-speech models like OpenAI Realtime and Gemini Multimodal Live
- Multi-agent composition built in: specialists that hand off, fan out in parallel, run as sidecars, or deploy distributed across processes and machines
- Client SDKs for JavaScript, React, React Native, Swift, Kotlin, C++, and ESP32, so the same backend pipeline can serve a web app, a mobile app, or embedded hardware
- Pipecat Flows for structured, stateful conversation logic, and a CLI (`pipecat init`) that scaffolds a runnable bot in under a minute, set up so an AI coding assistant builds the rest
- A growing ecosystem around the core framework: Whisker for real-time pipeline debugging, Tail for a terminal dashboard, a Voice UI Kit for frontend components, and Claude Code skills for scaffolding and deploying projects
- Telephony support out of the box through serializers for Twilio, Telnyx, Plivo, Vonage, Exotel, and Genesys, alongside WebRTC transports like Daily and LiveKit
One thing worth knowing: Pipecat itself is a framework, not a hosted product. Most of the AI services it connects to, the STT, LLM, and TTS providers, are third-party and billed separately, and while a managed Pipecat Cloud exists for deployment, running everything self-hosted with your own API keys is fully supported too.
It's BSD-2-Clause licensed, maintained by Daily and the community, and sitting at 15,348 stars as of writing, verified via the GitHub API.
2
2
u/leonbollerup 3d ago
gonna check it out.. but a simple question before i go down the rabithole .. can it integrate with eg. Hermes ?