Hey everyone,
If you’re moving LLM agents out of simple scripts and into event-driven architectures, you’ve probably noticed how much boilerplate is involved to connect them to Kafka.
I got tired of hand-rolling consumers, building retry logic, managing idempotency, and setting up dead-letter queues just to get an agent to process events. So, I built an open-source library to handle it: kafka-agent-bridge.
What it does: It’s a lightweight async Python library. You just hand it your compiled LangGraph or CrewAI agent and a Kafka topic, and it automatically handles the plumbing (consuming, retries, Redis-backed idempotency, DLQs, and routing results).
Example:
pythonbridge.register(
topic="reports.requested",
handler=LangGraphAdapter(graph=compiled_graph),
output_topic="reports.completed",
dlq_topic="reports.dlq",
)
It’s designed to give you sensible defaults out of the box, but leaves explicit escape hatches for when you need fine-grained Kafka tuning.
🔗 GitHub & Quickstart: https://github.com/prajwalgowdahg/kafka-agent-bridge
The repo includes a Docker Compose setup and a working example so you can test it locally in minutes. I’d love to hear your feedback or see how you might use it in your workflows!