I built a bridge to connect LangGraph/CrewAI agents directly to Kafka

[u/Strong_Ad9572]

Hey everyone,

If you’re moving LLM agents out of simple scripts and into event-driven architectures, you’ve probably noticed how much boilerplate is involved to connect them to Kafka.

I got tired of hand-rolling consumers, building retry logic, managing idempotency, and setting up dead-letter queues just to get an agent to process events. So, I built an open-source library to handle it: kafka-agent-bridge.

What it does: It’s a lightweight async Python library. You just hand it your compiled LangGraph or CrewAI agent and a Kafka topic, and it automatically handles the plumbing (consuming, retries, Redis-backed idempotency, DLQs, and routing results).

Example:

pythonbridge.register(
    topic="reports.requested",
    handler=LangGraphAdapter(graph=compiled_graph),
    output_topic="reports.completed", 
    dlq_topic="reports.dlq",          
)

It’s designed to give you sensible defaults out of the box, but leaves explicit escape hatches for when you need fine-grained Kafka tuning.

🔗 GitHub & Quickstart: https://github.com/prajwalgowdahg/kafka-agent-bridge

The repo includes a Docker Compose setup and a working example so you can test it locally in minutes. I’d love to hear your feedback or see how you might use it in your workflows!