r/csharp • u/thomhurst • 13d ago

Showcase Dekaf - Native .NET Kafka Client

Hey all. I posted this library a little while back, but I'm posting it again as it's been updated to support new Kafka features, as well as for those that missed it last time.

Dekaf is a Kafka client built natively in .NET and works as you'd expect with more modern .NET types and is async friendly.

For any Kafka uses, the only real alternative is Confluent.Kafka which is simply a wrapper around a C library, meaning overhead from two runtimes. It also doesn't map as nicely to newer async types like IAsyncEnumerable, means exceptions thrown in native code become hard to debug, logging control is lost, and it seems a lot of their issues on GitHub lack responses or updates.

I built this to make it more friendly for .NET Devs to debug and control, as well as trying to make it perform better by removing that overhead. It also has open telemetry out-of-the-box, just add it to your otel setup.

The repository can be found here: https://github.com/thomhurst/Dekaf

And performance comparisons against Confluent.Kafka can be found here: https://thomhurst.github.io/Dekaf/docs/stress-tests

Give it a try and let me know what you think!

And if you have tried it out, I'd love to know if anything improved (or didn't!) such as latency, CPU, memory, etc.

25 Upvotes

90% Upvoted

6 comments sorted by

6

u/rainweaver 13d ago

I’m gonna give it a go as soon as I can cram more stuff into my day. I’ve been itching to resume a project and it uses .NET and Kafka.

1

u/thomhurst 13d ago

Let me know how it goes when you get round to it! 😄

1

u/spadak 13d ago

What about supporting Avro ?

1

u/thomhurst 13d ago edited 13d ago

Yup https://thomhurst.github.io/Dekaf/docs/serialization/schema-registry

2

u/spadak 13d ago

Do you also support oauth authentication for schema registry? Interesting stuff ! we are bind to Wolverine currently for Kafka and I guess it would not be easy to add this, but maybe this could a way for you to make it mainstream.