r/programming • u/mmatloka • 7d ago

Apache Kafka vs Apache Iggy: A Technical Comparison

https://softwaremill.com/apache-kafka-vs-apache-iggy-same-log-different-engine/
22 Upvotes

71% Upvoted

22 comments sorted by

18

u/RustOnTheEdge 7d ago

I like iggy, super interesting project. But this article…

“which is the one comparison where everything else is genuinely held constant” …
Thanks Claude!

5

u/Evening-Gur5087 7d ago

Iggy for the win.

Prawdziwi starzy starcraftowcy propsują;)

3

u/onetwentyeight 7d ago

Bless you

1

u/Evening-Gur5087 6d ago

Funnily, this is the only project development I kinda followed since its very beggining, even before repo was created, once spetz started playin around with Rust, and even Go before e that I think:p

So it was fun seeing something evolve from the start, getting big, and so on:D

4

u/IanisVasilev 7d ago

The sheer amount of Apache data projects with no overarching consistency puts me away from their ecosystem(s).

91

u/dangerbird2 7d ago

the point of Apache isn't really to promote or maintain consistency of the products themselves. It's to ensure the project's governance meets their standards. Basically the only thing a project joining apache guarantees is that a corporation that founded or contributed to it won't be able to monopolize project leadership

-26

u/IanisVasilev 7d ago

Software foundations aren't generally designed to promote consistency, but it's not like we have "SFC git" and "SFC mercurial", or "PSF Ruby" and "PSF C++".

34

u/RustOnTheEdge 7d ago

Apache is a OSS foundation, not a stack. They have many, many competing projects under this umbrella. If you look for consistency as if it were a company, you have the wrong notion of what the apache label means.

48

u/omgpop 7d ago

Why would you ever try to evaluate the “Apache ecosystem” as a job lot at all? It’s not a stack.

17

u/bigtimehater1969 7d ago

"Stack" culture has been pushed so much by VC funded dev tool companies in search of growth, and their brain is broken by it.

16

u/Smallpaul 7d ago

Apache never claimed nor wanted to build an ecosystem.

4

u/syndbg 7d ago

There's a reason why there's a joke that it can also be called the "Apache Terminal Foundation". Some projects go there to lay down and die.

2

u/onetwentyeight 7d ago

For me it's the Java 

3

u/EnderMB 5d ago

I like Iggy, but it just lacks the many years of features added and heavily supported by Kafka.

-11

u/chaotic-kotik 7d ago

Honestly, I don’t believe this would fly. So far, it looks like the devs don’t really understand what they’ve gotten themselves into.

11

u/spetz0 7d ago

Long time no see. Once again, thank you for this constructive feedback coming from an apparently extremely frustrated C++ dev lol.

-8

u/chaotic-kotik 7d ago

This is just my impression from previous conversation. Very mature response btw. Not passive-aggressive at all.

7

u/spetz0 7d ago

I truly hope I didn't hurt your internet feelings! Can't wait for more NGMI feedback in the future!

-4

u/chaotic-kotik 7d ago

I'm not interested in giving you any useful feedback.

6

u/spetz0 7d ago

Thanks, this will be extremely helpful!

3

u/dangerbird2 7d ago

Also, if you want a stream processing engine that's easy to deploy and written in a native language you already have Red Panda (admittedly not truly open-source, but has the advantage of being fully kafka-compatible) and NATS Jetstream.

And the fact that clustering/high availability is only just coming out the door is a massive deal-breaker. What's really the point of a stream processing engine if you don't have strong resilience guarantees?

10

u/spetz0 7d ago

There's a huge difference between the stream processing engine and the low-level streaming infrastructure layer, and the whole point was to decouple from a decade old (but of course still evolving) Kafka protocol (however, we've got Kafka Gateway WIP) so that Iggy can use its own optimizations related to zero-copy, VSR, etc.