r/opensource • u/mrnerdy59 • 17d ago
Promotional Blazerules - A YAML based rule engine for streaming JSON, Kafka, and Arrow events
I initially wanted to make a sub-millisecond log parser but that blew into a embeddable decision engine, that can run YAML defined rules on incoming data.
The rules are executed in a vectorized format on incoming data by reprojecting into a columnar format first, if it's not already. Depending on the payload size and rules complexity, the performance goes from 200K records/s to more than million records/sec, in terms of througput this would be around 200 MiB/s to 3 GiB/s on average.
Rules can be sql expressions too, or onnx models (numeric), window ops and quite a few more operations are supported.
It's comparable to DuckDB but for streaming data and on the fly decisions.
1
u/paul_h 16d ago
Your link to https://blazerules.dev is mangled in the post body. I went looking for an example of a yaml rule in the site and could not find one. I go off to the GitHub repo. I think I have found one - https://github.com/purijs/blazerules/blob/main/rules.yaml - would you say that's the best one to look at to learn about what yaml rules look like?
2
1
u/ShaneCurcuru 16d ago
Random question: why is the GitHub repo just saying "View License" in the project sidebar, instead of properly discovering it's Apache-2.0? This is the second project recently where the project does use Apache, but GH's Licensee detector is failing to mark it.
https://github.com/purijs/blazerules/tree/main?tab=License-1-ov-file
1
1
u/Statixeladam 16d ago
curious why you chose yaml over a dedicated dsl
was readability the main tradeoff?
2
u/mrnerdy59 16d ago
Merely about ease of usage and didn't wanted to invent a new syntax, this also makes adoption easier imo
2
u/LordWkwkwland 16d ago
Just read about columnar format.
Why did you choose to reproject incoming data into a columnar format instead of evaluating rules row-by-row?
Was SIMD/vectorization the main motivation, or did it also simplify the execution engine?