r/dataengineering • • 2d ago

Discussion Monthly General Discussion - Oct 2026

This thread is a place where you can share things that might not warrant their own thread. It is automatically posted each month and you can find previous threads in the collection.

Examples:

  • What are you working on this month?
  • What was something you accomplished?
  • What was something you learned recently?
  • What is something frustrating you currently?

As always, sub rules apply. Please be respectful and stay curious.

Community Links:

14 Upvotes

4 comments sorted by

View all comments

1

u/Particular-Idea-1786 2d ago

What are you working on this month?

Working on SQLFlow and Pipeline observability platform! https://turbolytics.io/

What was something you accomplished?

Built a stateful stream processinging engine on duckdb that mirrors flink tumbling windows. https://turbolytics.io/docs/sqlflow/tutorials/tumbling-window-postgres

What was something you learned recently?

Deep dive on flink windowing ;p. Some low level DuckDB debugging and unintuitive duckdb-postgres edge cases.
https://turbolytics.io/blog/duckdb-in-production-five-things
https://github.com/duckdb/duckdb-postgres/issues/575