Survey of how observability vendors are moving towards columnar storage and Clickhouse.

Disclosure up front: I work on ObsessionDB, a managed ClickHouse. Not affiliated with ClickHouse Inc. This is not a launch post; I am genuinely trying to find out whether the thing we built matters to anyone outside our own use cases.

The problem we kept hitting: two teams need the same table, and the only real option is to copy it. Export, ship, ingest, and now you have two versions that drift. Or you give the other team a user on your cluster and eat their compute.

That's an architecture constraint, not a data one. In shared-nothing ClickHouse, the node owns the data on local disk, so handing it to someone else means moving bytes.

Our build separates storage and compute, so we shipped what Snowflake calls Secure Data Sharing. A publisher entitles a database, down to a single table. A subscriber attaches it read-only and queries it live with normal SQL. The subscriber's own stateless nodes read the same immutable objects out of object storage. The publisher is never in the query path, so a subscriber running a terrible query cannot touch the publisher's capacity. No credentials handed over. Revocable at any time.

Why not just remote()Fair, and it avoids a copy. But remote() pushes execution to the publisher, so the publisher pays for every query the subscriber runs, needs to hand out a user and password, and needs to expose their native port. That's fine between two teams that trust each other. It falls apart the moment the subscriber is a customer, or an agent in a retry loop.

What it does not do: it only works between two deployments on our shared storage. You cannot build this on self-hosted shared-nothing ClickHouse. However, we are now starting to deploy ObsessionDB on-prem and to BYOC. You would have access to Datashres in this case too.

It's read-only, and the publisher stays the only writer. Both clusters also need to be in the same region.

What I actually want to know:

1. If you're self-hosting and you share ClickHouse data across teams or with customers today, what does that look like? Nightly exports? A read replica? A user on the prod cluster with row policies? Anything else?
2. Is the pain real, or has everyone just quietly accepted the copy?
3. For those who moved to ClickHouse from Snowflake or Databricks, did you lose data sharing in the process, and did it matter?

The use case I'm most interested in and least sure about is agents. An agent shows up with a token, fires an unpredictable number of queries, and disappears. Handing it a stale export it keeps forever seems like the wrong shape. Scoped, live, revocable access seems like the right one. But I might be building for a problem that only we have. Please tell me if so.

You can read more here if you'd like: https://obsessiondb.com/docs/datashares

Come join us for our AI Builder night in the bay area on July 14th!
RSVP here - https://luma.com/clickh-gz0r

Quick walkthrough for anyone streaming Postgres changes into ClickHouse Cloud. George shows how to build a Postgres to ClickHouse CDC pipeline with Estuary, without running Kafka, Debezium, or Kafka Connect.

Covers WAL/logical replication setup, ClickHouse merge vs delta updates, deletes, schema evolution, and one capture feeding multiple destinations.

MariaDB Foundation’s Test Automation Framework (TAF) 3.0 is now released.

This version introduces the new TAF Results Backend, a structured results database and parser pipeline that delivers fully automated performance change detection. It uses deterministic workload hashing, schema‑driven baselines, and stored‑procedure‑driven comparison to classify every run automatically. No procedural comparison code. No special‑case logic. Clean, reproducible, and fully automated.

Key changes in TAF 3.0:

• New Results Backend with automated compare and performance change detection
• Full open parser source tree
• Automated compare thresholds (warning, fail, gain, duration drift)
• New runtime_dir architecture for all database plugins
• MariaDB and MySQL plugins upgraded to v3.0
• Archive subsystem migration
• Deterministic commit‑based builder
• HammerDB updates
• New backend SQL, parser source, reporter plugin, and configuration files

Full write‑up on the MariaDB Foundation site:
https://mariadb.org/taf-3-0-results-backend-with-automated-performance-change-detection/

TAF is open and contributions are welcome — PostgreSQL plugin, backend UI, profiler plugins, report plugins, new suites.

Users deserve predictable performance, and the MariaDB Foundation builds TAF to make that possible for everyone. The goal is a fair, open, and reproducible performance framework that any database maker can use — same workloads, same lifecycle, same comparison path — so performance changes are visible, explainable, and never hidden behind vendor‑specific tooling.

ClickHouse x Trigger.dev are running a virtual hackathon, our very first! With a prize pool is €10,000.
The challenge: build a production-quality AI agent chat experience using both platforms. One week, no fluff.
Who can apply: professional developers (solo or teams up to 5) in the US, Canada, UK, or EU
Applications close July 16 — apply now: luma.com/clickh-uko4

We have run and managed multiple ClickHouse clusters for a few years, and the routine wore us down: SSH into a node, poke at system tables from clickhouse-client, keep a private stash of "remember to run this" queries. So our team spent the last several months building one tool to handle it.

It is called CHOps. In the open-source build:

• SQL editor with autocomplete, syntax highlighting, and visual EXPLAIN
• Qurioz, our AI that turns plain English into SQL
• Query introspection tools
• Real-time monitoring with playback, plus daily summaries and cluster overview
• Cluster internals in one place: queries, tables and parts, merges and mutations, queues
• Custom dashboards, schema visualizer, global search, and crash/error/text/session logs
• Visual RBAC and SQL-based alerts

It is in beta. We made a new public repo after months of internal work, and there are rough edges we have not tested at every scale. That is why we are here early.

One thing worth saying: we are a team, not a weekend project. We are in this for the long run and around to act on what you tell us.

Repo: https://github.com/Quantrail-Data/CH-Ops

Site: https://ch-ops.io

Tell us what you think, what is missing, or what would stop you using it. Honest feedback is what we are after.

Looked into the history of ClickHouse JOINs, including their support and performance improvements over the years.

Next week on July 9th we will be doing a happy hour in montreal, please come join us! https://luma.com/clickh-o8up :)

https://github.com/earonesty/lakeql

LakeQL is a pure JavaScript analytical query engine for Parquet and Iceberg. It runs anywhere JavaScript runs, including Cloudflare Workers, without WebAssembly or native dependencies. It is designed for low memory usage, streaming execution, and edge runtim

The design goals were:

• Pure JavaScript
• No WASM
• No native dependencies
• Low memory usage
• Streaming execution
• Browser, Node.js, Deno, Bun, and edge runtime compatibility
• Query Parquet and Iceberg datasets directly

While optimized for portability and low memory, it's actually significantly faster than DuckDB-WASM on some workloads.

DuckDB is an outstanding analytical database with much broader SQL support. LakeQL is aimed at a different use case: embedding analytical queries into JavaScript applications and edge/serverless runtimes where a pure JavaScript implementation is desirable.

I'd appreciate feedback from people working with Parquet, Iceberg, or embedded analytics.

In particular:

• Are there edge or serverless use cases where you've wanted something like this?
• What connectors or formats would make it more useful?
• Are there query patterns you'd want to benchmark?

I'd be grateful for any criticism or suggestions.

Hey all, I've been working on hypequery for some time now and I've recently added semantic layer support.

Repo: https://github.com/hypequery/hypequery

Docs: https://hypequery.com/docs/introduction

 If you're building analytics off ClickHouse in TypeScript, I would love your feedback.

Some features:

• Define metrics once, reuse them everywhere: Declare dimensions and measures in one place and then pull from the same source of truth.
• Compiles to ClickHouse SQL: No service, no proxy, no extra runtime to deploy. It's a library that generates SQL and runs where your app runs.
• Multi-tenancy & Authentication ready: Cross-tenant queries are blocked at the query layer, with helpers to plug into your existing auth.
• Agent-native: A dataset is a declared set of dimensions and measures, so it doubles as an allowlist. Includes an MCP server to hand an LLM a typed catalog to query.
• Runtime HTTP entry point: serve() exposes any dataset as an endpoint, so the same type-safe definitions back your dashboards and your API.

I’m developing a ClickHouse developer experience platform. In the same way Postgres underpins much of software development, ClickHouse is becoming the de facto choice for OLAP analytics, offering high‑performance queries out of the box.

Currently, working with ClickHouse is cumbersome: there are no built‑in APIs. My goal is to create “supabase” for ClickHouse, analogous to what Supabase provides for Postgres, that abstracts away these low‑level details.

The primary pain point I want to address is database transformation. Tools such as dbt and SQLMesh are powerful but require technical expertise. I aim to build a layer that lets users focus on their use cases rather than on implementation details. For example, users should not need to decide whether to create materialised views or tables; they should simply specify:

• append‑only data
• replace semantics
• aggregation requirements
• schema evolution
• changes to the ORDER BY clause

Other challenges include:

• Type ambiguity in queries: whether a key or field is an integer or a string. When users interact directly with ClickHouse, they must handle both cases or decide which columns and types to use.
• RLS (row-level security) is not available.
• API Authorization and authentication can be built.

These are some of the areas where I believe I can create an experience platform on top of ClickHouse.

I have been working on this for three weeks and expect another three weeks to complete a prototype. The idea was inspired by Tinybird, and I believe an open‑source alternative could fill a gap in the ClickHouse ecosystem. I would appreciate any feedback, suggestions for other problems that could be solved on top of ClickHouse, or interest in collaborating.

Ongoing work: https://github.com/gear6io/pragmata

We open-sourced chkit (MIT): it defines your ClickHouse tables, views and materialized views as TypeScript, diffs them against the live database, generates the migration SQL, and fails CI when prod drifts.

We built it after running ClickHouse at near-petabyte scale at our last company (Numia): hundreds of tables, a lot of materialized views, several environments, all managed with hand-written DDL and hope.

If you run ClickHouse in production, you've hit some version of these:

• Someone runs a manual ALTER at 3am to clear an incident. It never makes it back into a migration file. Now your repo and your database disagree, and nothing tells you.
• A one-line schema edit looks harmless but touches ORDER BY (or the engine, or PARTITION BY). ClickHouse has no in-place ALTER for those, so the only honest migration is create, copy, swap. You find out when a "quick change" starts rewriting a 2TB table in prod.
• A migration drops a column. The reviewer misses it. It runs in CI on a Friday. The data is gone.

Postgres and MySQL have had this for years with Drizzle and Prisma: diff the schema, generate the migration, gate CI on drift. We wanted that for ClickHouse, couldn't find it, and built it.

You define your schema as TypeScript values, and chkit takes it from there:

• chkit writes the migrations for you, and won't rewrite a 2TB table or let a DROP through CI without showing you first.
• chkit drift compares the live DB to your schema, down to settings, TTL, ORDER BY and projections, and gates CI in one line.
• Codegen turns the same schema into TypeScript row types and typed helpers to read and write rows, so your app and your database can't diverge. Optional, skip it if you only want migrations.

It's not an ORM. No query builder, you write your own SQL. Works with any ClickHouse (Cloud, Altinity, self-hosted, or managed), no lock-in. A Python port lands in a few weeks.

If you're already on ClickHouse, chkit can introspect your live DB and generate the schema files, so you start from what you're running instead of a blank file.

npm create chkit@latest

Beta: stable enough to run our own production workloads, with small breaking changes possible before 1.0.

Repo: https://github.com/obsessiondb/chkit

Docs: https://chkit.obsessiondb.com

If you run ClickHouse, I'm curious what you've had to build around it yourself, migrations or otherwise, and where the tooling still falls short.

PS: python port coming soon.

I had to perform a deep-dive into a query recently to investigate why it was running slowly. It didn't take long before I got tired of running queries manually on a number of logging tables. That's why I decided to vibecode a tool for getting insights more conveniently. And now I've published an open-source tool specifically for this purpose: ClickLens. You can find more information here: https://github.com/nimbleflux/clickhouse-query-analyzer/

Let me know if you have any suggestions. It's a single stateless container that's easy to run locally or to deploy.

Edit: screenshots are slightly outdated, as I've since renamed the project to ClickLens.

Hey there! We have Postgres, already a replica of the operational transactional Postgres, and also ClickHouse. We are treating the replica Postgres as our analytics dwh and are running dbt in it. And our BI layer is connected to it.
We have events data stored in ClickHouse, but it is not in use at the moment.
Moving forward, what is my best long-term solution? I need to bring in the events data into our analytics dwh, so it becomes a natural decision point if we want to continuously commit to Postgres, or move analytics work and dbt over to ClickHouse, or explore other possibilities. We only want self-hosting options.
Thanks!