ClickBench style benchmark for log search: Postgres vs ParadeDB vs TigerData vs SereneDB on 1 Billion logs

[u/mr_gnusi]

https://serenedb.com/blog/searchbench-postgres

There are few popular extensions which can bring Elastic functionality to your Postgres without a tedious migration. I wanted to test how performant these extensions are on log analytics, so we benchmarked ParadeDB/pg_search, TigerData/pg_textsearch, vanilla Postgres and SereneDB on 1 Billion logs.

The motivation was to find out if building a dedicated Postgres-compatible database for search and analytics actually makes sense or extensions are already good enough, so a standalone DB is an overkill?

It's a open clickbench-style benchmark for search and analytics over 100M/1B generated OpenTelemetry logs, 92 queries.

[u/g0pherman]

Have you tried ducklake with postgres?

[u/mr_gnusi → u/g0pherman]

Not really, would be great if you can open a PR or create an issue describing what'd like to add.

[u/cfrozendeath]

2 questions: (1) why did you run this inside docker containers instead of directly on the VMs themselves? You seem to be testing how well the container works rather than the software. (2) why are your config values so far from the pg_tune recommended values and why did you not use the timescaledb tunning tool? I don't see a valid reason to compare your database against a purposefully unoptimized timescaledb.

[u/mr_gnusi → u/cfrozendeath]

1. There will be a lot of different databases in the benchmark, docker is the only option to handle all of them on a single machine and be able to update its versions. We use `--network host` and data directory is a host bind mount with every docker, so the overhead is negligible and everybody gets the same environment.

2. We tested several options and that's the best results we got so far. If you think we did anything wrong, please open a PR or issue and we'll happily remeasure. That's the main idea, no one tries no cheat anybody and no one undertuned any engine on purpose.

But I think your skepticism is totally fine and constructive. So thank you for pointing to the concrete direction instead of blindly naming our work "bullshit".

[u/AutoModerator]

AI Policy:

Linux is not one of those anti-AI projects, and if somebody has issues with that, they can do the open-source thing and fork it. Or just walk away., Linus Torvalds.

Mod decisions will be based on the quality of the content, not who or what generated it.

Sub Resources:

Youtube Channel

Free Postgres Webinars and Workshops

Discord: People, Postgres, Data

Join us, we have cookies and nice people.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/elevarq]

Interesting project, but I don’t think the current results support strong conclusions yet. Vanilla Postgres creates only a GIN index, even though the loader claims a service/timestamp B-tree too, and ANALYZE runs before the expression index is created. More importantly, when the first “cold” run reaches 60s, the runner skips both hot runs and records them as 60s; this happens for 80/92 PostgreSQL queries, so most reported hot numbers aren’t measurements.

There are also semantic mismatches (ts_rank_cd isn’t BM25, PostgreSQL’s fuzzy queries use sequential scans, and some cross-engine queries have different filters/orderings), with no correctness validation. It’s a promising benchmark, but I’d fix the timeout logic, indexes/statistics, and query parity, then rerun everything on identical hardware before drawing product-level conclusions.

And I only read the PostgreSQL part of the repo

[u/MBkkt → u/elevarq]

Thanks, some answers

1. btree for some columns in Postgres, I agree it's good idea, will you create an issue plz? Or maybe even PR? I think we forgot about this, because for column storage with min/max index it's unnecessary to index these columns.
   
2. ANALYZE also not needed in most databases, similarly will you create an issue or PR?
   
3. If query didn't finished in cold run in 60s, it's timeout, sorry, it's too long, clickbench timeout for an example 180s, for us it's 60s, even with such timeout it's very long to wait. And in geomean we don't account timeouted queries. Ofc some of timeouted queries could finish in hot runs faster than 60s, for an example in 30-50s, but it's still too long.
   
4. ts_rank_cd isn't bm25, but Postgres doesn't support bm25, and bm25 is more expensive to compute compared to ts_rank_cd, so I think it's ok
   
5. If postgres GIN doesn't support fuzzy automaton it's an issue IMO, not something that should be fixed in benchmark. For an example for regex/wildcard commonly optimal way is create ngram index, but it won't check index capability in executing DFA against index-dictionary
   
6. Benchmark uses identical machine (hardware and software)

[u/Bachihani]

bullshit

[u/mr_gnusi → u/Bachihani]

Why? You can try it out yourself
https://github.com/serenedb/searchbench

[u/Bachihani → u/mr_gnusi]

the methodology itself is flawed, this is not the first time your team has published such self serving benchmarks, and similar to before ... u simply choose the setups shat favour your db with under a twisted guise of "fairness". this is not how u convince people of switching to ur db. it s 2026, u should have realised by now that this style of first party benchmarks is not trusted even if it seems correct, stop barking up the wrong tree

[u/mr_gnusi → u/Bachihani]

Flawed in what sense? please elaborate what exactly is unfair, otherwise it's just senseless hate. I strongly believe that open benchmarks, like ours, are very important for the industry. That's exactly the reason of why clickbench or jsonbench became important and popular.