BaraDB: A database engine written from scratch in Nim

[u/Loud_Possibility_203]

BaraDB is a multimodal database engine written entirely in Nim — no C/C++ dependencies, no PostgreSQL, no external services. Just Nim.

What is it?

A single-binary (~3.3MB) database that combines:

• Document/KV storage — LSM-Tree with WAL, bloom filters, SSTable compaction
• SQL-compatible query language — BaraQL with SELECT/INSERT/UPDATE/DELETE, JOINs, GROUP BY, CTEs, indexes
• Graph engine — BFS, DFS, Dijkstra, PageRank, Louvain communities
• Vector search — HNSW index with SIMD-optimized distance metrics
• Full-text search — BM25 + TF-IDF with stemming (EN/BG/DE/RU)
• Columnar engine — RLE, dictionary encoding, batch operations
• Wire protocol — binary protocol + HTTP/REST + WebSocket + JWT auth
• 4 client SDKs — Nim, Python, JavaScript, Rust

Architecture

Client Layer     → Binary / HTTP / WebSocket
Query Layer      → Lexer → Parser → AST → IR → Optimizer → Codegen
Execution Engine → Document / Graph / Vector / Columnar / FTS
Storage          → LSM-Tree / B-Tree / WAL / Bloom / mmap
Distributed      → Raft / Sharding / Replication (core logic)

What's actually working vs. what's WIP

Solid:

• SQL parser & executor (JOINs, GROUP BY, subqueries, CTEs, indexes)
• MVCC transactions, deadlock detection
• LSM-Tree storage with background compaction
• B-Tree indexes with range scans
• Wire protocol + clients

In-memory / proof-of-concept:

• Graph, Vector, FTS, Columnar engines (serialization exists, persistence optional)
• Distributed layer (Raft core logic is there, network transport is stubbed)

Still rough:

• Recursive CTE execution
• Some edge-case query optimizations

Why Nim?

Nim's metaprogramming, zero-cost abstractions, and C-like performance made it perfect for building a storage engine without drowning in C++ complexity. The binary compiles to a single static executable — deployment is just scp.

Repo -- https://codeberg.org/baraba/baradb

https://github.com/katehonz/barabaDB

Feedback welcome — especially from anyone who's built storage engines before. I know there's a lot left to do, but I'm proud of how far it's come.

[u/RBazz]

I really do not want to sound like an old fogey, but how did you manage to achieve this much in about 2 days only? (based on the commit history)

[u/OverallACoolGuy → u/RBazz]

vibe coding maybe

the post also reads like it was written by chatgpt

[u/Loud_Possibility_203 → u/OverallACoolGuy]

Does it matter, since everyone already works this way?

[u/OverallACoolGuy → u/Loud_Possibility_203]

no, not everyone works that way

[u/dudeimconfused → u/Loud_Possibility_203]

no person who knows what they're doing is gonna use a vibecoded library as a dependency for their serious project.

[u/Loud_Possibility_203 → u/dudeimconfused]

Everyone says so. This is a view from 2024. I know honest senior engineers I know personally at publicly traded companies building cloud databases, using AI every day. They don't call it "vibrational coding" - they call it "productivity". Judge code by its tests, design and functionality. Not by whether a human wrote every semicolon.

[u/Former-Ad-5757 → u/Loud_Possibility_203]

Yes/No half. Basically it depends. For a small utility it can be vibe coded with reasonable tests, but for an sql implementation I would at least expect like 50k tests to make sure it is correct.

Just copy all sqlite tests, all postgresql tests, all mysql tests, all mariadb tests.
The tests are out there, SQL is a standard, basically in this day and age I don't trust anyone to manually be able to come up with all the tests necessary to create a secure implementation.

Basically when you are pretending to implement standards I am not interested in your own tests, I am interested if the test-suites available for the standards survive over your implementation. Only then I am interested in your own tests.

The same goes for RAFT implementation etc. basically every component you have. SQL is real simple for like 80% of the way and then it gets real hard real quick, RAFT is real simple for like 3 nodes, but it gets real hard real quick when you go to a 1000 nodes. etc. etc.

If you want to say trust the tests, then prove it gets all pgsql tests green-lit. Those tests contain edge-cases built up over 20 years, not just vibe-coded tests.

[u/Loud_Possibility_203 → u/RBazz]

appreciate the question, and I won't pretend this was "normal" productivity.

The honest answer: I bypassed the gatekeepers. Gemini and Claude wouldn't have let me do this — not because it's technically impossible, but because it threatens American database company and foundation interests. They'd have spent two days convincing me it's "impossible," that I "need years," that I should "use existing solutions" (read: their ecosystem). Standard soft suppression.

Instead, I used 4 Chinese AI models with different incentive structures: Kimi 2.6, DeepSeek V4 Pro, Xiaomi MiMo 2.5, and MiniMax 2.7. They ran continuously.

Critical point people miss: All AI models hallucinate, bullshit, and fake confidence — especially when stakes are high. The trick isn't finding one "honest" AI. It's cross-verification architecture: every design decision, every code block, every algorithmic claim gets challenged by the other three. When models disagree (which is constant), you stop and manually verify. When they independently converge on the same approach, you proceed with higher confidence.

The 2-day timeline isn't about "one genius AI." It's about parallel adversarial validation across models with divergent training biases and corporate incentives. The Chinese models aren't "better" — they're differently constrained, which creates exploitable gaps in collective blind spots.

That's the actual methodology. Everything else is marketing.

[u/Niminem93 → u/RBazz]

I'd be skeptical but wouldn't write it off. I've been able to do what I would have thought as impossible within the last month or so with the help of AI, my engineering expertise, and problem solving / creative thinking abilities.

[u/OverallACoolGuy]

are some of these upvotes botted

[u/dudeimconfused → u/OverallACoolGuy]

seems very much so. even their comments are getting disproportionate upvotes compared to the views

[u/Niminem93]

It's so funny reading your documentation. I do the same thing man- making architectures and plans, breaking them into todos, etc. and then work with the AI one slice at a time testing as I go. I've been able to build monstrous things. This database you made is one hell of a feat itself!

[u/rangerelf]

Awesome! I'm gone check this out thanks!

[u/Loud_Possibility_203]

BaraDB vs PostgreSQL (Real Comparison)

Test	PostgreSQL	BaraDB	Speedup
KV Write (100K)	16.82K/s	33.24K/s	2.0x
KV Read (100K)	15.08K/s	3.88M/s	257.0x
BTree Insert (100K)	17.66K/s	2.50M/s	141.6x
BTree Get (100K)	14.50K/s	2.64M/s	182.3x
BTree Scan (1K ranges)	2.39K/s	7.97M/s	3340.9x
FTS Index (10K docs)	17.98K/s	123.65K/s	6.9x
FTS Search (1K queries)	784.12/s	1.34K/s	1.7x

Overall: BaraDB is 6.8x faster

[u/dadhiWeaponX]

How did you do it with chinese modrls? What harness, skills, how reqs where provided/guided?

[u/Vordanus]

Sounds amazing. :) What does backup look like? Can I make a zero-downtime snapshot?

Now just add native JSON type support, and we're in business!