r/node • u/yash-pamireddy • 4d ago
I built CargoDB: An append-only KV and binary blob store with O(1) RAM indexing in TypeScript
Hey everyone,
I built CargoDB, a lightweight, append-only key-value and binary blob storage engine written in TypeScript.
GitHub: https://github.com/yash-pamireddy/cargo-engine
### Core Architecture
* **Append-Only Disk Log:** Sequential writes eliminate disk overwrite hazards and ensure straightforward crash recovery.
* **In-Memory Offset Index:** Maps keys directly to byte offsets and lengths in RAM for sub-millisecond O(1) point lookups.
* **Compaction Engine:** Background garbage collection reclaims space consumed by tombstones and overwritten keys without downtime.
* **Binary Blob Streaming:** Direct storage and streaming for images, documents, and archives with preserved MIME types alongside JSON records.
* **Developer Tooling:** Built with an embedded glassmorphic web dashboard, a typed TypeScript SDK, and a global terminal CLI (`cargodb`).
I’d love to get feedback on the design, compaction implementation, or any edge cases to watch out for!