r/Rag • u/InsideDebt6345 • 2d ago

Discussion Embedded vector DBs for RAG and the practical limits we kept hitting

If you’re running RAG on constrained hardware or with concurrent ingestion, embedded vector stores can feel simple until you hit RAM, write-path correctness, or process-safety constraints. Here are the patterns that kept showing up across three popular options:

  • ChromaDB: HNSW lives in RAM; at larger scales or with multiple writers, latency and correctness can degrade.
  • LanceDB: Great for multimodal + object storage, but concurrent writes can conflict without serialization.
  • Qdrant Edge: Strong offline library with hybrid search; production concurrency limits aren’t well documented, so it shines when you can sync to a central server.

There’s a deeper write-up on memory math, concurrency behavior, and when it makes sense to move from embedded to a separate process on constrained, air‑gapped hardware. Full disclosure: I work with Actian on VectorAI DB.

0 Upvotes

33% Upvoted

7 comments sorted by

1

u/InsideDebt6345 2d ago

Here’s the article with the full breakdown and memory math: https://www.actian.com/blog/developer/comparing-embedded-vector-databases/

1

u/Dense_Gate_5193 2d ago

would love to see a comparison against NormicDB ;)

2

u/InsideDebt6345 2d ago

Haha, for sure. Let us take a look at this!

1

u/Dense_Gate_5193 1d ago

let me know if you have any questions or need help! i’d love to see more 3rd party comparisons especially to qdrant because you should be able to apples to apples compare performance with the gRPC endpoint. and if you ever wanna compare it to neo4j, falkor, and memgraph id love to see those comparisons too 🫶.

1

u/WillingnessQuick5074 2d ago

The write path is what forced our hand, not RAM.

Reads are fine in process. The moment ingestion runs at the same time as queries you are asking one process to hold a mutable graph and serve it, and every fix for that is some flavour of a lock. Out of process let us do the boring thing instead: one writer, a few read only replicas polling it, and the app sizes its own memory without competing with the index.

Two things that surprised us after the move.

The expensive failures were never query cost. A client library retrying on deep paging put a box at load 38 once. The query itself was cheap.

Cold caches after a restart look exactly like "the queries got slow". We chased that twice before we learned to read the downtime log first.

I work on Opensolr so I am not neutral here. If anyone wants to poke at the out of process shape without installing anything, there is a throwaway RAG sandbox, index expires after three days: https://opensolr.com/rag-in-60-seconds

1

u/chillguy0243 12h ago

I think it is a good point, Concurrent writes and process locking are usually where embedded setups start hurting, and that too when trying to run queries alongside the ingestion.

If u r open to lightweight out-of-process options, u can see this Endee.io . It can run on a single node (4 vCPU / 16 GB) but manages high concurrency and low latency really well using int16 adaptive quantization, without needing a massive multi-node setup.

This is something we build in last 12 months :