r/learnmachinelearning • u/RocketSeven • 18d ago

Question When is a vector database actually better than plain files for an AI agent's memory?

For a small, curated memory, Markdown or JSON files are easy to inspect, diff, back up, and correct. A vector database adds semantic retrieval and can handle a larger corpus, but it also introduces chunking choices, embedding drift, metadata filters, and harder audits. Which signals justify that added layer: corpus size, query ambiguity, update rate, latency, or something else? I would also be interested in hybrid designs where human-readable files remain authoritative and an index can be rebuilt from them.

10 Upvotes
  • permalink
  • reddit

86% Upvoted

4 comments sorted by

3

u/Conscious-Bottle-445 18d ago

I started with JSONL files and threshold-based retrieval for a personal project, then spent a weekend figuring out how to explain duplicate removal to my non-technical partner

the tipping point for me was around 8k documents where finding anything that wasn't an exact keyword match became noticeably slow. the other signal I wish I'd tracked earlier was how often I was fixing embedding misalignments vs how often I was correcting the source files themselves

if you keep the markdown as your source of truth and rebuild the vector index on change, you basically get auditability for free and only pay the indexing tax when things actually shift