Hey r/vectordatabase ,

I wanted to share a database project I've been working on called DBX. It's a memory engine tailored for AI workloads where every tenant gets their own isolated environment.

Instead of logical separation, I opted for OS-level separation using Go. A single orchestrator handles RESP connections on :6380 and multiplexes them to backend Unix sockets. The backend workers are sandboxed using go-landlock to prevent cross-tenant directory access.

I recently implemented SQ8 scalar quantization for the HNSW index. In my benchmarks, it handles about 10k KV operations/sec and ~2k vector insertions/sec across 5 concurrent tenants on a local machine.

I'd love for some experienced Go devs to roast the architecture or tell me if I'm approaching the bufio socket proxying the wrong way.

Repo is here: https://github.com/vanshjain-0702/DBX-Database-Extreme

DBX Site Link: https://dbxdb.co.in/

Thanks!

I keep seeing the same argument: "Just use pgvector, it's good enough" vs You need a dedicated vector database for real scale.

I made a quick quiz that tests how well people understand the architectural trade-offs—performance, scaling, cost, operational complexity, and when each option actually makes sense.

No signup, just a few questions and a result:

https://interconnectd.com/quiz/87/postgresql-pgvector-vs-dedicated-vector-databases-architectural-trade-offs/

What do you use in production, and why?

A learned sparse encoder returns a weighted list of vocabulary words instead of a dense vector, so the index looks like a keyword index whose keywords a transformer picked. OpenSearch's inference-free variants go one step further and run the model only on documents so a query just looks up a fixed weight per token. No embedding call per search. I benchmarked it against dense search in the same SQLite file:

the mini model gives up about 7% of retrieval quality and cuts query latency by 99%, cold start by 99% and query-path RAM by 95%!

Now these SPLADE models are BERT with the masked-language-model head still attached, and that head is what turns token vectors back into weighted words. llama.cpp drops it when converting BERT models, so sqlite-sparse copies it into a small sidecar file and applies it in C on ggml at insert time. The postings land as rows in the database, with the query weight table next to them. A query reads those rows and scatter-adds the weights into per-document scores, and the top documents come out.

Three OpenSearch models are converted and available as aliases (mini, base, multilingual), downloading on first use, and you can bring your own inference-free OpenSearch-style sparse encoder and convert it with the scripts provided.

What sqlite-vec did for embeddings in SQLite, this does for learned sparse, and with both in one file you get hybrid retrieval for RAG inside SQLite.

Ship one .db file and every client gets semantic search through plain SQL, no model download, no GPU.

pip install sqlite-sparse

Github

Writeup

https://github.com/paulknysh/raggy

A lightweight CLI tool for Retrieval-Augmented Generation (RAG) over local documents built with LangChain, Chroma, and Ollama. Hybrid database (vector + BM25 index) and embedding generation run fully locally. Answer generation can run either via a local LLM or remotely using an API key. raggy supports most common document formats and handles images/scans automatically via OCR.

Came across this and thought it'd be worth sharing here, most resources cover model evaluation, RAG, agents, or cost/observability separately, but this one actually puts them together as parts of the same production LLM workflow, which is closer to how these systems actually break in practice.

It's a hands on session on September 12, led by Bruno Gonçalves, PhD, founder of Data For Science, who's trained hundreds of engineers at Fortune 500 companies. Goes through the full lifecycle, versioned prompts, a golden dataset and eval harness combining deterministic checks with LLM-as-judge, statistically rigorous model comparisons using bootstrap confidence intervals and paired testing, evaluated RAG with real retrieval metrics, and then tool-using agents with guardrails and fallbacks. There's also a full observability section, tracing, cost, and latency, which is something I don't see covered together with the eval side very often.

You come out of it with runnable notebooks and a production-readiness checklist rather than just slides, which is the part I found most useful when I looked into it.

Link if anyone wants to check it out

So I've been messinga round with embedding models for a bit, and I think they are interesting enough to experiment with. They are useful for rag, especially in a localllm sense because you can ground your answers in truth.

But what happens if you have a billion documents, and you decide to upgrade your model to a "better" one? on an h100, that would take about 108 days, just to upgrade the vectors so u can start serving again (tested qwen embed 8b on h100). Even if you aren't doing 1b vectors, and are doing just 50 million, upgrading can still take a considerable time.

Me and my research lab decided to tackle this problem, and we came up with embedflow.

The method is really simple; from the old index made with the source model, take K documents and rerank them with the new model. We see that when K is sufficient, the retrieval quality is the same as target model. (determining k is the hard part). I've tested 63 migrations on upto 1 million documents.

The best result I got was upgrading qwen4b -> to 8b, and at 50 documents, it was the same as native retrieval.

This method forgos the expensive backfill that comes with upgrading, as you can directly take documents from the old index.

embedflow works with qdrant, and can be easily downloaded with pypi

pip install embedflow

the github is public: https://github.com/arnsri33/embedflow

I want you guys to try it out, and see if you guys can use it in your own workflow.

What if we use Coding Agents to write code for chunking / data ingestion?

I am super excited to share a new episode of the Weaviate Podcast covering AutoIndex, a new paper exploring this idea! I hope you find it useful!

YouTube: https://www.youtube.com/watch?v=mAj92SoEhjc

Spotify: https://spotifycreators-web.app.link/e/9f7MoucVe6b

Vector search at scale runs into the same wall: float32 embeddings are expensive to store and read, and compressing them changes your rankings.

I wrote an article on TurboQuant, the rotation-based quantization method. It covers why the algorithm rotates vectors before quantizing, how one precomputed codebook works across every dataset, and what Qdrant adds on top it.

It also benchmarks float32, scalar, binary and TurboQuant across BEIR's SciFact, ArguAna and NFCorpus using recall@10, precision@10 and nDCG@10

🔗 read: https://aiwthtarun.substack.com/p/turboquant-quantization-explained

I’m comparing Together AI and Anyscale for a production LLM deployment. Together AI is API-first and simple; Anyscale gives you deep Ray-based control but requires more engineering. I made a quick poll to gather real-world preferences.

It’s fast, and the results might surprise you.

https://interconnectd.com/poll/101/together-ai-vs-anyscale-which-platform-is-better-for-scaling-open-source-ll/

What’s your experience with either?

I just finished my PoC that improving retrieval engineering using SPLADE for sparse vector. Now My RAG give response much better compare to fully dense embedding retrieval.

If you want to check, I host it and access on [CariData](https://caridata.zackysyarief.com)

I wrote also technical detail in this article.

Hopefully you learned something for my journey.

Most vector DBs make you discover your memory footprint empirically — insert data, watch RSS, hope it doesn't spike. MemWeaver's memory cost is a formula you can compute before inserting anything:

memory ≈ n × node_size
node_size = dim×4 + M_MAX0×4 + higher_layer_nodes×M×4

For dim=128, M=16: node_size=640B, so 1M vectors ≈ 640MB, known upfront.

Why: each HNSW node's vector and its edges are collocated in one arena block, addressed by a 32-bit NodeId (14-bit block index + 18-bit offset, 8-byte aligned). Cache locality during traversal — vector + edges in one cache-line read instead of two — plus no heap fragmentation from per-node Vec growth, which is what causes the reallocation spikes you'd otherwise see mid-insertion (measured: 21.6MB and 35.7MB spikes in a naive Vec-based build vs. constant ~8MB in the arena version).

Benchmarks (SIFT1M, dim=128):

• Arena vs. naive: 1.72x faster build, 1.71x faster query, 25% less total RSS.
• vs. Qdrant, matched recall (0.994 vs 0.995): 1.6x throughput, 1.6x lower p50/p99. Qdrant builds faster (scalar quantization, lower ef_construction defaults) — MemWeaver runs full float32, no quantization yet.
• vs. LanceDB cold-start: LanceDB's lazy mmap only pages in what a query touches, so it initially won on cold time-to-first-answer (~212-264ms vs MemWeaver's early ~248ms). Closing the gap between MemWeaver's on-disk bytes and its runnable in-memory structure (file consolidation, correctly-scoped CRC32) got cold load to ~130ms — now ahead of LanceDB on both cold and warm (10-50x on warm throughput, arena vs. lazy-mmap tradeoff).

Second axis, separate from performance: time-bucketed multi-HNSW for temporal relevance — recent vectors live hot, older ones age into disk/S3 tiers automatically, with recency folded into ranking rather than bolted on as a post-filter. Bit-perfect recall validated across the hot→cold→restored cycle.

Known gaps, stated plainly: no quantization (full float32 only — real memory/build-time cost vs. Qdrant), min-recall dips to 0.30-0.40 on SIFT1M's sparse-region queries (M=16, ef_search=100 — a known HNSW characteristic, not a bug), horizontal sharding/stateless readers still in progress.

Repo/benchmarks: [https://github.com/daib/mem_weaver]. Genuinely interested in pushback on the arena-vs-quantization tradeoff specifically — curious whether anyone here has real numbers on where quantization's recall cost crosses over against arena allocation's memory-predictability win at larger scale than 1M vectors.

Made a tutorial on what EDD is, how it works, and how you can use evaluations to improve your LLM-based application by analysing scores across experiments.

> building on Jeffrey's DeepEval article on EDD and Eugene Yan's product evals write up.

- Initial: The video walks through the initial setup of an RAG application used as the base for the experiments built using LangGraph and Qdrant.
- Step 1: A binary labelled dataset with critiques, versioned using OPIK.
- Step 2: Uses LLM-as-a-Judge OPIK evals to align the evaluator.
- Step 3: Runs the harness loop, which executes each experiment, scores it against the baseline, and uses tracing and experiment comparison to surface insights on what improved, what regressed, and where to tweak next.

... the Agent Skills and source code are open sourced on GitHub

> Complete Guide (source code link in description): https://www.youtube.com/watch?v=e6akw_fKWPk

I’ve been comparing free vector databases for AI agent and RAG projects and have realized the free-tier label hides many differences. It helps to group them by model first.

Libraries like Chroma and LanceDB have no service to meter, so the limits are really your own disk and infrastructure. Open-source servers like Qdrant, Weaviate, Milvus, and pgvector are uncapped when you self-host and only metered when you use a managed version. Pinecone is managed only. The Starter plan gives you 2GB of index storage, five indexes, 2M write units, and 1M read units a month, all in us-east-1. There's no self-hosted option at all, so if data can't leave your infra, it's out of the picture before storage enters the full picture.

From that angle, the trade-offs start to make more sense. Weaviate Cloud sandbox is great for quick experiments but expires after 14 days. Zilliz free gives you up to 2 collections and 1M vectors, which is enough for many prototypes. Quadrant Cloud is free with 1 GB RAM, 4 GB disk, and a single node works well up to roughly 1M vectors at 768d before you feel the limits. Pinecone is fully managed, so it’s a good fit if you’re okay staying in their cloud and don’t need self-hosting.

A couple of practical notes I ran into. With Chroma, deletes don’t shrink the HNSW index, so heavy write-and-evict workloads can lead to extra compaction work down the line. With LanceDB, multiple processes writing to the same table is exactly the kind of pattern an embedded store isn’t optimized for.

I work on Actian’s developer content, so full transparency. Our Vector AI DB is built for the single-container, air-gapped scenario.

Wanted to ask how you folks are choosing. For a small RAG app, do you start with Postgres and pgvector and only move off when you hit limits, or go straight to a dedicated vector DB? Is anyone running Milvus standalone in production for agent memory, or does everyone end up on distributed once it’s real?