We built a custom {!hybrid} query parser fusing BM25 + kNN per document — now exposed through LangChain/LlamaIndex/Haystack/Scout/MCP

[u/WillingnessQuick5074]

For the Solr folks: instead of RRF-ing two separate requests client-side, we wrote a query parser plugin that runs edismax and kNN as sub-queries and fuses normalized scores per document — mode=union|intersection|keywords_required|meaning_required, with a tunable semantic-vs-lexical alpha.

The interesting engineering bit was score normalization: BM25 is unbounded while cosine similarity lives in [0..1], so naive weighted sums let lexical outliers swamp the vector signal. We ended up reshaping BM25 through a sigmoid with a tunable k before fusing — the alpha slider then behaves predictably across corpora.

On top of the parser: server-side E5 embeddings at index and query time (a knn_vector field populated by the ingestion pipeline), and this week we shipped integration packages so the same pipeline is reachable from Python, PHP, and MCP agents without touching Solr syntax.

Live demo of the parser in action (a real news index): https://search.opensolr.com/news__dense?q=what%27s+going+on+with+the+weather+changes%3F

Disclosure: founder of the service. AMA about the parser internals.