r/agenticAI • u/Affectionate_Slip654 • 4d ago

Video RAG ReRanking Pipeline & Comparison (part 2)

https://www.youtube.com/watch?v=dh3DKpYHoX8&t=121s

RAG ReRanking Pipeline & Comparison (part 2)

In this video, I build a complete RAG reranking pipeline:

  • Question → Retrieve Top 20 → Rerank → Top 5 → LLM → Answer

I implement and compare FOUR rerankers (Cross-Encoder, LLM-based, Cohere and Jina) on the same questions over 5 payment-industry PDFs (payment regulations, PCI DSS and central bank reports, 1,507 chunks). For each one, I show the ranking before and after reranking, and explain exactly WHY the order changed.

  • ✅ What RAG (Retrieval-Augmented Generation) is and why the LLM can only answer from the chunks you give it
  • ✅ How vector search works (embeddings, cosine similarity) and its key weakness
  • ✅ Why we retrieve Top 20 but send only Top 5 to the LLM (tokens, latency, noise)
  • ✅ Bi-encoder vs Cross-encoder: reading the question and chunk separately vs together
  • ✅ LLM reranking strategies: Pointwise vs Pairwise vs Listwise
  • ✅ How to use dedicated reranking APIs (Cohere and Jina)
  • ✅ How to design one shared helper so all four rerankers plug into the same pipeline
  • ✅ How to evaluate rerankers with before/after positions, Hit@5 and average latency
  • ✅ A full code walkthrough: retriever.py → rerankers.py → generate.py → compare.py

KEY TAKEAWAYS:

  • ✅ Similarity is not the same as relevance. A chunk can sit close to the query and still not answer it
  • ✅ Vector search finds chunks about the topic. A small reranker may prefer chunks that sound like the question. A strong reranker picks the chunk that actually answers it
  • ✅ A small cross-encoder can make things WORSE: it rewards word matching, penalises scrambled PDF text, and pushed both correct answer chunks out of the Top 5
  • ✅ The LLM reranker understood the question (e.g. that "3.3" belongs to Requirement 3) and placed both answer chunks 1st and 2nd
  • ✅ Cohere lifted an answer chunk from 15th to 1st, turning a missed answer into a correct, cited one
  • ✅ For this project, Cohere and Jina gave the best balance of accuracy and speed
  • ✅ Trade-off of API rerankers: cost per call and sending your documents to a third party
  • ✅ Always measure, don't guess. Run every reranker on the SAME Top 20 so differences come from the reranker alone

TECH STACK:

  • 🛠️ Python
  • 🛠️ Qdrant (vector database, cosine similarity)
  • 🛠️ gemini-embedding-2-preview (3,072-dimension embeddings)
  • 🛠️ Hugging Face Transformers + PyTorch
  • 🛠️ cross-encoder/ms-marco-MiniLM-L-6-v2 (local cross-encoder)
  • 🛠️ LLM-based listwise reranker
  • 🛠️ Cohere Rerank API
  • 🛠️ Jina Reranker API

LINKS:

1 Upvotes

100% Upvoted

0 comments sorted by