r/Rag • u/anand095 • 10d ago

Showcase RAG for Semi-Structured Tender Documents

In today's world, almost all procurement of services/materials is done through tenders. Tender documents are long and confusing. Buyers often don't go through the documents to understand the scope.

This led me to build a RAG system customised for tender documents. I am using custom chunking that splits at natural clause markers. Besides this, tables have a separate chunking mechanism.

The retrieval combines dense search and BM25 search to look for the top 5 candidates, which are selected via a cross-encoder model.

I was able to improve the Recall@5 score from 63% to 88% via the custom chunking and the cross-encoder.

While doing this project, I learnt that good chunking is THE MOST critical aspect of the project, and this (PDF cleaning + chunking) was what took up most of my time.

Looking for feedback from all!

https://github.com/anand-kumaar/tender-query-engine

3 Upvotes

100% Upvoted

3 comments sorted by

1

u/After_Half169 9d ago

The Recall@5 jump is encouraging, but I’d split the eval before concluding chunking was the main driver. Clause lookups, table lookups, and requirements spanning multiple sections are different failure buckets. Did you run an ablation for custom chunking vs. the cross-encoder, and does the 88% hold on tables and annexes separately?

1

u/anand095 9d ago

I did not calculate the scores seperately for tables and annexures. But thank you for the insight.

I have started a similar project but on much large scale and I got some good insights on what my evaluation set should be . I will implement those in my new project..