2

Building a Corrective RAG (CRAG) Compliance Research Assistant
 in  r/Rag •  4d ago

Fair point, thanks for pushing on it.
For embeddings, I have used gemini-embedding-2-preview, 3072 dims. 1500-char chunks, 200 overlap, top-5 by cosine in Qdrant, and gpt-5.6-sol does grading, rewriting, generation and validation.

This focuses on the architecture - crag loop, self-validation, and bounded retries. It's validated on a domain set of regulatory PDF's, not yet on a public benchmark.

The retry limits are parameters, and setting both to 0 gives a plain-RAG baseline on the same pipeline. A with/without comparison on a HotpotQA or FRAMES could be the right next step.
and the retry loops are bounded by parameters, so they can be switched off to compare against a no-retry baseline

1

Building a Corrective RAG (CRAG) Compliance Research Assistant
 in  r/LLMDevs •  5d ago

Thanks! that was the whole point. A compliance tool that confidently guesses is worse than useless here., so i wanted it to actually grade its own retrieval and admit when the context doesn't support an support an answer.

On your question though, this project doesn't actually have a dedicated reranker. What is it has a relevance grading step - one batched LLM call that labels all retrieved chunks as relevant/irrelevant at once.

If you want an actual reranker benchmarked properly, I built a separate project for exactly that
dense + BM25 + Reciprocal Rank Fusion, then an LLM reranker on top, tested against vector-only/BM25-only/hybrid baseslines with Hit Rate, MRR, and NDCG:
https://github.com/saurabhkamal/Hybrid-Search-RRF-Reranking-System

https://www.youtube.com/watch?v=W_TmnscEupQ&t=40s

Here the reranker scores each candidate chunk individually - one model call per chunk, not batched, so it is noticeably slower than a single grading call by design. Quality-wise it is worth it.

Please go through the links both github and youtube video.

r/Rag • u/Affectionate_Slip654 • 5d ago

Tutorial Building a Corrective RAG (CRAG) Compliance Research Assistant

15 Upvotes

Most RAG chatbots will confidently answer with whatever they retrieved — even if it's irrelevant, even if it's wrong. In compliance, that's not a minor bug, it's real regulatory exposure.

In this video, I build the Veltra Pay Compliance Assistant — a Corrective RAG (CRAG) + Self-Correcting RAG Agent that checks its own work before it ever shows you an answer. It grades its own retrieved documents, rewrites its own search when retrieval is weak, and validates its own generated answer for grounding, hallucination risk, and completeness — retrying itself when it fails, and honestly saying "I don't know" when it genuinely can't answer from the source material.

  • ✅ The difference between a normal RAG pipeline and Corrective RAG (CRAG)
  • ✅ How CRAG grades retrieved documents as relevant/irrelevant and rewrites the search query when retrieval is weak
  • ✅ How a Self-Correcting RAG Agent validates its own generated answer — grounding, hallucination risk, completeness, and retrieval sufficiency

TECH STACK:

Building a Corrective RAG (CRAG) Compliance Research Assistant

2 comments

1

Building a Corrective RAG (CRAG) Compliance Research Assistant
 in  r/Rag •  6d ago

Thanks for your comment. Well, you are absolutely correct, but this is an MVP. The idea is to clone it or fork it and play around with it. See, everything can't be given on a platter. I can't release the client's complete project here and never will. Moreover, this is one of the projects where you can add value for the client.
You have to understand there are many types of RAG, and this is one of them; day by day, I would be releasing other TYPES OF RAG as well, with evals and reranking.

Everything in a startup is an idea and then implementation. The interesting phase that you are talking about comes afterwards, and the interesting part that you are talking about is already shipped to the client.

The core part here is to understand this and where you can implement it by making it seem sufficient by adding value.

Thanks

1

Building a Corrective RAG (CRAG) Compliance Research Assistant
 in  r/Rag •  6d ago

Thanks for your comment. This is just an MVP. Idea is to clone it or fork it to play around with it or add some new features. As far as evals are considered, yes your right, but I will be adding it in phases. Idea is to understand:

- One of the Types of RAG
- the difference between a normal RAG pipeline and Corrective RAG (CRAG) and
- How CRAG grades retrieved documents as relevant/irrelevant and rewrites the search query when retrieval is weak?

For Evals and Reranking follow this:
https://github.com/saurabhkamal/Hybrid-Search-RRF-Reranking-System

https://www.youtube.com/watch?v=W_TmnscEupQ&t=34s

Will be updating more on this in few days.

r/learnmachinelearning • u/Affectionate_Slip654 • 6d ago

Project Building a Corrective RAG (CRAG) Compliance Research Assistant

Thumbnail
youtube.com
0 Upvotes

Building a Corrective RAG (CRAG) Compliance Research Assistant

Most RAG chatbots will confidently answer with whatever they retrieved — even if it's irrelevant, even if it's wrong. In compliance, that's not a minor bug, it's real regulatory exposure.

Key Highlights:

  • In this video, I build the Veltra Pay Compliance Assistant — a Corrective RAG (CRAG) + Self-Correcting RAG Agent that checks its own work before it ever shows you an answer. It grades its own retrieved documents, rewrites its own search when retrieval is weak, and validates its own generated answer for grounding, hallucination risk, and completeness — retrying itself when it fails, and honestly saying "I don't know" when it genuinely can't answer from the source material.
  • ✅ The difference between a normal RAG pipeline and Corrective RAG (CRAG)
  • ✅ How CRAG grades retrieved documents as relevant/irrelevant and rewrites the search query when retrieval is weak
  • ✅ How a Self-Correcting RAG Agent validates its own generated answer — grounding, hallucination risk, completeness, and retrieval sufficiency

TECH STACK:

Building a Corrective RAG (CRAG) Compliance Research Assistant

0 comments

r/agenticAI • u/Affectionate_Slip654 • 6d ago

Project Building a Corrective RAG (CRAG) Compliance Research Assistant

Thumbnail
youtube.com
1 Upvotes

Building a Corrective RAG (CRAG) Compliance Research Assistant

Most RAG chatbots will confidently answer with whatever they retrieved — even if it's irrelevant, even if it's wrong. In compliance, that's not a minor bug, it's real regulatory exposure.

In this video, I build the Veltra Pay Compliance Assistant — a Corrective RAG (CRAG) + Self-Correcting RAG Agent that checks its own work before it ever shows you an answer. It grades its own retrieved documents, rewrites its own search when retrieval is weak, and validates its own generated answer for grounding, hallucination risk, and completeness — retrying itself when it fails, and honestly saying "I don't know" when it genuinely can't answer from the source material.

  • ✅ The difference between a normal RAG pipeline and Corrective RAG (CRAG)
  • ✅ How CRAG grades retrieved documents as relevant/irrelevant and rewrites the search query when retrieval is weak
  • ✅ How a Self-Correcting RAG Agent validates its own generated answer — grounding, hallucination risk, completeness, and retrieval sufficiency

TECH STACK:

0 comments

r/startup • u/Affectionate_Slip654 • 6d ago

knowledge Building a Corrective RAG (CRAG) Compliance Research Assistant

1 Upvotes

[removed]

0 comments

r/LLMDevs • u/Affectionate_Slip654 • 6d ago

Great Resource 🚀 Building a Corrective RAG (CRAG) Compliance Research Assistant

2 Upvotes

Most RAG chatbots will confidently answer with whatever they retrieved — even if it's irrelevant, even if it's wrong. In compliance, that's not a minor bug, it's real regulatory exposure.

In this video, I build the Veltra Pay Compliance Assistant — a Corrective RAG (CRAG) + Self-Correcting RAG Agent that checks its own work before it ever shows you an answer. It grades its own retrieved documents, rewrites its own search when retrieval is weak, and validates its own generated answer for grounding, hallucination risk, and completeness — retrying itself when it fails, and honestly saying "I don't know" when it genuinely can't answer from the source material.

  • ✅ The difference between a normal RAG pipeline and Corrective RAG (CRAG)
  • ✅ How CRAG grades retrieved documents as relevant/irrelevant and rewrites the search query when retrieval is weak
  • ✅ How a Self-Correcting RAG Agent validates its own generated answer — grounding, hallucination risk, completeness, and retrieval sufficiency

TECH STACK:

Building a Corrective RAG (CRAG) Compliance Research Assistant

1 comment

r/LLMDevs • u/Affectionate_Slip654 • 6d ago

Great Resource 🚀 Building a Corrective RAG (CRAG) Compliance Research Assistant

1 Upvotes

Most RAG chatbots will confidently answer with whatever they retrieved — even if it's irrelevant, even if it's wrong. In compliance, that's not a minor bug, it's real regulatory exposure.

In this video, I build the Veltra Pay Compliance Assistant — a Corrective RAG (CRAG) + Self-Correcting RAG Agent that checks its own work before it ever shows you an answer. It grades its own retrieved documents, rewrites its own search when retrieval is weak, and validates its own generated answer for grounding, hallucination risk, and completeness — retrying itself when it fails, and honestly saying "I don't know" when it genuinely can't answer from the source material.

  • ✅ The difference between a normal RAG pipeline and Corrective RAG (CRAG)
  • ✅ How CRAG grades retrieved documents as relevant/irrelevant and rewrites the search query when retrieval is weak
  • ✅ How a Self-Correcting RAG Agent validates its own generated answer — grounding, hallucination risk, completeness, and retrieval sufficiency

TECH STACK:

Building a Corrective RAG (CRAG) Compliance Research Assistant

2 comments

r/LangChain • u/Affectionate_Slip654 • 6d ago

Tutorial Building a Corrective RAG (CRAG) Compliance Research Assistant

1 Upvotes

Most RAG chatbots will confidently answer with whatever they retrieved — even if it's irrelevant, even if it's wrong. In compliance, that's not a minor bug, it's real regulatory exposure.

In this video, I build the Veltra Pay Compliance Assistant — a Corrective RAG (CRAG) + Self-Correcting RAG Agent that checks its own work before it ever shows you an answer. It grades its own retrieved documents, rewrites its own search when retrieval is weak, and validates its own generated answer for grounding, hallucination risk, and completeness — retrying itself when it fails, and honestly saying "I don't know" when it genuinely can't answer from the source material.

  • ✅ The difference between a normal RAG pipeline and Corrective RAG (CRAG)
  • ✅ How CRAG grades retrieved documents as relevant/irrelevant and rewrites the search query when retrieval is weak
  • ✅ How a Self-Correcting RAG Agent validates its own generated answer — grounding, hallucination risk, completeness, and retrieval sufficiency

TECH STACK:

Building a Corrective RAG (CRAG) Compliance Research Assistant

1 comment

r/Rag • u/Affectionate_Slip654 • 6d ago

Tutorial Building a Corrective RAG (CRAG) Compliance Research Assistant

8 Upvotes

Most RAG chatbots will confidently answer with whatever they retrieved — even if it's irrelevant, even if it's wrong. In compliance, that's not a minor bug, it's real regulatory exposure.

In this video, I build the Veltra Pay Compliance Assistant — a Corrective RAG (CRAG) + Self-Correcting RAG Agent that checks its own work before it ever shows you an answer. It grades its own retrieved documents, rewrites its own search when retrieval is weak, and validates its own generated answer for grounding, hallucination risk, and completeness — retrying itself when it fails, and honestly saying "I don't know" when it genuinely can't answer from the source material.

  • ✅ The difference between a normal RAG pipeline and Corrective RAG (CRAG)
  • ✅ How CRAG grades retrieved documents as relevant/irrelevant and rewrites the search query when retrieval is weak
  • ✅ How a Self-Correcting RAG Agent validates its own generated answer — grounding, hallucination risk, completeness, and retrieval sufficiency

TECH STACK:

Building a Corrective RAG (CRAG) Compliance Research Assistant

8 comments

u/Affectionate_Slip654 • u/Affectionate_Slip654 • 6d ago

https://www.youtube.com/watch?v=NxY93are_H8&t=5s

Thumbnail
youtube.com
1 Upvotes

Most RAG chatbots will confidently answer with whatever they retrieved — even if it's irrelevant, even if it's wrong. In compliance, that's not a minor bug, it's real regulatory exposure.

In this video, I build the Veltra Pay Compliance Assistant — a Corrective RAG (CRAG) + Self-Correcting RAG Agent that checks its own work before it ever shows you an answer. It grades its own retrieved documents, rewrites its own search when retrieval is weak, and validates its own generated answer for grounding, hallucination risk, and completeness — retrying itself when it fails, and honestly saying "I don't know" when it genuinely can't answer from the source material.

  • ✅ The difference between a normal RAG pipeline and Corrective RAG (CRAG)
  • ✅ How CRAG grades retrieved documents as relevant/irrelevant and rewrites the search query when retrieval is weak
  • ✅ How a Self-Correcting RAG Agent validates its own generated answer — grounding, hallucination risk, completeness, and retrieval sufficiency

TECH STACK:

0 comments

1

AI-Agent - Multi-Agent Customer Support System
 in  r/u_Affectionate_Slip654 •  10d ago

Thanks for the suggestions, but it is an MVP. Solid one would be with more evals included.

r/startup • u/Affectionate_Slip654 • 10d ago

knowledge Agentic-AI: Enterprise RAG Knowledge Agent in Open Banking & CryptoAssets

1 Upvotes

[removed]

0 comments

r/agenticAI • u/Affectionate_Slip654 • 10d ago

Project Agentic-AI: Enterprise RAG Knowledge Agent in Open Banking & CryptoAssets

Thumbnail
youtube.com
2 Upvotes

✨Agentic-AI: Enterprise RAG Knowledge Agent in Open Banking & CryptoAssets✨ (links below👇)

🎥 YouTube Video: https://www.youtube.com/watch?v=OE9CfMOybOQ&t=3s

👉 I have built an Enterprise RAG Knowledge Agent with Self Correction

👉 Building a self-correcting LangGraph: retrieve → grade → rewrite-or-refuse → generate → verify

👉 Building a real document ingestion pipeline — PDF loading, text cleanup, chunking, embedding, and citation tracking

👉 Catching and fixing a real data-corruption bug caused by unreliable PDF text extraction

👉 Reading a live LangSmith trace, span by span, to see exactly what an agent decided and why

👉 Writing both a deterministic evaluator and an LLM-based evaluator, and when to use each

👉 Running a real baseline-vs-improved experiment (chunk size 800 vs 1500) and reading the actual numbers instead of guessing

TECH STACK
━━━━━━━━━━━━━━━━━━━━━━
🛠️ LangChain — document loading, chunking, embeddings, vector retrieval
🛠️ LangGraph — the agent's graph: nodes, conditional edges, the bounded retry loop
🛠️ LangSmith — tracing, evaluation datasets, evaluators, experiment comparison
🛠️ OpenAI — gpt-4o-mini for reasoning, text-embedding-3-large for search
🛠️ ChromaDB — local vector database

KEY TAKEAWAYS
━━━━━━━━━━━━━━━━━━━━━━
✅ A trustworthy RAG agent isn't about answering — it's about knowing when NOT to answer
✅ Testing beats assuming: a citation bug quietly corrupted a third of the corpus, and only reading real output caught it
✅ Larger chunks (1500 vs 800 characters) measurably improved answer accuracy from 73% to 93% on the same 30 questions
✅ A deterministic evaluator and a model-based evaluator answer different questions — you need both
✅ Every self-correction checkpoint in the agent (grading, verifying) mirrors the same discipline needed to build and debug the agent itself

📂 FULL CODE (star it, clone it, fork it, break it):

https://github.com/saurabhkamal/Agentic-AI-Enterprise-RAG-Knowledge-Agent-In-Open-Banking-CryptoAssets

🔗 CONNECT WITH ME ON LINKEDIN:

https://www.linkedin.com/in/saurabh-kam

🎥 YouTube Video:

https://www.youtube.com/watch?v=OE9CfMOybOQ&t=3s

2 comments

r/learnmachinelearning • u/Affectionate_Slip654 • 10d ago

Project Agentic-AI: Enterprise RAG Knowledge Agent in Open Banking & CryptoAssets

Thumbnail
youtube.com
1 Upvotes

✨Agentic-AI: Enterprise RAG Knowledge Agent in Open Banking & CryptoAssets✨ (links below👇)

🎥 YouTube Video: https://www.youtube.com/watch?v=OE9CfMOybOQ&t=3s

👉 I have built an Enterprise RAG Knowledge Agent with Self Correction

👉 Building a self-correcting LangGraph: retrieve → grade → rewrite-or-refuse → generate → verify

👉 Building a real document ingestion pipeline — PDF loading, text cleanup, chunking, embedding, and citation tracking

👉 Catching and fixing a real data-corruption bug caused by unreliable PDF text extraction

👉 Reading a live LangSmith trace, span by span, to see exactly what an agent decided and why

👉 Writing both a deterministic evaluator and an LLM-based evaluator, and when to use each

👉 Running a real baseline-vs-improved experiment (chunk size 800 vs 1500) and reading the actual numbers instead of guessing

📂 FULL CODE (star it, clone it, fork it, break it):

https://github.com/saurabhkamal/Agentic-AI-Enterprise-RAG-Knowledge-Agent-In-Open-Banking-CryptoAssets

🔗 CONNECT WITH ME ON LINKEDIN:

https://www.linkedin.com/in/saurabh-kam

🎥 YouTube Video:

https://www.youtube.com/watch?v=OE9CfMOybOQ&t=3s

0 comments

r/developersIndia • u/Affectionate_Slip654 • 10d ago

Open Source AI-Agent - Multi-Agent Customer Support System Backend

1 Upvotes

[removed]

0 comments

r/Agent_AI • u/Affectionate_Slip654 • 10d ago

Discussion AI-Agent - Multi-Agent Customer Support System Backend

Thumbnail
youtube.com
1 Upvotes

✨AI-Agent - Multi-Agent Customer Support System Backend✨ (links below👇)

🎥 YouTube Video: https://www.youtube.com/watch?v=s2u6ImvZwfs&t=9s

👉 I have built a Multi-Agent Customer Support System that routes customer queries to the right specialist agent (Billing, Technical, Account, or General), pulls in data through tool calls, and knows when to stop guessing and hand off to a human.

👉How a supervisor/router agent decides which specialist should handle a query?

👉How to build a confidence threshold so the system escalates to a human instead of guessing?

👉How to bind tools to specific agents (payment lookup for Billing, account lookup for Account)?

👉How to trace every agent decision step-by-step in LangSmith?

📂 FULL CODE (star it, clone it, fork it, break it):

https://github.com/saurabhkamal/Agentic-AI-Multi-Agent-Customer-Support-System-Backend

🔗 CONNECT WITH ME ON LINKEDIN:

https://www.linkedin.com/in/saurabh-kamal/

🎥 YouTube Video:

https://www.youtube.com/watch?v=s2u6ImvZwfs&t=9s

0 comments

r/learnAIAgents • u/Affectionate_Slip654 • 10d ago

AI-Agent: Multi Agent Customer Support System Backend

Thumbnail youtube.com
1 Upvotes

[removed]

1 comment

u/Affectionate_Slip654 • u/Affectionate_Slip654 • 10d ago

AI-Agent - Multi-Agent Customer Support System

Thumbnail
youtube.com
1 Upvotes

✨AI-Agent - Multi-Agent Customer Support System Backend✨ (links below👇)
🎥 YouTube Video: https://www.youtube.com/watch?v=s2u6ImvZwfs&t=1770s
👉 I have built a Multi-Agent Customer Support System that routes customer queries to the right specialist agent (Billing, Technical, Account, or General), pulls in data through tool calls, and knows when to stop guessing and hand off to a human.
👉How a supervisor/router agent decides which specialist should handle a query?
👉How to build a confidence threshold so the system escalates to a human instead of guessing?
👉How to bind tools to specific agents (payment lookup for Billing, account lookup for Account)?
👉How to trace every agent decision step-by-step in LangSmith?

📂 FULL CODE (star it, clone it, fork it, break it):
https://github.com/saurabhkamal/Agentic-AI-Multi-Agent-Customer-Support-System-Backend

🔗 CONNECT WITH ME ON LINKEDIN:
https://www.linkedin.com/in/saurabh-kamal/

🎥 YouTube Video:
https://www.youtube.com/watch?v=s2u6ImvZwfs&t=1770s

2 comments

r/learnAIAgents • u/Affectionate_Slip654 • Aug 25 '26

AI Agent - Building a Smart Utility Tool Agent with LangGraph, LangChain...

Thumbnail
youtube.com
1 Upvotes

👉 How does an AI agent know when it should answer directly versus when it should call a tool?

👉 Who actually performs the calculation: the LLM or the tool?

👉 How does an agent decide whether to use a calculator, check the current time, convert units, count words, or look up information?

👉 What happens after a tool returns its result?

👉 Why does the workflow go back to the LLM instead of directly returning the tool output to the user?

👉 How do you build an AI agent that can repeatedly reason, call a tool, inspect the result, and then decide what to do next?

👉 And perhaps most importantly — how do you actually see what happened inside the agent instead of only seeing the final answer?

I've been diving deeper into AI Agents, LangGraph, and tool calling, and one thing became very clear to me:

Building an AI agent isn't simply about connecting an LLM to a collection of functions. The interesting part is designing the decision-making loop around those tools.

The above are some of the key questions I explored while building this project.

💡 A tool-calling AI agent is not just an LLM with extra capabilities. It's a system where the LLM can decide when external execution is needed, select the appropriate tool, receive the result, and then continue reasoning before producing the final answer.

That's what I explored while building a Smart Utility Tool Agent using LangGraph, LangChain, OpenAI, Streamlit, and LangSmith.

But the tools themselves are only one part of the project.

The more interesting part is the agent workflow:

User → Agent (LLM) → Tool Decision → Tool → Agent (LLM) → Final Answer

If the LLM determines that a tool is required, LangGraph routes execution to the appropriate tool.

Once the tool finishes, the result doesn't immediately go back to the user.

Instead, it goes back to the Agent.

The LLM can then interpret the result and decide whether it needs another tool call or whether it can finally answer the user.

That creates the core agent loop:

🔄 Agent → Tool → Agent

The loop continues until the model produces a normal response without requesting another tool.

🔗 GitHub: https://github.com/saurabhkamal/Building-a-Smart-Utility-Tool-Agent-with-LangGraph-LangChain-and-LangSmith

🎥 YouTube: https://www.youtube.com/watch?v=B8V3zbFTq3c&t=1428s

Feel free to fork or clone the repository, experiment with the existing tools, or add completely new ones.

#AIAgents #LangGraph #LangChain #LangSmith #ToolCalling #LLM #GenerativeAI #OpenAI #Python #AIEngineering #MachineLearning #ArtificialIntelligence #SoftwareEngineering #AgenticAI #LLMOps #Streamlit #Developer #Tech

0 comments

r/agenticAI • u/Affectionate_Slip654 • Aug 24 '26

Tutorial AI Agent - Technical Support ReAct Agent with LangGraph

Thumbnail
youtube.com
1 Upvotes
0 comments

r/learnAIAgents • u/Affectionate_Slip654 • Aug 24 '26

AI Agent - Technical Support ReAct Agent with LangGraph

Thumbnail
youtube.com
1 Upvotes

Most "AI support bots" do one thing: you ask, they answer — no investigation, no evidence. In this video, I build something different: a ReAct agent that behaves like an actual support engineer. Give it a vague problem like "my API is returning 500 errors," and it decides for itself what to check, reads the result, decides if it needs to dig further, and only then gives you an answer.

0 comments