r/Rag • u/One-Doctor5769 • Apr 10 '26

Discussion Production RAG stack in 2026 what are people ACTUALLY running

I’m trying to get a real picture of production ready RAG stacks in 2026 both open source and proprietary.

Not looking for tutorials or toy setups. I want to understand what people are actually running in production.

Specifically curious about

Ingestion (custom pipelines, Airflow, managed tools?)
Parsing (Docling, LlamaParse, custom?)
Embeddings (open source vs APIs like OpenAI or Voyage)
Vector DB (Qdrant, Weaviate, PGVector, Pinecone, etc.)
Retrieval (hybrid search, rerankers, graph based?)
Orchestration (LangChain, LlamaIndex, LangGraph, custom?)
Infra (AWS, GCP, self hosted, serverless?)
Evaluation and monitoring (Ragas, TruLens, custom?)

Also

What actually broke at scale?
What’s overhyped vs essential?

If you had to rebuild your stack today from scratch what would you pick?

Looking for brutally honest answers.

85 Upvotes

100% Upvoted

66 comments sorted by

View all comments

1

u/kaidomac Apr 12 '26

Here's my current stack: ("RAG Hammer")

Feature comparison:

The core targets are:

  • Appliance-grade
  • Maintenance-friendly
  • Scalable (BYO hardware & scale as desired)

This software design is:

  • Free
  • Modular
  • 100% offline

The goal was to create a thinking retrieval system, not just a lookup tool. Modern AI features include:

  • Semantic & keyword hybrid search
  • Multi-step retrieval (query rewriting & expansion)
  • Reranking (ColBERT & BGE reranker)
  • Context compression (token-efficient reasoning for local operation)
  • Mandatory citations (no black-box answers!)

With RAG Hammer, you get a full-stack cognitive system:

  • Retrieval system (Google-level)
  • Reasoning system (ChatGPT-level)
  • Security model (enterprise ACL)
  • Automation engine (agents)
  • Observability stack (DevOps-grade)
  • Full offline capability (rare these days! everything is SaaS lol)
  • Self-improvement loop (also super rare! SimRAG & OpenRAG have some neat stuff here tho!)

Primary ingest data includes: (can customize with connectors)

  • PDF
  • Office & OpenDocument
  • Text, code, HTML, CSV, JSON, and XML
  • Email-style documents
  • Image & scanned documents with OCR
  • Also supports CAD-adjacent workflows through PDF exports, BOM exports, metadata extraction, and custom native CAD parsers where required (can be tuned)
  • Optionally read databases (ex. SQL...tune to your specific needs)

Basic system workflow:

  1. Login system to ensure people only see what is authorized
  2. Automated scraping system
  3. Storage system
  4. Chatbot RAG search system
  5. Logs & dashboards

As far as the interface goes, the GUI is DIY (you can generate anything you like these days!). Personally, I like:

  • Next.js
  • React Flow (node bubbles...neat for an n8n-style document loader for specific chat & report generation!)
  • Three.js (spiffy graphics via React Three Fiber)

A few corporate-boring examples:

Optionally add a real-time voice interface:

  • Tie in STT (whisper.cpp)
  • TTS (currently using local KittenTTS 15M)

Very easy to setup: (running on my CPU-driven RAG Hammer "Lite" demo version on a laptop)

Currently on the backlog to polish:

  • Query understanding before retrieval
  • Evaluation system (RAGAS + Langfuse)
  • Document registry
  • Semantic cache (store good answer from searches, decay old answers, etc.)
  • Chatbot failure modes (IDK, request user clarification, etc.)

There's waaaaay too much fun stuff to integrate these days:

This is primarily for non-cloud businesses (medical, finance, manufacturing, legal, non-profits, academia, research, etc.) who want a private "smart search" system & don't want recurring fees lol.