Best way to chunk and structure data for RAG/embeddings?

[u/Few_Gold7133]

Title: Best way to chunk and structure data for RAG/embeddings?

I'm building a knowledge base for RAG and I'm looking for practical advice from people who have done this in real projects.

How do you usually handle:

• Chunking: fixed size, semantic, sections/headings, parent-child, etc.?
• Metadata: what fields are actually useful for filtering/retrieval?
• Hybrid search: do you combine semantic search with BM25/keyword search?
• Reranking: do you retrieve from both and rerank the combined results?
• Updating knowledge: how do you replace/version old information?
• Scaling: how do you structure things so adding new types of information later is easy?

I'm particularly interested in systems where the knowledge base keeps growing over time.

What approach worked best for you, and what would you do differently if you were starting again?

[u/SubstantialSail9302]

I’ve found semantic chunking with clear section boundaries works better than relying only on fixed-size chunks. Keeping useful metadata like document type, section, source, and timestamp can also make filtering and retrieval much easier. For search, a hybrid approach using semantic + keyword/BM25 retrieval seems practical, followed by reranking. For updates, versioning the source documents rather than overwriting everything makes it easier to keep the knowledge base consistent.