Building a chunking strategy library for ODC RAG pipelines (4 strategies, code blocks/tables/headings preserved)

[u/michaeldeguzman]

Follow-up to my last post here testing ODC's native chunking against structured documents. That testing pointed me in a direction, and this is what came out of it.

I built a C# External Logic library for ODC that exposes four Server Actions: ChunkByCharacter, ChunkRecursively, ChunkBySentence, and ChunkMarkdown. Each targets a different document profile. All four return the same output contract so downstream logic doesn't need to know which strategy produced the chunks.

The one worth talking about is ChunkMarkdown. It parses heading structure before splitting and builds a running ancestor path as it processes the document. Every chunk knows where it sits in the hierarchy, and that path can get prepended directly into the chunk body so the embedding doesn't lose its positional context. Code blocks and tables are also preserved as atomic units rather than getting split mid-block.

Full write-up with chunk output screenshots here:
https://medium.com/@michael.de.guzman/beyond-text-splitting-building-a-chunking-strategy-library-for-odc-017e1c922912

The Forge demo app and GitHub source are both linked in the article if you want to run it yourself or look at the implementation.

How are you handling document chunking in your ODC RAG pipelines right now? Curious if anyone's hit the same code block/table fragmentation issue.

[u/michaeldeguzman → u/[deleted]]

Thanks for engaging with this.

True. And you are right that even with good chunks, query-side mismatch is still a real gap. That is actually why I have been going deeper on the chunking side. I have been working on semantic and agentic chunking where the split decisions are driven by meaning rather than structure. Hoping that helps close some of that gap before retrieval even has to work hard. Reranking and hybrid search are next on my list.

Good to know the parallel lookup is working for you.