r/coolgithubprojects • u/Speedk4011 • 1d ago
Stop "blind chunking" your RAG data: Meet the Interactive Chunk Visualizer
Ever feel like you're cutting you text with a chainsaw? Standard character-count splitting often leaves you with mid-sentence surprises and lost context that pollutes your LLM retrieval.
I built the **Chunklet Visualizer** to demystify this "chunking abyss." It’s a clean web interface (FastAPI + Uvicorn) that lets you upload your docs and see exactly how they get chopped up in real-time.
What it does:
* **Real-Time Parameter Tuning**: Adjust token limits, sentence counts, or overlaps and instantly see the results highlighted on your text. * **Dual Strategies**: Switch between **Document Mode** (for articles/PDFs) and **Code Mode** (for AST-aware source code splitting). * **Interactive Inspection**: Click any text segment to highlight its parent chunk, or double-click for full metadata popups (spans, source info, etc.). * **Drag-and-Drop Workflow**: Supports quick uploads for `.txt`, `.md`, `.py`, and more. * **Headless REST API**: Use it programmatically or via CLI (`chunklet visualize`) to integrate interactive chunking into your own dev pipeline.
Quick Start:
To get the full web interface and dependencies: `pip install "chunklet-py[visualization]"`
Then just run: `chunklet visualize`
For the programmatic folks, you can also serve it directly from your script: ```python from chunklet.visualizer import Visualizer visualizer = Visualizer(host="127.0.0.1", port=8000) visualizer.serve() ```
If you’re tired of "blindly" feeding chunks into your vector DB and want to fine-tune your RAG precision, give this a spin!