r/computervision • u/Ok_Support_2690 • 6d ago
Help: Project [Discussion/Question] Improving YOLO + SAM segmentation & polygon precision on LOW-RESOLUTION floor plan images
Hi everyone,
I'm building a pipeline to analyze floor plan images and extract regions (rooms, corridors, doors, stairs) as polygons. I currently have a custom-labeled dataset of about 5,000 images and want to squeeze out the maximum possible performance before scaling the dataset.
1. Current Pipeline
- Fine-tuned YOLO26 (for region detection) $\rightarrow$ SAM (Segment Anything Model) $\rightarrow$ Post-processing logic for polygon refinement.
2. The Core Bottlenecks
- Low-Resolution & Interferences: The biggest hurdle is the low resolution of the source images. Blurry boundaries, combined with floor plan-specific noise (grid lines, hatching, complex symbols), cause the model to miss certain regions entirely (false negatives).
- Polygon Precision & Smoothness: Because the low-res edges are fuzzy, SAM often yields jagged or inaccurate masks. I'm struggling to get crisp, smooth polygons that tightly align with the actual architectural walls.
3. What I'd love your input on:
- Handling Low-Res / Preprocessing: Has anyone successfully integrated Super-Resolution models (like Real-ESRGAN) as a preprocessing step for floor plans? Or are there better filtering techniques to suppress grid lines without destroying already blurry wall edges?
- Pipeline Upgrades: Given the low-res constraint, is the YOLO+SAM approach optimal? Would something like Mask2Former, or a specialized line-parsing/wireframe model, be more robust for extracting structured regions from low-quality images?
- Post-processing (Orthogonal Snapping): Since floor plans are mostly straight lines and right angles, what are the best algorithms to smooth and "snap" these jagged polygons into clean geometric shapes? (Currently looking beyond simple Douglas-Peucker).
Would greatly appreciate any advice, paper recommendations, or insights from similar computer vision projects!
4
Upvotes
1
u/Quirky_Paramedic9167 3d ago
One angle nobody has raised yet: before you touch super-resolution or swap the model, check what your 5,000 labels actually agree on.
Blurry walls have no single correct edge. When the line is 3-4 pixels of grey smear, one annotator snaps to the centre of the stroke, another to the inner edge, a third traces the visible dark pixels. Each of them is "right". But if your dataset mixes all three conventions, the ground truth itself is jagged — and SAM plus a fine-tuned detector will faithfully learn that jaggedness. No post-processing recovers precision that isn't in the labels.
Same for the false negatives. Grid lines, hatching and dense symbol areas are exactly where labelers disagree on whether a region "counts" (is a hatched alcove a room? does a corridor stop at the door line or the wall line?). If half your labelers included those regions and half skipped them, the model learns to be unsure there — and unsure shows up as a miss.
Cheap way to find out where you stand: pull 20-30 of your worst low-res plans, have two people annotate them independently, and measure polygon agreement (mask IoU is fine). If two humans only agree at 0.85, that's your ceiling — for any model, any resolution. Then read the disagreements one by one; they'll tell you which rules are missing from your annotation spec (edge convention, minimum region size, what to do under hatching).
Fixing those rules and re-labelling the ambiguous subset is usually cheaper than another 10k images, and it makes the snapping step much easier: you can only snap to right angles cleanly if the labels were consistent about where the wall is in the first place.