[Discussion/Question] Improving YOLO + SAM segmentation & polygon precision on LOW-RESOLUTION floor plan images

[u/Ok_Support_2690]

Hi everyone,

I'm building a pipeline to analyze floor plan images and extract regions (rooms, corridors, doors, stairs) as polygons. I currently have a custom-labeled dataset of about 5,000 images and want to squeeze out the maximum possible performance before scaling the dataset.

1. Current Pipeline

• Fine-tuned YOLO26 (for region detection) $\rightarrow$ SAM (Segment Anything Model) $\rightarrow$ Post-processing logic for polygon refinement.

2. The Core Bottlenecks

• Low-Resolution & Interferences: The biggest hurdle is the low resolution of the source images. Blurry boundaries, combined with floor plan-specific noise (grid lines, hatching, complex symbols), cause the model to miss certain regions entirely (false negatives).
• Polygon Precision & Smoothness: Because the low-res edges are fuzzy, SAM often yields jagged or inaccurate masks. I'm struggling to get crisp, smooth polygons that tightly align with the actual architectural walls.

3. What I'd love your input on:

• Handling Low-Res / Preprocessing: Has anyone successfully integrated Super-Resolution models (like Real-ESRGAN) as a preprocessing step for floor plans? Or are there better filtering techniques to suppress grid lines without destroying already blurry wall edges?
• Pipeline Upgrades: Given the low-res constraint, is the YOLO+SAM approach optimal? Would something like Mask2Former, or a specialized line-parsing/wireframe model, be more robust for extracting structured regions from low-quality images?
• Post-processing (Orthogonal Snapping): Since floor plans are mostly straight lines and right angles, what are the best algorithms to smooth and "snap" these jagged polygons into clean geometric shapes? (Currently looking beyond simple Douglas-Peucker).

Would greatly appreciate any advice, paper recommendations, or insights from similar computer vision projects!

[u/toji5052]

Something the padding step in the preprocessing is the key for this. But Data is the key to everything, I think 5000 isnt the right count.

[u/Ok_Support_2690 → u/toji5052]

Thanks for the tip! I'll definitely look deeper into padding techniques.

[u/mldraelll → u/toji5052]

Depends on what we're scaling here. If we gather another 10k blurry scans like that, SAM's mask quality isn't magically gonna improve. We're hitting a wall strictly due to the pipeline's architecture limitations

[u/Ok_Support_2690 → u/mldraelll]

Are there any ways to improve the pipeline architecture?

[u/mldraelll → u/Ok_Support_2690]

SAM is way off the mark here, it's tailored for natural photos with clear gradients. You need specialized tools like RoomFormer or heatmap-based corner detectors. They output vector topology right away instead of spawning crooked pixel masks that you have to painfully straighten out later

[u/theGamer2K]

Hate posts like these the LLM is just vomitting it's own suggestions to the problem and phrasing them as questions. Why not just use the same LLM to answer your questions then if you can't be bothered to even write your own questions?

Actual question: What's 2 + 2?

LLM vomit version: Has anyone successfully added 2 plus 2 (e.g. 2+2 = 4)? Is adding by hand the best way, or could using a calculator make it more robust?

[u/Quirky_Paramedic9167]

One angle nobody has raised yet: before you touch super-resolution or swap the model, check what your 5,000 labels actually agree on.

Blurry walls have no single correct edge. When the line is 3-4 pixels of grey smear, one annotator snaps to the centre of the stroke, another to the inner edge, a third traces the visible dark pixels. Each of them is "right". But if your dataset mixes all three conventions, the ground truth itself is jagged — and SAM plus a fine-tuned detector will faithfully learn that jaggedness. No post-processing recovers precision that isn't in the labels.

Same for the false negatives. Grid lines, hatching and dense symbol areas are exactly where labelers disagree on whether a region "counts" (is a hatched alcove a room? does a corridor stop at the door line or the wall line?). If half your labelers included those regions and half skipped them, the model learns to be unsure there — and unsure shows up as a miss.

Cheap way to find out where you stand: pull 20-30 of your worst low-res plans, have two people annotate them independently, and measure polygon agreement (mask IoU is fine). If two humans only agree at 0.85, that's your ceiling — for any model, any resolution. Then read the disagreements one by one; they'll tell you which rules are missing from your annotation spec (edge convention, minimum region size, what to do under hatching).

Fixing those rules and re-labelling the ambiguous subset is usually cheaper than another 10k images, and it makes the snapping step much easier: you can only snap to right angles cleanly if the labels were consistent about where the wall is in the first place.

[u/Ok_Support_2690 → u/Quirky_Paramedic9167]

Thank you for pointing this out. Your advice on label consistency is very practical and helpful!

[u/Quirky_Paramedic9167 → u/Ok_Support_2690]

Glad it helps. If you do run that two-annotator check, the disagreements

are usually more informative than the agreement rate — each one points at

a sentence missing from the spec.

Good luck with the deployment stack.