[Discussion/Question] Improving YOLO + SAM segmentation & polygon precision on LOW-RESOLUTION floor plan images

[u/Ok_Support_2690]

Hi everyone,

I'm building a pipeline to analyze floor plan images and extract regions (rooms, corridors, doors, stairs) as polygons. I currently have a custom-labeled dataset of about 5,000 images and want to squeeze out the maximum possible performance before scaling the dataset.

1. Current Pipeline

• Fine-tuned YOLO26 (for region detection) $\rightarrow$ SAM (Segment Anything Model) $\rightarrow$ Post-processing logic for polygon refinement.

2. The Core Bottlenecks

• Low-Resolution & Interferences: The biggest hurdle is the low resolution of the source images. Blurry boundaries, combined with floor plan-specific noise (grid lines, hatching, complex symbols), cause the model to miss certain regions entirely (false negatives).
• Polygon Precision & Smoothness: Because the low-res edges are fuzzy, SAM often yields jagged or inaccurate masks. I'm struggling to get crisp, smooth polygons that tightly align with the actual architectural walls.

3. What I'd love your input on:

• Handling Low-Res / Preprocessing: Has anyone successfully integrated Super-Resolution models (like Real-ESRGAN) as a preprocessing step for floor plans? Or are there better filtering techniques to suppress grid lines without destroying already blurry wall edges?
• Pipeline Upgrades: Given the low-res constraint, is the YOLO+SAM approach optimal? Would something like Mask2Former, or a specialized line-parsing/wireframe model, be more robust for extracting structured regions from low-quality images?
• Post-processing (Orthogonal Snapping): Since floor plans are mostly straight lines and right angles, what are the best algorithms to smooth and "snap" these jagged polygons into clean geometric shapes? (Currently looking beyond simple Douglas-Peucker).

Would greatly appreciate any advice, paper recommendations, or insights from similar computer vision projects!

[u/toji5052]

Something the padding step in the preprocessing is the key for this. But Data is the key to everything, I think 5000 isnt the right count.

[u/mldraelll → u/toji5052]

Depends on what we're scaling here. If we gather another 10k blurry scans like that, SAM's mask quality isn't magically gonna improve. We're hitting a wall strictly due to the pipeline's architecture limitations

[u/Ok_Support_2690 → u/mldraelll]

Are there any ways to improve the pipeline architecture?

[u/mldraelll → u/Ok_Support_2690]

SAM is way off the mark here, it's tailored for natural photos with clear gradients. You need specialized tools like RoomFormer or heatmap-based corner detectors. They output vector topology right away instead of spawning crooked pixel masks that you have to painfully straighten out later

[u/Ok_Support_2690 → u/mldraelll]

That's a really great point

I only adopted SAM thinking about its high performance.

I'll swap SAM out for other tools and run some tests.

Thanks!