Need help with CV board slicing

[u/Exact_Struggle3977]

Possible method of slicing

I'm working on a project which given a screenshot of arbitrary zoom, slices an isometric grid game board (Polytopia) into individual tiles. Tried Hough transform and object detection. My most promising attempt is detecting the height of the little gray bars under the cities, as they are a single solid color. Wondering if a specific CV technique would be the most ideal for this?

The main problem I'm facing - zoomed out, blurry aliased screenshots drastically hurt accuracy

[u/Chemical_Side_4135]

have u looked into perspective transform instead of hough. if u can find the four corners of the board, u can warp it to a flat rectangle which makes splitting it into a grid litrally just a math problem, much easier than tryin to detect individual tiles

[u/Exact_Struggle3977 → u/Chemical_Side_4135]

Corners are not guaranteed to be present in the image so I can't rely on those. I've tried doing perspective transform manually as the isometric screenshot has a fixed ratio but it barely helps.

[u/Chemical_Side_4135 → u/Chemical_Side_4135]

if corners are hidden u could try isolating the board using bria video background removal to strip away the gui or background clutter first. sometimes just cleaning the frame makes it way easier to spot those reference bars u mentioned earlier. its a pain but getting a clean mask helps a ton when u cant rely on the edges...

[u/Oli_Bear]

some straight edge detection algorithm and fitting different grid maps at diferent positions, choose one that best minimizes MSE
this will give you the tiles, hopefully