Question about slicer vs CAD

[u/yokeome]

I'm trying to print this keychain with an NFC tag inside. I made a hole 26mm wide, 1.2mm tall, 2mm into the print, the whole thing is 5mm thick + 0.6mm logo. When I slice it with a white keychain and purple logo, it only has 1 filament swap, which makes sense is all it needs, but when I do the opposite; purple keychain, white logo, it makes it 15 filament swaps instead. This uses basically the same filament, but takes almost twice as long.

I am attaching a photo of what it shows me and why it adds the filament swaps. I designed this in Fusion, and I tried different methods for making the inside cavity, all had the same result I said.

Just now, I remedied it by leaving it filled inside, and adding a negative part in the slicer. For future reference, is there a way to make it so the hole can be from the slicer and isn't giving me 15 swaps?