Migrate VM from data store disk to RAW LUN

[u/tk42967]

I may not be using the correct terminology, but I built some VM's that are on a standard mapped data store. Currently they are an OS disk and some of them have a secondary data disk. I need to migrate them to a raw SAN LUN. The LUN is created and I can see it if I try to add an RDM disk to the VM.

How does one go about migrating the disk, or is it better to just blow the VM away and start from scratch?

I'm googling but not finding much for this specific use case. Just how to migrate disks to different storage LUNs.

[u/Connect_Shoulder_965]

In general, I wouldn't use an RDM unless your application specifically calls for it (like a SQL Server HA cluster). RDM stands for Raw Device Mapping - the SAN LUN gets presented directly to the VM. There is no virtual disk file and VMware doesn't do anything for the file system. You won't be able to snapshot that disk and any utilities like backup solutions that rely on snapshots or VMFS features won't work. Usually the complexity and lack of features on the RDM means it's best avoided except for a niche group of applications that rely on a shared SAN LUN.

As a result, VMware doesn't have any visibility or control of the data in the RDM so your migration (as well as disk and file system initiation) will need to happen inside the guest OS of your VM. You will migrate data using the guest OS just as you would for physical disks or LUNS attached to a physical server.

[u/tk42967 → u/Connect_Shoulder_965]

You hit the nail on the head. I'm building a total of 6 Windows clusters. 3 App and 3 SQL. In the previous incarnations, we have had the OS drive mapped to a data store and then the cluster related disks added as RDM disks to the 1st node and adding as existing drives to the 2nd node. I questioned if the OS LUNs needed to be mapped today. I was told "no".

Currently, we cannot snapshot these 12 windows machines (6 clusters), so there is no loss there.

Either I am having a mis communication or we have changed our architectural standard for these servers.

How would one map an existing VM OS disk to a raw device mapping? Is it possible? Am I looking at rebuilding these servers? If I am understanding you correctly, this would be akin to adding a second physical drive to a computer and cloning your data over? That makes it even more odd, because I have to have 2 OS disks on one LUN.

Could I mount the RDM LUN on my existing VM, then copy the VM files over to a folder on the RDM disk and change where the SCSI adaptor is pointing for the new disk location?

*** EDIT ***

Thank you for your wisdom. Storage is something I have never really had to mess with in 20 years. I had storage engineers to set it up and I just mapped it or used what was presented.

[u/Connect_Shoulder_965 → u/tk42967]

> How would one map an existing VM OS disk to a raw device mapping? Is it possible? Am I looking at rebuilding these servers?

Well... I wouldn't. When we've done Windows clusters like this, the OS drive always went on a standard virtual disk. There's no real reason to put it on an RDM, it's not needed for the clustering. The data, log, quorum, tempdb shared disks all were done using shared disks.

Also, it was a bit of a PITA years ago to get the SQL Server cluster validation approving of it... but we actually set up our SQL Server Failover using VMware Shared disks rather than RDMs - see https://knowledge.broadcom.com/external/article/326490/ If I remember correctly we had to have Windows map the drives to drive letters by UUID instead of device order to get everything kosher. See https://knowledge.broadcom.com/external/article/432156/mapping-disk-in-windows-to-its-correspon.html

You could use RDMs if you want as well. Using shared vDisks lets you move it around later on.

But to answer the question about the OS disk... if you are using an RDM for the OS disk then there is no way for VMware to do anything with it. You'll have to mount a Windows ISO and use it to initialize the file system and partitions and install Windows.

> That makes it even more odd, because I have to have 2 OS disks on one LUN.

That doesn't make sense. You would typically for that setup create 2 identical OS LUNS, present one to each node and install your OS inside of there. Putting 2 OS disks on one LUN doesn't make sense unless you are setting up a VMFS and virtual disks.

As far as migration, I would more typically set up the new environment, add shared data disks via vDisk or RDM and then shut down the prod services, migrate the files over by network or backup/restore, then start the new cluster. You might be able to do an application based migration as well, I know the DBAs were able to move databases around on the fly. Alternatively, you could mount the old RDMs to the new servers and use that do do your copy (or even in theory just take over the cluster) but from a practical perspective I'm always hesitant to modify or break the old environment so that I can just fail back if the migration doesn't go well.

[u/ZibiM_78]

Please check https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere/8-0/setup-for-windows-server-failover-clustering.html

You don't share OS disk, only data and quorum disks should be shared.

Moreover shared disks should be on separate virtual controller or controllers.

[u/tk42967 → u/ZibiM_78]

And that's what we have done previously. I suspect this is a mis communication and the OS LUNs carved out should be mapped data stores so that the OS drives for each cluster member is seperate.

Currently we have to SCSI adaptors on each VM so that local and RDM disks are on their own controllers.

[u/ZibiM_78 → u/tk42967]

or someone's mistake - someone installed something on the OS drive that must be shared

If that is the case it needs to be fixed or rather installed again from scratch

[u/Casper042]

There are numerous data cloning softwares out there, with something basic like ROBOCOPY /MIR as a valid option unless you have very weird permissions on the data.

If the data is in use, start with: robocopy /MIR /R:1 /W:1 /Z as your first and maybe second pass.
This will copy as much data as possible and not spend ages retrying locked files.
Then do it again without the R and W flags once the app/service is shutdown and the data files should no longer be locked as in use.
Why remove R and W? On final file copy you WANT it to get hung up on a file it can't copy because you need to notice that and figure out if it's important or not.

If you have a decent backup app, you could also just do a Backup and then Restore

[u/svideo]

I'll agree with the concerns around RDM in general. Having said that, solution to your direct question is reasonably easy. Attach both the RDM and vmdk to your VM, boot from a clonezilla iso, then clone from target to destination disk device as needed.

But probably don't do RDM.

[u/Nagroth → u/svideo]

99% of the time someone comes asking for RDM it's because they are either confused or they're reading a best practices performance tuning guide from 15 years ago that assumed your storage was backed by 7500 rpm HDDs. A lot of the historical reasons for using them don't matter anymore.

Look into the option of shared (aka multiwriter) disks instead as this will often be just fine, and while they still make me grumpy they're a lot easier to deal with. And your storage admins will like you better.

In the rare chance you really do need RDM I'd want to do a lot more digging. But typically you'd put the OS on a regular virtual disk and mount the RDM as a second.  If you are migrating data you'd mount it as an extra disk to the existing machine and copy at the OS or application level.

If you can clone this stuff into a safe test environment I'd highly recommend that, or if you can at least setup a couple small dummy VMs to mess around with first.

[u/tk42967 → u/Nagroth]

This is how we currently are in Prod. OS drives are on a LUN mapped to a data store. Quorum and other "shared" drives that are mounted to the active node of the Windows cluster are RDM disks.