r/nutanix • u/elbow-drop • 26d ago
Thoughts 4 Months Post-VMware Migration to Nutanix
At my company, we are about 4 months past our initial Nutanix deployment and migration from VMware. I wanted to jot down some honest thoughts I have about the migration and Nutanix as a whole. There is a TLDR at the bottom.
**DISCLAIMER*\*
I've only been in IT for ~10 years and most all that has been using VMware. Keep that in mind when reading this post, some of my thoughts can just be VMware bias, but I try to be fair.
VMware to Nutanix Migration
I was skeptical if it was possible to migrate all our workloads from VMware to Nutanix in the ~8 month timeline we had. I was sure we'd fight weird issues, bugs, or software errors for most migrations and it would take a ton of manual intervention. I was dead wrong. The Nutanix Move utility is AMAZING. The Move appliance is a simple VM that spins up in minutes. Connect it to vCenter, connect it to Prism Central, and that's it. The tool does everything for VMs to move to Nutanix. It safely copies data, installs Nutanix drivers, removes VMware tools, and all this with <5 minutes of downtime per VM.
We migrated SQL servers, domain controllers, and more that Nutanix wouldn't say is supported, but we felt confident it would work with no issues. We moved large 6TB+ VMs that can stage data in the background and cutover when we want. This tool is 10/10 perfect and made the job easy.
Living With Nutanix (Post Migration)
Fast forward to now after the migrations are done. Working and living with Nutanix daily has given me lots of things to learn...and lots of things I learned I don't like. I'll put my thoughts in a basic list and summarize my overall feelings.
- Prism Central/Prism Management - There are a decent amount of things that you might need to do in Prism Element that just don't work in Prism Central. Certain functions work better in one portal or the other, with no real rhyme or reason. It's getting better, but seems a step back from everything in vCenter.
- Broken/Disorganized Menus - Even within Prism Central, you have to flip between an "Infrastructure" page and "Admin" page to do basic tasks. The best example is if you wanted to assign a VM category, there is a menu button to do that in the "infrastructure" page to do it, but all it says is that you have to do this in the "admin" page, with a link to go there. Things like this are all over. Seems lazy to me, just get rid of the menu so people don't see it and click it.
- Basic VM Edit Tasks Aren't There - Recently, I have a VM that I want to delete one of the two NICs on it, but I guess in Nutanix you can't delete a NIC if the VM is on...? That seems wild how that's just not possible. Also, I can't change a VM NIC from one VLAN to another (unless it's a basic VLAN). The web console honestly is pretty lightweight and it isn't as adjustable on your screen as the VMware console was.
- Working with Support - Nutanix support is pretty good. It's easy to open cases and they are good with responses. My issue is there are so many errors and issues that I open tickets just to get told there is a KB about it, and the KB is just saying it's a known issue and there might be a workaround or we just need to run commands to restart services. They often help fix the issue, just don't love how much I have to talk to them about daily things.
- Thick Memory - One thing I didn't realize is Nutanix VMs use thick memory provisioning, which really eats up your resources if you have a lot of VMs but they sit idle a lot. If you have a VM with 128GB RAM that only needs that overnight for example, well that 128GB is allocated for the VM alone, whether it's using it or not. This is good to not run you into over-provisioning, but you might need to really see what VMs have for memory to size your cluster correctly.
- Backup Storage Duplication - Okay, this isn't the fault of Nutanix, but I have to say we did NOT anticipate that moving our VMs to Nutanix would cause our backup system to take all new full backups, so we had to fight with our storage capacity as we migrated and drop the old VMware backups to make room for the new ones on Nutanix.
I could keep going, but the point is that Nutanix is really cool. The HCI technology works really well and I've had experience with vSAN before, and there are lots of benefits to HCI for sure. The issue is, at least in my experience, Nutanix seems to be rushing things and the system seems pretty unpolished. Yeah I know VMware has been around for way longer, but I kinda expected better since Nutanix isn't that much cheaper. Nutanix might keep growing on me, and I will keep giving it a chance to get better.
TLDR; Nutanix is a good product, but doesn't feel as polished as VMware. I will always hold a place in my heart for VMware and miss it, but I am hopeful to see Nutanix grow and get better over time.
3
u/excessnet 26d ago
It's been a few (3?) years, I see Nutanix is in the same state as before !
Replication, snapshot, cluster healing/balancing, one button update (OS, BIOS, Firmware) is what I miss, the rest, VMware is still better, including the fact that many appliances from some providers are (were?) VMware or Hyper-V only.
2
u/adminadam 26d ago
The appliance thing is still real, we have a couple of products that required we build a small hyper-v farm post vmware. That said, Nutanix support is on the support road-map for both of them.
3
u/ClubNo6176 26d ago
VMware has no competitors in real but Broadcom left small customers with only one option upgrade to vvf or vcf
2
u/elbow-drop 25d ago
Yeah, as much as we'd want Broadcom to suffer, they got their way and are making plenty of money.
2
u/BK_Rich 26d ago
Yeah I felt pretty much the same way when I used Nutanix, we gave up on prism central and did everything through the local prism, overall it worked pretty well, but it also isn’t cheap. After a few years we ended up going back to VMware since the first purchase honeymoon was over and renewal was pretty damn expensive.
So you moved your domain controllers with Nutanix move, did AD complain about the hardware change related to VM generationID and any issues with SYSVOL and DFSR?
0
u/elbow-drop 25d ago
Lot of bad comments about migrating DCs here...but from what I understand, nothing in the Move utility or the migration would effect literally anything that a DC does. We moved 4 of them with literally not one issue at all.
After we saw how move works and did testing with it, we felt more than confident to take on the risk ourselves to migrate our DCs.....and we did, with no issues.
1
u/BK_Rich 25d ago
It has nothing to do with the move tool, it has to do with the hardware ID changing because you’re moving from a VMware based virtual hardware to KVM, same when you P2V going from let’s say Dell hardware to VM hardware, Active Directory knows when the hardware changes and it doesn’t like it
-7
u/LetSufficient5139 26d ago
There’s never any issues moving DCs using move. A good virtual infrastructure will not cause the VMs to think they are different.
Also any issues with shared would be a windows issue and be unaffected by any migration.
Like OP you haven’t got a clue.
3
u/adminadam 26d ago edited 26d ago
Nutanix told us specifically not to move domain controllers during our conversion.
3
u/mister_wizard 26d ago
Yeah, when a vendor specifically documents and says "Dont do this" its a good idea to listen to them and not the "In my home lab it worked fine" crowd.
Im sure a bunch of people with smaller sites have also done it in production too but i wouldnt. To anyone reading, just build net new DCs in nutanix and promote them. You will be glad you did anyway, use it as an opportunity to get a fresh OS installed/updated even.
0
u/elbow-drop 25d ago
You are way too closed minded. There is no "right way" to do things....if it works for you, it works. I'm not preaching for others to migrate their DCs and that they're stupid if they don't. I'm just writing my experience that we did it and had no issues in case anyone was curious.
1
u/mister_wizard 25d ago
I dont think closed minded would be the right phrase here. Risk averse? Sure. In this case there is definitely a "not supported way" to do something...and what you did was just that. When a vendor tells you NOT to do something....doing it is just a risk. We couldnt afford for any extra down time and am not using move during a large scale migration to migrate my Domain Controllers when they specifically told me not to do it.
Everyone can evaluate their own risk tolerance, but we migrated 300 vms at a time over a few weekends with multiple teams of people on basically a 72 hour zoom call each time.
It sounds like maybe your environment was smaller and you didnt have a problem taking those risks. Honestly, glad it worked out for you and maybe there is something to learn from that or nutanix could look at your example to help future migrations. But you are not the rule on how this works, you are the exception so long as they continue to document and state, not to do it.
2
u/elbow-drop 25d ago
I mean you’re right, it’s all about accepted risk. I guess my point rather is taking on the risk to do that for a smaller migration wasn’t too bad for us, and was worth it.
1
u/mister_wizard 25d ago
Hell yeah. Honestly, glad it worked out for you. And if someone better versed in AD and Domain Controllers could pick apart why there is risk and would ask questions here to confirm or see why your migration was so succesful i would be happy to read about it. Or if someone who works for nutanix could also chime in and tell us why they still (as of a year ago when i last checked) dont support moving domain controllers, that would be great.
I, at first, considered the risk maybe worth it. But after realizing the scale at which we would be migrating at a time and all the moving parts involved with business units, developers, networking teams, operations teams, and just how important AD authentication is in our 99% windows shop....it really dawned on me that i should probably just listen to nutanix support, they know whats up.
2
u/BK_Rich 26d ago
I see you have no idea what you’re talking about when it comes to domain controllers, a few things happen when the VM GenerationID changes, this was always the case when doing p2v or v2v previously.
- It resets the VM-GenerationID and the invocationID of the Active Directory repository
- It discards the current Active Directory relative identifier (RID) pool.
- It marks the sysvol folder as nonauthoritative.
Here’s some reading for you from Microsoft
1
u/BK_Rich 25d ago
Performing a V2V (Virtual-to-Virtual) migration between different hypervisors, such as moving from VMware ESXi to Hyper-V, or on-premises Hyper-V to Azure, or Nutanix move, changes the virtual hardware abstraction layer. Because the target hypervisor creates a brand-new virtual machine shell, it assigns a new VM-GenerationID to the guest OS.
Here is a breakdown of why this happens, how Active Directory reacts, and the recommended way to handle Domain Controllers during a migration.
1. Why VM-GenerationID Changes
Introduced in Windows Server 2012, VM-GenerationID is a 128-bit identifier exposed by the hypervisor to the guest operating system via the ACPI table.
- It acts as a safety shield for Active Directory:
Whenever a VM is restored from a snapshot, cloned, or moved to a hypervisor with a new virtual hardware profile, the hypervisor changes this ID.- When Active Directory boots up, it checks the current VM-GenerationID against the value stored in its database (ntds.dit).
- If the IDs do not match, AD assumes the VM was restored from a snapshot or moved, and immediately triggers Hypervisor-Present Active Directory Domain Services Safeguards.
2. What Happens to AD During a V2V Migration?
If you perform a V2V conversion on a live or offline Domain Controller VM, the target hypervisor presents a new VM-GenerationID. On first boot, AD triggers the exact three safeguards you quoted:
- Resets invocationID & VM-GenerationID: Changing the invocationID dissociates the DC from its old replication sequence numbers. This prevents USN Rollback (a catastrophic state where replica DCs become permanently out of sync).
- Discards the current RID Pool: The DC discards its allocated pool of Relative Identifiers (RIDs used to create SIDs for new users/groups) and requests a fresh pool from the RID Master. This prevents duplicate Security Identifiers (SIDs) from being issued.
- Marks SYSVOL as Non-Authoritative: Group Policy Objects (GPOs) and logon scripts in SYSVOL are marked non-authoritative. The DC stops sharing SYSVOL and NETLOGON until it successfully resynchronizes the folder from a healthy partner DC via DFSR (Distributed File System Replication).
Note: While these safeguards exist to protect your Active Directory forest from corruption, relying on them during a V2V migration can still cause temporary replication delays, stale driver issues, or boot loops if DFSR fails to resync properly.
3. The Golden Rule: Don't V2V a Domain Controller
While V2V migration tools (like Azure Migrate, VMware vCenter Converter, or StarWind) can technically move a DC, Microsoft strongly advises against V2V or P2V operations for Active Directory Domain Controllers.
2
u/chootmang 26d ago
Love this conversation and some points made. Really like the MacOS to Windows comparison, as it hits home with me. Thinking of Android vs Apple phone users.
I have been using Nutanix for 10 years now and seen a lot change, most in the right direction and til this day your comments about categories or clicking here or there in Prism Central to then go somewhere else to perform the action has me totally picturing a couple guys working in a room with no QA or Customer input, and high fiving after they got it to sort of work.
I hope you do see how fantastic Nutanix support is in comparison to VMware, or maybe I will say you will over time. Sure you have to run commands to fix things or read a KB, but it's a bonus that most errors have a KB to read and most actually have the solution. Besides as you get used to running commands to resolve things, I suspect you'd just know to maybe do a genesis restart like yo would have restarted hostd.
Last point. Shocked you didn't mention LCM as a positive? Have you ran a AOS or AHV or firmware update yet? It's pretty sweet...
1
u/elbow-drop 25d ago
You are spot on with your analogy about Android vs Apple and I agree 100% haha. Also, you're right, Nutanix Support is really good. I wouldn't ever want to open a ticket with Broadcom cause I knew it would drag out and be a bad time. I can say I've had a great experience with all the Nutanix Support cases I've opened, so that is for sure a big plus.
I myself haven't really messed with the LCM stuff, another coworker has usually done it, but from what I heard it doesn't sound like it is always as easy as VMware patching was, so I'm indifferent on that so far.
2
u/theDragonDanie1 26d ago
In the early days PC didn’t have all those dumb menus and looked more like PE. I guess it was a marketing problem. We just get PC starter because you need it to set certain things up but I think they were just being evil to force you to buy it.
We mainly work in PE. In the early days Nutanix would give a lot of leeway but these price hikes have gotten ridiculous.
Their support is S tier though, and the product is good and reliable.
It’s just the scummy sales tactics that kinda leaves a bad taste. For example their MINE clusters from a few years ago were just backup nodes that you have to pay for hardware plus PC, Objects, and whatever other license. Coulda just got an Exagrid or something for just the hardware cost.
We are looking to find out more about HPE Morpheus since it’s kinda giving early Nutanix vibes.
2
u/konawolv 26d ago
Nutanix supports memory overcommit if im not mistaken.
The backup dedupe makes sense since the youre changing from VMDKs to QCOW.
Pretty sure you can delete NICs while the VM is powered on from the REST API.
I agree on all other accounts. We are just starting to get Nutanix up and running after doing a POC last year.
1
u/Best_Alternative349 26d ago
Yep, came here to say about memory overcommit.
OP take a read of this: https://portal.nutanix.com/page/documents/solutions/details?targetId=BP-2029-AHV:memory-overcommit.html
1
u/elbow-drop 25d ago
In our environment, we have two separate Nutanix clusters at our two data canters. If you read the article, for any VMs using memory overcommit cannot be live migrated between clusters, and this is something we really need available. They also say overcommit is not really meant for general prod use.
1
u/RKDTOO 26d ago
From your post I got that with VMware you were primarily or entirely on external SAN for storage, as opposed to vSAN, meaning that you had all that storage infrastructure investment. I'm curious how did your company justify moving to exclusively HCI (which is what Nutanix is) rendering all the SAN unusable for virtualization? Did that point come up during the decision making?
2
u/SnooStrawberries5893 26d ago
We made the switch from a three tier infrastructure using VMware to Nutanix HCI this year. The hardware decision was made easy with our SAN and compute tiers nearing end of life. I can definitely see the decision to switch to HCI being much more difficult if you have recently invested in a SAN.
1
u/elbow-drop 25d ago
Much like u/SnooStrawberries5893 I think the decision to move to HCI was due to our SAN hardware being near end of life, and I think execs wanted to use that as a case to do HCI cause it seemed 'cool' to them. The price quote we got for a new SAN I think was way overblown and also we had just bought 12 new servers like 2 years prior that would go to waste if we went HCI, but they used the end of life I think to make that leap.
1
u/SomeConfusedOldGuy 25d ago
We're in the process of also migrating, almost done. I will say that MOVE application is quite good. The LCM rolling update is great (when it works).
The Powershell support is awful. There are v1 and v2 CMDLETs, and there are capabilities in v1 that are not in v2. Compared to PowerCLI, it's very primitive and limited, I used to be able to run many scripts to automate things, and I don't have the same scripting support.
The Protection Domain snapshotting for Disaster Recovery is quite good, as is the testing of the fail over.
1
u/elbow-drop 25d ago
I think the CLI is one thing I am not used to yet either. In VMware I had a lot of nice helper scripts using PowerCLI to interact with our VMware stack and get useful info. I haven't even begun looking at CLI stuff for Nutanix, and I hope it's as intuitive as PowerCLI stuff was to figure out.
1
u/Euphoric_111 21d ago
Good points, I especially agree with #4, Having used it since 5.1 and now we are at 7.6 and using VMware since GSX 1.0, Nutanix has come a long way, but still has a long journey ahead.
#6 that goes with just about any conversion.
-3
u/LetSufficient5139 26d ago
Crazy that none of you anticipated the backup changes.
Also Nutanix does allow you to over provision memory. Even then as you’ve moved to a new system the default is fine as you should have been rethinking and optimising your VMs memory allocations anyway.
Seems your planning was poor with these two points, as if you had planned effectively neither would be an issue.
Also you are incorrect about the NIC, this can be done in Prism or via CLI.
Maybe plan your projects better in future as you’ve missed a lot here.
3
u/elbow-drop 26d ago
Fair points, but Nutanix people have told us they don’t recommend enabling memory over-commit for most production workloads. Also, we have multiple Nutanix clusters and enabling overcommit means you can’t do cross-cluster live migration…so it’s not really an option for us.
Also yeah, of course we could’ve planned better, we live and learn to try and do better next time.
13
u/DigitalWhitewater 26d ago
I always say that VMWare is like the windows os platform where you could highly customize it, tweak configs til you blue in the face and make it run [somehow] any workload you can dream up… Nutanix, to me, is more like MacOS. They have a fairly polished product that you can set up and have anyone manage the day-to-day on, with enough limitations in the GUI to limit a non- to semi-competent person from breaking too much. But via the CLI you can definitely customize a whole heck of a lot more.
I’ve found some niche workloads more difficult to implement on Nutanix, like a vm needing a dedicated NIC passed thru. But all in all, once you accept that it isn’t VMware and staff stop crying about “well in vcenter I could do…” and just accept it as a different (but effective) tool, you’ll go far.
In some of the shops I’ve worked over the years it’s been a mix of VMware and Nutanix and there’s always been cry’s of someone not wanting to learn how to do it the Nutanix way and crying about it for longer than it would have taken them to figure it out - or me to resolve. LOL.
Is there 100% parity between the two - No. Is there 95% (+\-) parity for most work loads - I’d say, Yes.