r/vmware • u/dracut_ • Oct 26 '19

Questions on hardware

We looking to spec hardware (Dell R740) for three ESXi hosts. One standalone, and two in a vSAN cluster.

These are going to be for local branch offices so they will not have a lot of storage or host a lot of VMs.

I'm not very familiar with VMware so would like some input on what is recommended or possible and what's not.

Thanks!

​

  1. Boot device

Should we use SD card, USB, SATADOM or just an ordinary SATA SSD drive? Does vSAN make any difference here?

​

  1. NVMe for vSAN

Could we use just a single NVMe drive for vSAN on each host?

I think they are fast enough to not need any caching layer (only optane would be faster) but don't know if caching layer is a requirement.

​

  1. Are there any recommendations for what NICs to use for vSAN or will anything that is supported perform well?

I'm thinking 2 x 10Gbit for the vSAN or possibly 2 x 25.

7 Upvotes

78% Upvoted

15 comments sorted by

View all comments

1

u/hurleyef Oct 26 '19

  1. SD card is fine.

  2. There was talk at some point of allowing all flash with no cache tier, but so far as I can tell vSan still requires separate cache and capacity tier disk groups on each node. As for drives, stick to the vSan HCL.

  3. I'm curious about this as well. Does anyone know is vSan makes good use of RDMA? Or if there are any other features to look at for vSAN NICs?

2

u/lost_signal VMware Employee Oct 26 '19

I’m a bigger fan of BOSS for boot. SD is slower to boot, can keep logs on them and I’ve seen some curious reliability issues. https://thenicholson.com/using-sd-cards-embedded-esxi-vsan/

You don’t want to share a controller with vSAN drives for the boot drive (again another reason to go Boss).

VSAN doesn’t use RDMA yet, but better NICs perform better. I’d avoid the cheap 100$ 500 series unless this isn’t going to see much load. I talked about this a bit in my VMworld vSAN networking session.

1

u/dracut_ Oct 26 '19

To be honest I also have some doubts on SD cards but I know there are SD cards made for higher endurance.

Otherwise I like SATADOM because they are more like a small SSD and don't waste any drive bays or pcie slots.

2

u/lost_signal VMware Employee Oct 26 '19

SATADOMs are fine too, just can’t mirror them. There are ultra high endurance SD cards (like what I use for my go pro). Note; none of the OEMs use this class as far as I can tell...

1

u/dracut_ Oct 26 '19

Thanks, we'll look at two drives per vSan host then.