How to Run a VMware vSphere Cluster With One $700 PC

One of the most frequent questions I am asked is how to build the least expensive functional VMware vSphere (ESX / vCenter) environment.

The answer is nested virtualization - the ability to start a virtual machine inside a virtual machine. VMware Workstation 7.x and ESX 4.x have made this remarkably easy to do. Workstation 7 supports this - you can set the guest type to "ESX" - and it is no longer necessary to use the monitor_control.restrict_backdoor = "TRUE" directive in order to power on a nested guest.

I have a two host cluster running in VMware workstation 7 on a $700 PC with an AMD Phenom(tm) II X4 830 2.8 Ghz Processor with the second-generation AMD virtualization technology (VT) extensions. This PC has 8 GB RAM and 2 1 TB disks, one of which I reserve for storing virtual machines and templates. I formatted this second disk with the largest possible cluster size under Windows 7 and disk access seems pretty fast but I have only anecdotal evidence that using the largest cluster size improves performance working with large files typically used by VMs.

My virtual cluster consists of two ESX 4 hosts, a vCenter server, and a VMsafe firewall manager controlling a VMsafe firewall / IDS product. As you can see in the vCenter screenshot, each virtual ESX host has a VMsafe firewall control VM powered on and a Windows Server VM has been started on one of the hosts.


In the ps output visible on the console of the second host you can see the monitor processes running nested virtual machines inside this virtual ESX host.


I have exported and reconfigured the VMsafe firewall / IDS manager to run as a stand-alone VM alongside the vCenter instance, as I see no advantage to nesting these VMs and I am more interested in running nested consumer guests than management substrate guests.

Performance is good, though not quite equal to bare-metal, but I have not evaluated running production systems like this. One performance tuning step that can be taken in Workstation is to minimize disk IO related latency by configuring the memory setting shown in the third screenshot. In my case, I find that I will need more than 8 GB RAM to completely light up this cluster with this setting enabled. I plan to build another system dedicated to nested virtualization with at least two Intel i7 processors and 24 GB of overclocked RAM. I believe this amount of hardware will comfortably run a complex nested virtualization environment with up to a dozen nested virtual machines powered on.