Nested Hyper-V 2012 R2 on ESXi 5.5

Today I`m gonna show in quick step-by-step guide how to prepare ESXi and properly install Microsoft Hyper-V 2012 R2 as a VM inside ESXi 5.5 host. I assume that you have basic knowledge of how to install ESXi, use vSphere client, SSH and a bit of Linux commands. I actually have two ASRock Z87M Pro4 with Intel i5-4440 based whiteboxes with ESXi 5.5 installed on them. You can find in my previous post how to build it. Before you start make sure you have tool to manage Hyper-V. I use Windows 2012 R2 virtual machine to do it. It is also a domain controller of my home lab. After completely fresh installation of ESXi here is what you need to do and why:

  1. Set up ssh access to the host. This is just for your convinience. To do this log in to your Esxi host through vSphere client (simpliest way) and go to Configuration tab and then click on “Security Profile”. Click on the “Properties” in “Services” section. In new window locate SSH, click on it and then click “Options” button. Click “Start” and then “start and stop with the host”. Click ok to apply changes and ok again to close window.
  2. Second thing on host configuration side is to allow promiscuous network communication from and to VMs inside Hyper-V. Go to configuration tab of your ESXi host and click on “Networking”. Then go to your vSwitch properties by clicking “Properties..”. In new windows select vSwitch end click “Edit” then go to “Security” tab  and change “Promiscuous mode” to “Accept”. Click Ok, and then Close.
  3. Now you have to ssh to your server. After that browse to /etc/vmware/ catalog and either use vi to put additional parameter (vhv.allow="TRUE") to config file or simply enter:
    echo 'vhv.allow = "TRUE"'>>config
  4. You can check with “tail config” command if you are succesfull:
    tail_config
  5. Now you can create new virtual machine. Choose “custom one” on the first screen. Use defaults settings until you reach “Guest operating system”. Choose “Windows 2012 64bit” Get at least 2 vCPUs (I believe its better to use 2 virtual sockets with 1 core per socket than 1 socket with 2 cores, I will do research to be more sure on that some day and post results – read this document ), and you get all defaults later. If you have SSD as a storage system you can use thin disk, I do. Use VMXNET3 network interfaces.
  6. Get back to your ssh session to the host. Browse to “/vmfs/volumes/your_data_store/name-of machine” in my case it would be:
    browse_to_VM-folder
  7. Again use vi to enter following values to your machine`s vmx file:
    monitor.virtual_mmu = "hardware"
    monitor.virtual_exec = "hardware"
    cpuid.1.ecx = "---- ---- ---- ---- ---- ---- --H- ----"
    hypervisor.cpuid.v0 = "FALSE"
    vhv.enable = "TRUE"
    sched.mem.maxmemctl = "0"
    You can also copy and paste this (remember to use some editing tool to change your vm`s name first):
    echo 'monitor.virtual_mmu = "hardware"' >> vhv.22.vmx
    echo 'monitor.virtual_exec = "hardware"' >> vhv.22.vmx
    echo 'cpuid.1.ecx = "---- ---- ---- ---- ---- ---- --H- ----"' >> vhv.22.vmx
    echo 'hypervisor.cpuid.v0 = "FALSE"' >> vhv.22.vmx
    echo 'vhv.enable = "TRUE"' >> vhv.22.vmx
    echo 'sched.mem.maxmemctl = "0"' >> vhv.22.vmx
    Check if it worked fine, use cat or tail command.
  8.  Make ESXi reload your machine`s settings. Enter these commands:
    vim-cmd /vmsvc/getallvms
    Note the Vmid of the machine you are preparing. In my case this is “1” because I`m doing it on fresh installation of ESXi. If you had any VMs before, you will get this number different. Last command is:
    vim-cmd /vmsvc/reload
    followed by your Vmid number. In my case it would be:
    vim-cmd /vmsvc/reload 1
    Check a screnshot as a reference:
    make_ESXi_refresh_your_machines_settings
  9. After that in your vSphere client you should have all properties of your VM in right way. This is not the only way to achieve it, but the fastest in my opinion. In many articles you can find how to do it using a gui.
  10. Now you can install Hyper-V 2012 Server R2. Get your ISO image from here. You don`t need any help with that, do you?
  11. After installing Hyper-V install VMtools on it. You have to browse in command prompt window to your cdrom drive and manually start setup64.exe to get it done. Now restart Hyper-V and make sure it has proper IP address, DNS settings, gateway and time .
  12. Join Hyper-V to your domain for easier management.
  13. Now you can connect to your hyper-v virtual host, create VM and run them. On a single physical host like mine you can run up to 3 hyper-v “virtual hosts” without problems.

One thing worth noticing it is the line you put into your vm`s vmx file:

sched.mem.maxmemctl = "0"

It tells your ESXi host no to use baloon driver inside vm. So if you ran out of RAM on ESXi don`t use it, get some more memory. Make sure you have enough RAM to run all of your hyper-v vms because you won`t be able to start any vm inside hyper-v because vmware`s baloon driver can consume a lot of it. And this driver is not really fast, you have to wait a long time to get its job done, so keep it in mind. I generally do not overprovision RAM on my hosts.

You should have abilities to install hyper-v hosts in your virtualized environment. In next posts I will explain to you how to install and use System Center Virtual Machine Manager to manage your virtualized environment. 

2 thoughts on “Nested Hyper-V 2012 R2 on ESXi 5.5

  1. Pitchdown

    Hello,
    i have followed your article step by step.
    When starting a VM from Hyper-v i receive the following error :

    Failed to start the virtual machine “test” beause one of the Hyper-V Components is not running.

    What to do to fix this?

    Thx

Comments are closed.