Google Cloud Nested Virtualization: How to Enable It on a GCE Instance

There are times when you need to run a virtual machine -- but you're already ON a virtual machine.  Fortunately, it's possible, but you need to enable nested virtualization.  For me, this comes up often when I'm running OpenStack or Kubernetes on a Google Compute Engine instance on the Google Cloud Platform (GCP).  To help solve the problem, you can enable Google Cloud nested virtualization.  Follow these steps:

1. Install the latest version of the Google virtualization beta command-line tool  gcloud (no need to sign in to your Google account).
2. Create a new instance so you have a base disk to work with.  Because you'll eventually want to use the image in a zone that includes Google Cloud nested virtualization,  create it in zone us-central1-b.  You can do this from the UI, or using the command line. By default, the disk will have the same name as the instance:

   gcloud compute instances create temp-image-base --image-family=projects/ubuntu-os-cloud/global/images/family/ubuntu-1604-lts --zone=us-central1-b

3. Stop the instance:

   gcloud compute instances stop temp-image-base --zone=us-central1-b

4. Now create a new disk, based on that disk, with Google Cloud nested virtualization enabled:

   gcloud compute images create nested-vm-image \
     --source-disk=temp-image-base --source-disk-zone=us-central1-b \
     --licenses="https://www.googleapis.com/compute/v1/projects/vm-options/global/licenses/enable-vmx"

5. Next create the new instance using the new image:

   gcloud compute instances create nested-vm --zone us-central1-b --image=nested-vm-image --machine-type=n1-standard-32 --boot-disk-size=250GB

6. Connect to the instance:

   gcloud compute ssh nested-vm --zone=us-central1-b

7. Confirm that nested virtualization is enabled by looking for a non-zero response to:

   > grep -cw vmx /proc/cpuinfo
   > 1

8. Finally, install a hypervisor  for your nested GCP (KVM is a fantastic choice.)

   sudo apt-get update && sudo apt-get install qemu-kvm -y

There you have it. That’s all you need to know about setting up nested virtualization. GCP can now run another VM instance from within the platform.
While these instructions only apply to GCP when you want to enable nested virtualization, Google Cloud isn’t the only platform where you’ll encounter this problem., you can easily run into the same issues on bare metal machines running an OS like Windows Server, or on cloud instances that come from another cloud platform, such as Amazon Web Services.  The solution depends on your environment. For example, if you're trying to run nested virtualization on your desktop intel machine (maybe for better data privacy), it might be a matter of changing BIOS settings. For Hyper-V, you need to change settings on the VM itself (assuming that other requirements, such as an appropriate processor, are met).

For Amazon Web Services, in most cases, when you run an "instance" that is, itself, a virtual machine, so if you want to run Virtualbox or other virtualization software such as VMware or Hyper-V, you're going to have a problem.  Fortunately, many cloud service providers also include the ability to rent "bare metal" instances, which give you direct access to the hardware. From there, you will need to follow the same instructions as you would for local hardware.