Type 1 and type 2 hypervisors, KVM and QEMU

Objective

Understand what a hypervisor is, place KVM and QEMU in the landscape (VMware ESXi, Hyper-V, VirtualBox, Proxmox VE), and be able to tell in two minutes whether a server can host virtual machines.

Key points

• A hypervisor shares one physical computer (the host) between several virtual machines (the guests), each with its own kernel.
• Type 1: the hypervisor runs directly on the hardware (ESXi, Hyper-V, Xen, KVM). Type 2: it is an application inside an already installed system (VirtualBox, VMware Workstation).
• KVM is a Linux kernel module: it turns Linux into a type 1 hypervisor, using the processor extensions Intel VT-x (vmx) or AMD-V (svm).
• QEMU provides everything else: the virtual motherboard, the disks, the network cards. A KVM VM is a qemu-system-x86_64 process on the host.
• Without vmx/svm in /proc/cpuinfo and without /dev/kvm, QEMU emulates the processor in software (TCG): it works, but 10 to 50 times slower.

1. Why virtualize

The MasterLinux office in Fès had five small physical servers: one for mail, one for the website, one for the database, one for backups, one for tests. Each one used 5 to 10% of its processor. After consolidation, two identical physical servers carry the five virtual machines, with room to spare:

• less hardware to buy, power and cool;
• a VM is created in minutes, and can be copied and backed up like a file;
• isolation: a crash or a compromise stays (in principle) inside its VM;
• mobility: a VM can move to another host during maintenance, sometimes without stopping (live migration).

The downside: a host that fails takes all its VMs with it. This is why shared storage, clusters and high availability come up quickly (level 3 of this course).

2. Type 1 and type 2

 Type 1 (bare metal)                     Type 2 (hosted)
 ┌─────┐ ┌─────┐ ┌─────┐                 ┌─────┐ ┌─────┐
 │ VM1 │ │ VM2 │ │ VM3 │                 │ VM1 │ │ VM2 │
 └─────┘ └─────┘ └─────┘                 └─────┘ └─────┘
 ┌───────────────────────┐               ┌───────────────┐ ┌──────────┐
 │      hypervisor       │               │  VirtualBox   │ │ browser  │
 └───────────────────────┘               └───────────────┘ └──────────┘
 ┌───────────────────────┐               ┌──────────────────────────────┐
 │       hardware        │               │  host system (Windows, …)    │
 └───────────────────────┘               └──────────────────────────────┘
                                         ┌──────────────────────────────┐
                                         │           hardware           │
                                         └──────────────────────────────┘

KVM is a special case: you first install Debian, Ubuntu or Rocky, then you load a module. But once the module is loaded, the Linux kernel itself schedules the VMs like processes: there is no intermediate layer, hence the type 1 classification. All the major Linux-based public clouds use KVM or a derivative.

3. Sharing the work: KVM, QEMU, libvirt

  virsh / virt-manager / Proxmox (qm)      ← administration tools
              │
           libvirt (virtqemud)             ← API, XML, networks, storage
              │
   qemu-system-x86_64 (one process per VM)  ← virtual hardware: disks, network, BIOS/UEFI
              │  ioctl()
           /dev/kvm  (modules kvm + kvm_intel or kvm_amd)  ← runs the guest CPU
              │
          VT-x / AMD-V processor

• KVM runs the guest code directly on the real processor, in a protected mode designed for this. It also manages the guest memory (EPT on Intel, NPT on AMD).
• QEMU is the emulator: it presents the guest system with firmware (SeaBIOS or UEFI/OVMF), a disk controller, a network card… Without KVM, it can also emulate the processor (TCG mode), which is useful to run an ARM image on an x86 PC.
• libvirt (next lesson) saves you from writing long QEMU command lines: the VM is described in XML and driven with virsh.

4. Check that a server can virtualize

# 1. Does the processor have the extensions? (counts the logical cores that report them)
grep -cwE 'vmx|svm' /proc/cpuinfo          # 0 = no, or disabled in the BIOS
lscpu | grep -i virtualisation              # "Virtualisation : VT-x" or "AMD-V" (French locale)

# 2. Are the modules loaded?
lsmod | grep kvm
# kvm_intel             409600  0
# kvm                  1388544  1 kvm_intel

# 3. Does the device exist?
ls -l /dev/kvm                              # crw-rw---- 1 root kvm 10, 232 …

# 4. Full diagnosis (package libvirt-clients on Debian, libvirt-client on Rocky)
virt-host-validate qemu
#  QEMU: Checking for hardware virtualization          : PASS
#  QEMU: Checking if device /dev/kvm exists            : PASS
#  QEMU: Checking for secure guest support              : WARN (…)

On an English-language system, lscpu prints Virtualization: VT-x or AMD-V.

The processor flag tells you the vendor, and therefore the module:

If the flag is missing on a recent Xeon, the feature is almost always disabled in the BIOS/UEFI (menus "Intel Virtualization Technology", "SVM Mode"). The kernel says so in its messages:

dmesg | grep -i kvm
# kvm_intel: VMX not enabled (by BIOS) in MSR_IA32_FEAT_CTL on CPU 0     (recent kernels)
# kvm: disabled by bios                                                  (older kernels)

On a VM rented in the cloud, vmx is often missing: the provider does not expose nested virtualization. You can then only use slow emulation.

5. Nested virtualization

Running a hypervisor inside a VM (Proxmox lab, training, OpenStack tests) requires the module's nested option:

cat /sys/module/kvm_intel/parameters/nested     # Y or N  (kvm_amd: 1 or 0)

# Permanent activation (applied the next time the module is loaded)
echo 'options kvm_intel nested=1' > /etc/modprobe.d/kvm-nested.conf
modprobe -r kvm_intel && modprobe kvm_intel       # all VMs must be stopped

Then the VM that will act as a hypervisor must receive the flag: -cpu host with QEMU, or <cpu mode='host-passthrough'/> with libvirt. Recent kernels often enable nested by default; check it instead of assuming it.

6. Start a VM with QEMU alone

You almost never do this in production, but doing it once shows you what libvirt and Proxmox write for you:

# A 20 GiB disk (details in the storage lesson)
qemu-img create -f qcow2 /var/lib/libvirt/images/ml-test-01.qcow2 20G

# -enable-kvm -cpu host : KVM acceleration, CPU identical to the host
# -m 2048 -smp 2        : 2 GiB of RAM, 2 vCPUs
# -cdrom … -boot d      : boot from the installation ISO
# -nic user,…           : "user" network, a NAT built into QEMU, no configuration needed
# -nographic            : serial console in the terminal (no window)
qemu-system-x86_64 \
  -enable-kvm -cpu host \
  -m 2048 -smp 2 \
  -drive file=/var/lib/libvirt/images/ml-test-01.qcow2,format=qcow2,if=virtio \
  -cdrom debian-13-netinst.iso -boot d \
  -nic user,model=virtio-net-pci \
  -nographic

While the VM is running, on the host:

ps -eo pid,%cpu,rss,cmd | grep [q]emu-system   # the VM is a process like any other
top -H -p <pid>                                 # one thread per vCPU ("CPU 0/KVM", "CPU 1/KVM")

A VM therefore follows the same rules as any process: it uses memory (RSS) and CPU time, and the kernel can kill it when memory runs out (OOM killer).

By version

Common pitfalls

• Searching for vmx without -w: grep vmx also finds vmx flags on some kernels and can mislead you; grep -w searches for the whole word.
• Concluding "not compatible" too quickly: a Xeon without vmx almost always has the feature turned off in the BIOS. Check dmesg before you order a new server.
• Forgetting that the host needs resources: 64 GB of physical RAM does not give you 64 GB of VMs. Keep 2 to 4 GB and one or two cores for the host.
• Nesting in production: nested virtualization is handy in a lab, but not recommended for real services (performance, migrations impossible).

Lab

On a Debian machine, three suppliers sent extracts of /proc/cpuinfo and dmesg for the future hypervisors of the Fès office. You will:

1. sort out the servers that can run KVM;
2. match each one with the right kernel module;
3. enable nested virtualization on the Intel server;
4. write the QEMU command that starts a test VM.

Quiz

1. What is the difference between a type 1 and a type 2 hypervisor?
2. What does KVM do, and what does QEMU do?
3. An Intel server does not show vmx in /proc/cpuinfo. What is the most likely cause, and where can you confirm it?
4. Which module do you load on an AMD EPYC processor?
5. How does a KVM VM appear in the host's process list?