pvm

Why there are no Windows guests

PVM nodes run Linux guests only. This is a property of PVM’s design, not a missing feature, and the analysis below is recorded so the question does not have to be re-opened.

What PVM asks of a guest

A PVM guest kernel executes at hardware CPL3; there is no ring 0. Guest “supervisor mode” is a software flag plus a second shadow CR3, toggled by the switcher. Linux copes because it is recompiled: CONFIG_PVM_GUEST selects PARAVIRT_XXL, so the operations that cannot work at CPL3 are replaced with hypercalls at build time, and the kernel is linked PIE into an address range the hypervisor confines it to.

PVM does emulate privileged instructions for guests that still execute them — a #GP at CPL3 lands in handle_exit_exception(), which calls kvm_emulate_instruction() with KVM’s full x86 emulator. So the barrier is not “there is no emulator.” It is what that path costs and what it cannot do.

Why Windows cannot use that path

• CR8. Windows x64 implements IRQL directly as CR8 — KeGetCurrentIrql is mov rax, cr8, and every spinlock acquire/release pair is two CR8 writes. The Hyper-V TLFS has no enlightenment for it; it explicitly directs 64-bit guests to set the TPR via CR8, because with hardware virtualization the TPR shadow makes that free. Under PVM every one of those becomes a trap. This single instruction plausibly consumes a large fraction of a core.
• PatchGuard actively probes for exactly this kind of hypervisor. KiErrata704Present sets TF, executes SYSCALL, and inspects the faulting RIP to discover the real LSTAR. PVM repurposes supervisor-mode SYSCALL as its hypercall instruction — a direct ABI collision, not an implementation detail. KiErrataSkx55Present probes POP SS/#DB semantics that the PVM spec deliberately deviates from. These end in bugcheck 0x109, not in slowness.
• The emulator’s gaps are the spec’s forbidden list. 64-bit IRETQ returns X86EMUL_UNHANDLEABLE; SWAPGS, SYSRET, STAC/CLAC are unimplemented — precisely the instructions the PVM spec forbids supervisor software from using, because Linux compiles them away. Windows executes all of them on every syscall and every interrupt return.
• Smaller but real: PVM removes IST (Windows uses several IST stacks per core), has no real mode (Windows AP bring-up needs the INIT-SIPI-SIPI trampoline), and lets SGDT/SIDT read host values.

Why the alternatives are worse

Binary translation is the architecture that would actually fit — translate guest kernel code, run guest user code natively, turn CR8 into a memory write. VMware shipped exactly this before 2006. It died with x86-64: the BT monitor protected itself using segment limits, which long mode does not enforce, and no replacement mechanism was ever productized. Building one now means recreating VMware’s engine to win back an advantage that hardware virtualization erased around 2008.

Paravirtualizing Windows itself is not ours to do: Microsoft Research ported Windows XP to Xen’s ring-1 PV mode in 2003 and could never ship it for licensing reasons, and ring 1 is unusable in long mode anyway.

Implementing the Hyper-V interface is legally clean — the TLFS is published under Microsoft’s Open Specification Promise, which is why KVM, Xen and others implement it. Modifying Windows or defeating PatchGuard is not.

What we do instead

Windows workloads run on nodes that have hardware virtualization, under KVM with the full hv_* enlightenment set — the UEFI firmware boot path cocoon already uses. PVM nodes and hardware-virt nodes form one mixed fleet, and scheduling keeps Windows on the latter.

This site is open source. Improve this page.