cocoon

CLI Reference

Every command and flag. Concept guides: VM lifecycle, networking, snapshots, devices, GC.

CLI Commands

cocoon
├── image
│   ├── pull [--force] IMAGE [IMAGE...]  Pull OCI image(s) or cloud image URL(s) (--force re-downloads a cloud-image URL)
│   ├── list (alias: ls)           List locally stored images
│   ├── rm ID [ID...]              Delete locally stored image(s)
│   ├── import NAME [FILE...]      Import image from file(s) or stdin
│   └── inspect IMAGE              Show detailed image info (JSON)
├── vm
│   ├── create [flags] IMAGE       Create a VM from an image
│   ├── run [flags] IMAGE          Create and start a VM
│   ├── clone [flags] [SNAPSHOT]   Clone a new VM from a snapshot (or --from-dir)
│   ├── start VM [VM...]           Start created/stopped VM(s)
│   ├── stop VM [VM...]            Stop running VM(s)
│   ├── list (alias: ls)           List VMs with status
│   ├── inspect VM                 Show detailed VM info (JSON)
│   ├── console [flags] VM         Attach interactive console
│   ├── exec [flags] VM -- CMD     Run a command in a running VM via cocoon-agent (vsock)
│   ├── reseed [--machine-id] VM   Force a CRNG reseed inside the guest (fresh entropy over vsock)
│   ├── logs [-f] [--tail N] VM    Print the per-VM hypervisor log file
│   ├── rm [flags] VM [VM...]      Delete VM(s) (--force kills running VMs immediately)
│   ├── reconcile-stale-create VM  Reclaim an ownerless creating placeholder (JSON outcome)
│   ├── restore [flags] VM [SNAP] Restore a VM (running or stopped) to a snapshot (or --from-dir)
│   ├── hibernate [flags] VM       Atomically snapshot a running VM and stop it
│   ├── status [VM...]             Show VM status (--watch refreshes, --event streams)
│   ├── fs
│   │   ├── attach [flags] VM     Attach a vhost-user-fs share (CH only)
│   │   └── detach [flags] VM     Detach a vhost-user-fs share by --tag
│   ├── device
│   │   ├── attach [flags] VM     Attach a VFIO PCI device (CH only)
│   │   └── detach [flags] VM     Detach a VFIO PCI device by --id
│   ├── disk
│   │   ├── attach [flags] VM     Hot-attach an existing raw disk file (CH, or FC with --pci)
│   │   └── detach [flags] VM     Detach a hot-attached disk by --name (keeps the file)
│   ├── net [flags] VM             Resize NIC count on a running VM (CH, or FC with --pci)
│   └── debug [flags] IMAGE        Generate hypervisor launch command (dry run)
├── snapshot
│   ├── save [flags] VM            Create a snapshot from a running VM
│   ├── list (alias: ls)           List all snapshots
│   ├── inspect SNAPSHOT           Show detailed snapshot info (JSON)
│   ├── rm SNAPSHOT [SNAPSHOT...]  Delete snapshot(s)
│   ├── export [flags] SNAPSHOT    Export snapshot to portable archive (or stdout)
│   └── import [flags] [FILE]      Import snapshot from archive (or stdin)
├── gc [flags]                     Remove unreferenced blobs, VM dirs; --snapshot for LRU snapshot eviction
├── meta
│   ├── init                       Initialize a fresh sqlite meta store (normally automatic on fresh roots)
│   ├── convert                    Convert existing metadata to the configured meta_backend (crash-resumable)
│   └── backup DEST                Back up the sqlite meta store to a single consistent file (leaves a DEST.lock beside it)
├── daemon [flags]                 Supervise cocoon-managed VMs (optional resident process)
├── version                        Show version, revision, and build time
└── completion [bash|zsh|fish|powershell]

daemon is optional: every other command works standalone with no daemon running. See Daemon.

The meta engine is selected by meta_backend in the config; unset auto-resolves — an existing store binds its engine (legacy json roots keep json), fresh roots get sqlite and bootstrap themselves. meta convert always converts TO the configured meta_backend (unset = sqlite). Run scripts/meta-upgrade.py --root-dir <root_dir> (in the release archive next to the binary) once after a binary swap that raises the schema generation: it creates the tables the new generation declares, writes the placement rows for VMs the previous binary launched, and is safe to rerun. A sqlite store from an older generation is refused by name until then; a json root has no generation stamp, so it is not refused, but its running VMs stay out of the placement tally until the script has run.

Global Flags

Flag Env Variable Default Description
--config     Config file path
--root-dir COCOON_ROOT_DIR /var/lib/cocoon Root directory for persistent data
--run-dir COCOON_RUN_DIR /var/lib/cocoon/run Runtime directory for sockets and PIDs
--log-dir COCOON_LOG_DIR /var/log/cocoon Log directory for VM and process logs
--log-level COCOON_LOG_LEVEL info Log level: debug, info, warn, error
--cni-conf-dir COCOON_CNI_CONF_DIR /etc/cni/net.d CNI plugin config directory
--cni-bin-dir COCOON_CNI_BIN_DIR /opt/cni/bin CNI plugin binary directory
--dns COCOON_DNS 8.8.8.8,1.1.1.1 DNS servers for VMs (comma separated)

Config-file / env-only keys (no CLI flag):

Key Env Variable Default Description
pull_conns COCOON_PULL_CONNS 8 Concurrent HTTP Range connections per cloud-image download (image pull); raise for fat pipes, lower to be gentle on the registry
cgroup_parent COCOON_CGROUP_PARENT cocoon.slice cgroup v2 slice under /sys/fs/cgroup holding the per-VM CPU scopes; GC removes any empty vm-<id>.scope under it that none of its own VMs owns, so co-hosted installations need distinct slices, as with net_scope; see CPU Isolation
cgroup_cpus COCOON_CGROUP_CPUS empty (all cores) Host cpu list fencing the whole VM population (e.g. 0-14 reserves core 15 for the host); kernel cpu-list syntax
net_scope COCOON_NET_SCOPE empty (legacy names) Two alphanumerics keying this installation’s host network families — bridge TAPs <scope><vmid8>-<nic>, CNI netns <scope>-<vmid> — so co-hosted installations never GC each other’s; see Host device namespaces
ch_binary COCOON_CH_BINARY cloud-hypervisor cloud-hypervisor executable, path or $PATH name
fc_binary COCOON_FC_BINARY firecracker firecracker executable, path or $PATH name
meta_backend COCOON_META_BACKEND auto-resolved Metadata engine, json or sqlite; see the note above Global Flags
pool_size COCOON_POOL_SIZE host CPU count Goroutine pool size for concurrent operations
stop_timeout_seconds COCOON_STOP_TIMEOUT_SECONDS 30 Guest ACPI shutdown grace before SIGTERM/SIGKILL escalation
log.maxsize COCOON_LOG_MAXSIZE 500 Rotation: max log file size in MB; applies only when log.filename is set
log.maxage COCOON_LOG_MAXAGE 28 Rotation: max age in days of a retained log file; applies only when log.filename is set
log.maxbackups COCOON_LOG_MAXBACKUPS 3 Rotation: max number of retained rotated files; applies only when log.filename is set
use_firecracker COCOON_USE_FIRECRACKER false Make Firecracker the default backend, as if every VM command carried --fc
log.filename COCOON_LOG_FILENAME empty (stderr) Log file path; setting it is what switches logging from stderr to a file and enables the log.maxsize / log.maxage / log.maxbackups rotation knobs
log.usejson COCOON_LOG_USEJSON true unless stderr is a terminal Emit JSON-structured log records instead of plain text
socket_wait_timeout_seconds COCOON_SOCKET_WAIT_TIMEOUT_SECONDS 5 Per-phase launch timeout (API socket up, then VM running, then the vsock UDS on CH); exceeding it aborts the launch and kills the VMM — raise on slow storage
terminate_grace_period_seconds COCOON_TERMINATE_GRACE_PERIOD_SECONDS 5 SIGTERM→SIGKILL window when force-killing a VMM
metering.backend COCOON_METERING_BACKEND file Lifecycle-event recorder: file, meta, stderr, or nop
metering.file.path COCOON_METERING_FILE_PATH <root-dir>/metering/ledger.jsonl Ledger path for the file backend; see the rotation note in Daemon

The log section is decoded by field name, so its keys are the compact spellings — level, filename, usejson, maxsize, maxage, maxbackups (not use_json / max_age / max_backups). A dotted key maps to its env variable by replacing . with _, hence COCOON_LOG_MAXSIZE for log.maxsize.

VM Flags

Applies to cocoon vm create, cocoon vm run, and cocoon vm debug:

Flag Default Description
--fc false Use Firecracker backend (OCI images only)
--name cocoon-<image> VM name
--cpu 2 Boot CPUs — also the VM’s hard CPU cap (quota = N cores unless overridden; see CPU Isolation); refused above the host’s online core count
--memory 1G Memory size (e.g., 512M, 2G, 2Gi); minimum 512M
--storage 10G COW disk size (e.g., 10G, 20G, 20Gi); minimum 10G
--nics 1 Number of network interfaces (0 = no network)
--queue-size 0 (default 512) Virtio-net ring depth per queue; a power of 2 no greater than 32768 (larger = better bulk throughput, smaller = better RPC latency; CH only, ignored by FC)
--disk-queue-size 0 (default 512) Virtio-blk ring depth per device; a power of 2 no greater than 32768 (CH only, ignored by FC)
--network empty (default) CNI conflist name (empty = first conflist)
--bridge empty TAP-on-bridge mode (value is bridge device, e.g. cni0); mutually exclusive with --network
--user root Guest username for cloud-init (cloudimg only)
--password cocoon Guest password for cloud-init (cloudimg only)
--no-direct-io false Disable O_DIRECT on writable disks (use page cache; CH only, useful for dev/test with few VMs)
--no-watchdog false Omit the virtio watchdog device (CH only; use for guests whose watchdog driver cannot safely handle reboot)
--no-balloon false Omit the virtio-balloon device so the guest keeps its whole memory (the default balloons 25% back to the host; build-heavy guests that thrash before deflate-on-OOM fires want this)
--data-disk empty (repeatable) Attach an extra data disk: size=20G[,name=...][,fstype=ext4|none][,mount=/mnt/x][,directio=on|off|auto]. See Data Disks
--windows false Windows guest (UEFI boot, kvm_hyperv=on, no cidata)
--shared-memory false Enable CH memory shared=on; required for later vm fs attach (CH only, fixed for VM lifetime)
--hugepages false Back guest memory with hugetlbfs (CH only, fixed for VM lifetime); snapshots of such a VM restore via eager copy, never mmap
--pci false Boot on the virtio-pci transport instead of MMIO (FC only, fixed for VM lifetime); prerequisite for device hot-plug, snapshots keep the transport. See PCI transport
--mergeable false Mark guest memory MADV_MERGEABLE for host KSM dedup (CH only, fixed for VM lifetime; persists through clone/restore); needs KSM enabled on the host, excludes --hugepages/--shared-memory
--cpu-weight 0 (= vCPU count) cgroup cpu.weight 1..10000 — work-conserving share under host contention
--cpu-quota-us 0 (= vCPU count × period) cgroup cpu.max quota in µs per period — caps the long-run average; add --cpu-burst-us -1 for a strict per-period ceiling
--cpu-period-us 0 (= 100000) cgroup cpu.max period in µs
--cpu-burst-us 0 (= quota) cgroup cpu.max.burst credit in µs; -1 = none; kernel requires burst ≤ quota
--cpuset-cpus empty (anywhere in fence) Pin the VM to specific host cpus (kernel cpu-list, e.g. 0-3) or auto (the least-loaded last-level-cache domain at launch); non-work-conserving, explicit opt-in. Writable disk queue threads are pinned on every Cloud Hypervisor launch of a multi-vCPU VM, cpuset or not — see CPU Isolation

Every size flag (--memory, --storage, --data-disk size=, gc --snapshot-size) accepts the Docker spelling (20G, 20GiB) and the Kubernetes spelling (20Gi); all units are binary, so 20G is 20 GiB.

Clone Flags

Applies to cocoon vm clone:

Flag Default Description
--name cocoon-clone-<id> VM name
--nics inherit from snapshot Override NIC count at clone time; lets a 0-NIC snapshot clone with networking (CH hot-swaps NICs after restore; Firecracker only from a --pci snapshot)
--queue-size 0 (inherit) Virtio-net ring depth per queue; a power of 2 no greater than 32768 (0 = inherit from snapshot)
--disk-queue-size 0 (inherit) Virtio-blk ring depth per device; a power of 2 no greater than 32768 (0 = inherit from snapshot; CH only)
--network empty (inherit) CNI conflist name (empty = inherit from source VM)
--bridge empty TAP-on-bridge mode (value is bridge device); mutually exclusive with --network
--no-direct-io false (inherit) Disable O_DIRECT on writable disks (inherit from snapshot if not set)
--cpu-weight / --cpu-quota-us / --cpu-period-us / --cpu-burst-us / --cpuset-cpus 0 / empty (defaults, not inherited) The clone’s cgroup CPU policy; a snapshot’s knobs record its source VM and are never applied — omit for Guaranteed-at-N defaults; --cpuset-cpus also accepts auto, resolved at the clone’s launch
--restore-mode mmap for plain private-anon snapshots, else copy Memory restore mode: copy, ondemand (UFFD) or mmap (CoW map, shares page cache across clones); CH only, mmap needs Cloud Hypervisor v54 or newer (cocoon-check --upgrade installs the cocoonstack fork build) and ondemand v52 or newer: v52–v53 reject a mode they lack, and builds before v52 ignore the field and restore by copy; hugepages/shared snapshots degrade mmap to copy with a warning
--pull false Auto-pull base image if not found locally (for cross-node clone)
--from-dir empty Clone from a snapshot directory (must contain snapshot.json); mutually exclusive with positional SNAPSHOT
--data-disk empty (repeatable) Create a fresh data disk for the clone and hot-add it after restore: size=20G[,name=...][,fstype=ext4|none] (CH, or FC --pci snapshots; names must not collide with disks inherited from the snapshot)

CPU, memory, and storage all inherit from the snapshot — both hypervisors restore the guest from the snapshot’s binary device state, so those values are fixed at snapshot time. NIC count inherits by default but --nics N overrides it: Cloud Hypervisor hot-swaps the snapshot’s NICs for a fresh set right after restore, a Firecracker --pci clone restores the snapshot’s NICs and hot-plugs the delta afterwards; a Firecracker MMIO snapshot rejects the override (--nics override on clone needs a --pci snapshot on Firecracker). Use cocoon vm run to create a fresh VM with different CPU/memory/storage.

Network backend is decided per clone (the snapshot does not persist a bridge device). --bridge and --network are mutually exclusive:

  1. --bridge X → bridge backend with bridge device X.
  2. --network Y → CNI backend with conflist Y.
  3. neither → CNI backend, conflist inherited from the snapshot’s recorded vmCfg.Network (empty = CNI default).

A bridge-backed source snapshot cloned without --bridge silently defaults to CNI. Pass --bridge X at clone time to keep bridge mode.

Restore Flags

Applies to cocoon vm restore:

Flag Default Description
--restore-mode mmap for plain private-anon snapshots, else copy Memory restore mode: copy, ondemand (UFFD) or mmap (CoW map); CH only, mmap needs Cloud Hypervisor v54 or newer (cocoon-check --upgrade installs the cocoonstack fork build) and ondemand v52 or newer: v52–v53 reject a mode they lack, and builds before v52 ignore the field and restore by copy; hugepages/shared snapshots degrade mmap to copy with a warning
--from-dir empty Restore from a snapshot directory (must contain snapshot.json); mutually exclusive with positional SNAPSHOT
--force false Skip the snapshot-belongs-to-VM check. Valid only with --from-dir; used on its own it is rejected with --force only applies with --from-dir

CPU, memory, and storage come from the snapshot (the hypervisor reconstructs the guest from snapshot state, so the persisted record is realigned to match). NIC count must match the target VM — restore reuses its existing network namespace, TAP devices, and IP allocation. --force only skips snapshot ownership validation; it does not switch the target’s base image, and an incompatible base image is rejected before the VM stops. See restore constraints.

Snapshot Flags

Applies to cocoon snapshot save:

Flag Default Description
--name   Snapshot name
--description   Snapshot description

Export Flags

Applies to cocoon snapshot export:

Flag Default Description
--output, -o <name-or-id>.tar (.tar.gz with --gzip) Output file path (- for stdout)
--gzip false Compress output with gzip
--to-dir   Export into a directory (must be empty/absent) instead of a tar; pairs with vm clone --from-dir

--to-dir writes a snapshot.json envelope alongside reflink-copied data files. Useful for NFS golden images or rsync-friendly handoff: cocoon snapshot export snap --to-dir /nfs/golden && rsync .... Mutually exclusive with --output and --gzip.

Import Flags

Applies to cocoon snapshot import:

Flag Default Description
--name   Override snapshot name
--description   Override snapshot description

When FILE is omitted, data is read from stdin. This enables piping: cocoon snapshot export snap1 -o - | ssh host2 cocoon snapshot import --name snap1.

Direct Clone / Restore From a Directory

vm clone --from-dir DIR and vm restore --from-dir DIR accept a directory produced by snapshot export --to-dir. Do not unpack an export -o file.tar with a third-party tar and pass the result: cocoon stores sparse disks with private pax records that only its own extractor understands, so another tar rebuilds a short, content-shifted disk. To go through a tar, use snapshot import. The snapshot does not need to be in the local snapshot DB:

# Build a portable snapshot dir, ship it, clone from it:
cocoon snapshot export my-snap --to-dir /nfs/golden
# ... rsync /nfs/golden to host B if needed ...
cocoon vm clone --from-dir /nfs/golden --name fresh-vm --pull

# Restore the same VM's externally-staged backup (envelope ID matches → silent OK):
cocoon vm restore my-vm --from-dir /sync/from-host-a

# Override the envelope ownership check for a compatible backup of the same base image:
cocoon vm restore my-vm --from-dir /sync/from-host-b --force

The dir is read-only across the call, so multiple clones of the same dir (golden image use case) are safe. Pass --pull if the base image may not be present locally — EnsureImage pulls the envelope’s image by digest when it is missing, and image_blob_ids pin its blobs against GC for the clone’s lifetime.

Reconcile Stale Create

cocoon vm reconcile-stale-create VM reclaims a creating placeholder whose owning create or clone died mid-flight, freeing the record and its name. A free VM ops lock is the proof of ownerlessness — create and clone hold it from prereserve through the final record commit — so the verb never races an in-flight operation and needs no age heuristics (contrast vm rm --force, which queues behind a live clone and then deletes the freshly created VM). With --output json it prints {"id": "...", "outcome": "..."}:

Outcome Meaning
collected Placeholder reclaimed; record and name are free
busy An in-flight operation owns the VM; nothing was touched — not worth retrying blindly
not-creating The record has left the creating state; nothing was touched
not-found No record under that ref (already collected, or never existed)

All four outcomes exit 0; a non-zero exit is a real failure (store I/O, an orphan VMM that would not die). At a startup reconcile, busy means an external create or clone legitimately owns the record: leave it un-indexed and revisit on the next pass — it either becomes a live VM or becomes collectable. The daemon’s ownerless-create reconcile and GC’s stale-creating sweep run the same reclaim; this verb is for embedders that must clear skeletons synchronously (e.g. a startup reconcile) without waiting for either.

Reseed Flags

Applies to cocoon vm reseed (forces fresh guest entropy over vsock; runs automatically best-effort after clone/restore, invoke manually to retry a failed auto-reseed):

Flag Default Description
--machine-id false Also regenerate /etc/machine-id (use after clone, not restore)

Status Flags

Applies to cocoon vm status:

Flag Default Description
--interval, -n 5 Poll interval in seconds (only with --watch or --event)
--watch, -w false Refresh-loop mode (full-screen redraw each tick); omit for a one-shot snapshot
--event false Event stream mode (append changes instead of refreshing)
--format   Output format: json (one-shot and event modes; --watch always renders a table)

--event and --watch are mutually exclusive; passing both fails with --event and --watch are mutually exclusive. With neither, status prints a one-shot snapshot and exits.

Debug-only Flags

Applies to cocoon vm debug:

Flag Default Description
--max-cpu 8 Max CPUs for the generated command
--balloon 0 Balloon size in MB (0 = auto); ignored when the VM gets no balloon (--no-balloon or --windows)
--cow   COW disk path (default: auto-generated)
--ch ch_binary from the config cloud-hypervisor binary path

vm debug prints the VMM launch command only. It does not prepare a netns or a TAP, so it emits no --net and no ip=, and --nics, --queue-size, --network and --bridge are ignored — setting any of them warns on stderr, like --data-disk. --max-cpu, --balloon, --cow and --ch shape the Cloud Hypervisor command only; vm debug --fc prints the Firecracker REST sequence and ignores all four. Queue placement happens at launch against the store, so the printed command carries no queue_affinity whatever --cpuset-cpus says.

Console Flags

Flag Default Description
--escape-char ^] Escape character (single char or ^X caret notation)

For a running VM, cocoon vm inspect reports the console resolved at boot (start, clone, restore) as console_path: the console.sock UDS (UEFI serial, Firecracker relay) or the Cloud Hypervisor-allocated PTY (/dev/pts/N, direct-boot OCI). External supervisors can read the console from there without opening the owner-only API socket. A direct-boot VM booted by an older cocoon reports it from its next start. It also reports the resolved placement as cpuset and queue_cpus, both absent once the VM is not running.

Exec Flags

cocoon vm exec runs a command inside a running VM via the cocoon-agent (vsock, no SSH). Stdin/stdout/stderr stream like kubectl exec; the host shell sees the guest command’s exit code.

Flag Default Description
--env, -e   Extra env var KEY=VALUE (repeatable)
--interactive, -i false Attach the caller’s stdin to the command (otherwise stdin closes immediately)
$ cocoon vm exec myvm -- uname -n
myvm
$ echo hello | cocoon vm exec -i myvm -- cat
hello
$ cocoon vm exec -e FOO=bar myvm -- sh -c 'echo $FOO'
bar

Requires cocoon-agent to be running inside the guest. All official ghcr.io/cocoonstack/cocoon/ubuntu:*, ghcr.io/cocoonstack/cocoon/debian:* and ghcr.io/cocoonstack/cocoon/android:* images bake the binary and enable it on boot (systemd unit on Ubuntu and Debian, init.rc service on Android). The official ghcr.io/cocoonstack/windows/win11:* images bake cocoon-agent v0.2.4 as a Windows service via SCM; DIY Windows images need to install the agent themselves.

Logs Flags

cocoon vm logs prints the per-VM hypervisor process log (cloud-hypervisor.log or firecracker.log under the configured log_dir). The log captures VMM-side activity — device init warnings, API errors, virtio messages, shutdown — but not guest console output (use cocoon vm console for that). The file lives under the VM’s log dir for as long as the VM record exists (cleaned up on vm rm); each vm run / vm start appends to it, so the previous boot’s lines stay above the new one’s; -f also detects an external truncation and seeks back to the start of the file.

Flag Default Description
--follow, -f false Stream new log lines as they are written (Ctrl-C to stop)
--tail 0 Show only the last N lines (0 = all); pairs with -f for tail+follow
$ cocoon vm logs myvm
cloud-hypervisor:   0.003732s: <vmm> WARN:virtio-devices/src/block.rs:793 -- sparse=on requested but backend does not support sparse operations
$ cocoon vm logs --tail 5 myvm
... last 5 lines ...
$ cocoon vm logs -f --tail 10 myvm
... last 10 lines, then live tail ...

List Flags

Applies to cocoon vm list, cocoon image list, and cocoon snapshot list:

Flag Default Description
--format, -o table Output format: table or json

With json, an empty list prints []; table prints a one-line “No … found.” message.

Additionally, cocoon snapshot list supports:

Flag Default Description
--vm   Only show snapshots belonging to this VM