cocoon sandbox

Security model

What the stack defends, where the trust boundaries sit, what the operator must provide, and what it deliberately does not defend. Read this before exposing any part of a deployment beyond a single trusted host.

Trust boundaries

  agent workload (untrusted code)
      │ syscalls
  guest kernel + virtio drivers
      │
══ VM boundary (KVM) ═══════════════ the isolation unit
      │ vsock (none lane) / nft-locked NIC (egress lane)
  host: sandboxd + cocoon            trusted
      │ HTTP control/data plane, bearer tokens
  clients: SDK / MCP / operators     trusted per token scope

Deployment assumptions

These are hard constraints, not suggestions; the guarantees above assume all of them.

  1. The control/data plane runs on a trusted private network. sandboxd serves plain HTTP; api_token, tenant tokens, and per-sandbox tokens are cleartext bearers on the wire, and on a cluster the SDK dials every node it is redirected to. Keep nodes and clients inside a VPC, WireGuard mesh, or equivalent, or put a TLS edge in front of every node (TLS for SDK clients). The only surface designed to face a browser is the preview listener, and it belongs behind a TLS-terminating proxy (deploy). Never expose listen itself publicly. A node without api_token authenticates no node-level call.
  2. One sandboxd per host (why).
  3. A cell is the boundary of a host compromise. Every node of a cell holds the cell’s credentials, so host root on one node reaches every claim, checkpoint and template of its cell. Cells share no credential, database, store or network path (cells). Set mesh.cluster_key unless the gossip network is itself trusted, and open the memberlist port to the cell’s nodes only: on a mesh without the key, any host that joins is sent client bearers, claim tokens and tenant tokens through redirects, Lookup and the cluster-wide tenant calls.
  4. Treat the checkpoint store like backups. A checkpoint, an archive and a replica each embed full guest memory, including any secret the workload held at capture. Restrict store access; use S3 server-side encryption or an encrypted mount where that matters. With archive_failover, treat the shared PostgreSQL the same way: its rows hold claim env in plaintext, guest: false values included unless host_env_memory_only is on.
  5. No custom NAT64 prefix routed on a sandboxd host — the SSRF guard cannot see through an operator-specific translator prefix (egress).

Cells

A host compromise on one node is bounded to its cell, and operators can cut a node off without its cooperation. Both hold only when the deployment meets this contract.

Per cell, each with its own permissions, not only its own name:

Per node, recommended: its own PostgreSQL role and store key with the cell’s grants, so one node can be cut off without a restart of the rest.

Ingress: the control, preview and mesh ports, PostgreSQL and the store admit only the cell’s node addresses, plus the control plane where it calls in. The backends refuse the cell’s credentials from anywhere else.

Anti-spoofing outside the host: VPC rules on a cloud; port security or routed ports on bare metal.

Cutting off a node, rotating a cell’s credentials and recovering a cell are in the runbook.

Check the boundary when a cell is built and after each change to it:

Token model

Four credentials in descending scope: the root api_token, the optional operator_token (root’s authority on the claim lifecycle, tenant and node-read routes only, for a public control plane), tenant tokens (isolated at the API layer) and per-sandbox tokens (that sandbox only, nothing node-level); what each may call is in the API reference. The operator token reaches every claim’s data — it reads any claim’s token by id and execs in any claim — and, through the tenant API, every tenant’s authority: it can mint or re-key any tenant and act as it, including volume claims that tenant may make. Its own bearer cannot claim volumes, it holds no node or cluster configuration power, and with audit_log on it leaves an audit record (op token) for each token it reads. A set api_token_next or operator_token_next carries exactly the authority of api_token or operator_token, so a node accepts the old and the new token during a rolling rotation; remove the old one when the rotation ends. The config refuses a node token equal to another one, and every tenant write, and the tenant file at boot, refuse a tenant token equal to any of them. The root, operator and tenant tokens are valid on every node of their cell and nowhere else. A preview URL is a capability on top, HMAC-signed with the cell’s preview_secret and no revocation list. A checkpoint branches only for its own tenant, and a delete does not remove it fleet-wide the instant it runs (why).

Dataset volumes

The volume catalog is an operator-owned data boundary. A volume name and its access list must mean the same thing fleet-wide, although membership is node-local. Entry tenants lists follow the access-list rule. Claim lookup returns byte-identical errors for an unknown and a forbidden name, and catalog discovery filters before replying, so a tenant cannot enumerate restricted entries by probing. Gossip carries names only. Neither gossip, discovery, persisted claims, usage events, nor the sandbox index exposes host image paths or access lists.

Runtime registration and unregistration require a configured root api_token; these endpoints return 403 when the token is unset. New catalog records use mode 0600. Registration uses static configuration’s path validation plus, when set, the volume_roots bound. It does not verify backing type or pin backing identity. Without volume_roots the root token can register any absolute host path, writable if requested, for every scope its access list permits. Protect the root token like host root, or set volume_roots.

Read-only is integrity protection for the shared image, not confidentiality. Mounting a dataset into an egress-lane sandbox gives that sandbox an export path to every destination its tenant egress policy permits; review the volume access list and egress policy together. With directio=off, concurrent readers share the host page cache, which improves reuse but lets one tenant’s large scan evict another’s cached pages — the same holds for a writer: a large rw write can evict cached pages backing another volume’s readers exactly like a large scan would. directio=on is the per-volume mitigation; there is no per-tenant cache quota or accounting. Operators must keep a read-only entry’s attached image immutable and publish a new name/path for new content.

Changing a running claim’s volumes needs the root token or the owning tenant’s API token. A sandbox token cannot, so a guest holding its own sandbox token cannot attach another dataset, and every addition passes the same access list as a claim.

A writable entry (writable: true) adds a channel the read-only model does not have: whichever tenant is permitted to claim it rw changes what every other permitted tenant reads next, and any of them can be the writer in turn — a multi-tenant access list on a writable entry is a bidirectional channel between those tenants, not just a shared read. Recommend a writable entry’s tenants name exactly one tenant — an empty list permits every authenticated scope, which for a writable entry means every tenant can write to every other tenant’s next read. A dataset that genuinely needs multiple writers needs an out-of-band process for who writes when, which the catalog ACL does not provide.

Known limitations

Facts to plan around, stated so the boundary is honest: