cocoon sandbox

Guarded egress

A sandbox gets the effect of a credential, never the credential. Outbound access is an allow-listed, audited resource enforced on the host: the secret lives host-side and enters no guest memory, so prompt injection can exfiltrate at most the proxy’s answers, and every credentialed call lands in the audit journal keyed by sandbox and tenant when audit_log is on (and in the usage journal with egress_usage_events). Default-deny.

The same host proxy also serves the none lane (a NIC-less Cloud Hypervisor guest) over vsock, so a network-less sandbox can call approved APIs with injected credentials — no NIC required in the guest.

How it works (none lane)

The guest has no network device. silkd binds 127.0.0.1:3128 and relays each connection to sandboxd over vsock (CID2:2049 → the host UDS <vsock_socket>_2049). sandboxd serves a per-sandbox forward proxy there, bound to that sandbox’s identity by the socket path — no in-band token. The proxy evaluates the policy, injects the matched rule’s secret host-side, dials the origin, and relays the response. An origin that dies mid-response has the guest’s connection cut, so the client sees a truncated body, never a clean end.

The base image sets http_proxy/https_proxy/no_proxy on silkd’s unit, and silkd forwards exactly those variables into every exec — whenever nothing routes directly: the guest has no NIC beyond lo/sit0, or the host has locked the NIC it has (the egress lane below). A lane with a working NIC is never steered into a relay whose host side is closed. An unconfigured client just works, and -x still does:

curl https://api.github.com/user                            # allowed + credentialed
curl https://evil.example/                                  # 403 egress denied: evil.example
curl -x http://127.0.0.1:3128 https://api.github.com/user   # explicit form, same path

Non-HTTP traffic (SOCKS5)

Clients that speak no HTTP proxy for their own protocol — IMAP and SMTP, SSH via ProxyCommand, database drivers, rclone and friends over ALL_PROXY — use the SOCKS5 listener silkd binds at 127.0.0.1:1080 and relays over CID2:2050 (<vsock_socket>_2050). It is the same per-sandbox proxy behind a second door: CONNECT only (BIND and UDP ASSOCIATE answer command not supported), no authentication (the socket path already carries the sandbox’s identity), and a DOMAINNAME destination is resolved host-side, so the guest still needs no resolver. Each door serves at most 256 connections per sandbox at a time; later ones sit unserved in the socket backlog until one closes, and the proxy keeps at most 64 idle upstream connections per sandbox and exit (the node’s own and each upstream proxy the claim has used), as many again for decrypted traffic when the pool intercepts, so a guest cannot turn its proxy into a host descriptor sink. A request on the HTTP door or inside an intercepted tunnel whose headers pass 64 KiB answers 431, and an origin response whose headers pass 256 KiB answers 502, so an open connection holds no more than that each way.

A SOCKS5 tunnel takes the decision an HTTP CONNECT to the same host takes, through the same code: a rule with a nonempty methods list that omits CONNECT denies it, and a rule with a secret opens it without injecting anything, on both ports. The one difference is intercept: on 3128 such a rule terminates the TLS and filters the requests inside, on 1080 there is no HTTP to filter, so the tunnel is refused. An "inject" rule is a plain one on 1080: the door never injects, so it splices whatever credentials the claim holds. The door is opt-in: the pool policy sets "socks5": true, otherwise the host side stays unwired and the guest’s dial is refused like the HTTP one with no policy. The tenant policy’s rules gate every tunnel through the door as they gate CONNECT, so the tenant does not opt in separately; its own socks5 counts only on a claim outside any configured pool, where the tenant policy is the whole policy. A policy that opts in with rules that all carry a nonempty methods list without CONNECT, or all intercept: true, is rejected at load. A pool whose policy does not opt in pays nothing for the door on the claim path, whatever its tenants’ policies say. Audit lines (with audit_log on) carry "method":"SOCKS5".

curl --socks5-hostname 127.0.0.1:1080 imaps://imap.example.com/   # allowed: {"socks5": true, "allow": [{"host": "imap.example.com"}]}

Name resolution (DNS)

A none-lane guest has no resolver of its own. Clients that hand the proxy or the SOCKS5 door a name need none, but dig, getent hosts, tools with their own resolver, and programs that look a host up before they connect by address do. When the pool policy sets "dns": true, the guest’s queries to 127.0.0.153:53 reach a per-sandbox resolver in sandboxd, which asks the node’s nameservers (the node’s /etc/resolv.conf, read again every 5 s) and answers with real addresses. silkd only carries bytes, through two doors:

Behavior of the relay:

A connect by address still goes through the proxy or the SOCKS5 door and is judged there. Hibernate, wake and move need nothing: every UDP query opens a new vsock connection.

dig @127.0.0.153 example.com   # answered: {"dns": true, "allow": [...]}
dig +tcp @127.0.0.153 example.com

How it works (egress lane)

A net:"egress" guest owns a real NIC on the host bridge and reaches the same proxy over the same vsock path. To stop it bypassing the proxy, sandboxd locks the NIC at claim — until then a warm VM of the pool, and the pool’s golden build, sit on the bridge unlocked, running only the image’s own services, never tenant code: an nftables netdev table (sandbox_egress_<tap>) with an ingress hook on the guest’s tap drops every packet the guest sends, so the guest’s only egress is the audited vsock proxy. A locked guest therefore cannot renew a DHCP lease and loses its NIC address when the lease ends, which nothing on this lane needs. The lock is fail-closed — a claim whose NIC cannot be locked is rejected, not handed out unlocked, and no policy still means a locked NIC (default-deny), never a free one. It lives in the host root netns and is removed once the VM is gone (a failed remove keeps an existing lock in place; each reap tick retries the remove). A lock that never applied plus a failed remove leaves the VM unguarded until a later remove succeeds.

Because the lock is invisible from inside the guest, sandboxd writes its verdict into /etc/silkd-lane before the golden snapshot (or on a cold boot): relay on a locked bridge lane, direct on a CNI lane. Where silkd reads relay it hands the proxy variables to every exec exactly as on the none lane, and the image’s units wait for the same file instead of guessing from NIC presence. A golden built under another verdict is rebuilt, not adopted.

Egress-lane sandboxes never archive and do not hibernate, fork, checkpoint, or promote: cocoon resumes a restored or cloned guest inside the call, before sandboxd can re-lock its tap, so any resume from a snapshot would open an unlocked-NIC window. Keeping the lane live holds the lock unbroken from claim to release; the idle sweep skips the lane and those four verbs are refused (409) on it.

The proxy also refuses to connect to internal addresses — every IANA special-purpose range that is not globally reachable (loopback, link-local incl. cloud metadata, private, carrier-grade NAT, benchmarking, documentation, reserved, per the registry snapshot in sandboxd/outbound/dial.go) plus the IPv4-embedding IPv6 forms (NAT64, 6to4, Teredo, IPv4-compatible), and the node’s own public addresses, re-read from its interfaces every 2 s — so an allow-listed host that resolves, or is rebound, to one cannot reach the sandboxd host or a sibling VM. A node address the NIC does not carry (a cloud 1:1 NAT IP) is not seen; block it on the host.

egress_internal_allow re-admits named prefixes through that guard for nodes whose sandboxes legitimately need internal services:

{ "egress_internal_allow": ["10.8.0.1/32:18090,9000", "fdc8::/16"] }

An entry is a CIDR prefix, optionally followed by : and a comma-separated port list; the same form covers both families (fdc8::1/128:443), since the colon after the prefix length cannot be part of the address. A bare prefix admits every port. Prefer the port form: with a bare prefix plus a * rule, a guest reaches every port on the address — the node’s sshd, the kubelet, the Kubernetes API, and anything that listens there later.

It is node-wide (every pool and tenant on the node gets the same re-admission) and checked after the well-known 64:ff9b::/96 unwrap, so an IPv4 embedded in that prefix matches as the IPv4 it is; an address in the local-use 64:ff9b:1::/48 is blocked as a whole and needs that prefix itself in the list. Name service prefixes, never the whole private space: the guest bridges are themselves ULA/RFC1918, so a blanket permit would open sandbox-to-sandbox and the host’s own gateway. 0.0.0.0/0 + ::/0 turns the guard off entirely — for fleets with a policy-enforcing proxy in front — and requests still pass the domain policy first; the allow-list widens the IP gate only.

Deployment constraints

Configuration

A config reload applies policy, internal allow, secret and upstream changes without a restart: a live claim’s next request or connection is evaluated against the new policy. An open tunnel keeps running, and a packet flow of the advanced network mode keeps its first packet’s verdict until it ends or idles out (an hour for an open TCP flow). Whether a claim has the SOCKS5 and DNS doors is settled when it is claimed, and a claim armed with no policy has no door at all until it is armed again, at a wake, a move or a restart.

Policy is per pool and per egress class; a tenant claim’s effective policy is the intersection of its pool’s and its tenant’s class (a request must pass both). The pool rule’s secret is the only one injected on a composed claim; a class rule’s secret applies only to a tenant claim of an unpooled key, where the class policy stands alone. egress_classes names each tenant layer in the config file, and a tenant references one by egress_class through the tenant API. The API only references a class the operator wrote, never defines one, so a tenant added at runtime gets egress without a config edit. A pool with no policy, or a tenant with no class, is an empty allow-list, not a pass (a key in no pool has no pool layer at all, below): a tenant without a class reaches nothing through the proxy (the direct NIC of a CNI egress lane is outside it), so granting a tenant egress — and the secret injection that rides it — is always an explicit act. A claim takes its tenant’s class when it is made and keeps it for life: a class’s policy follows a reload, and a tenant moved to another class takes it on its next claim. Root claims have no tenant layer and take the pool’s policy whole. A clone of a promoted template takes the policy of the pool its template was promoted from, as if it were that pool’s claim: the template records its source pool, a re-promote records the new one, and a fork, checkpoint or branch of such a clone keeps it. So a clone of a template promoted from a pool with no policy has no egress, as that pool’s claims have none, even for a tenant with a class; a template promoted from a sandbox with no pool layer records no source. A pool’s egress block belongs to the config file and stays bound to its key: when a PUT /v1/pools drops a config pool that has one, the claims made after it, now cold-booted on demand, still take that policy. Any other key that is in no pool at claim time — an image cold-booted on demand — has no pool layer, so a tenant’s claim of one takes its class’s policy alone (a root claim of one stays denied). A claim with "egress": false takes no policy at all, whatever its pool’s: no door is bound, so the guest’s dial is refused. Which layers a claim has is settled when it is made and kept for its life: a PUT /v1/pools that adds or drops a pool without a config-file policy changes the claims made after it, never a live one. A reload is the one exception: a config-file policy it adds for the key of a live tenant claim made without a pool layer applies to that claim, and removing it again restores the class alone. Secrets are registered separately and referenced by name — the value comes from the environment, never the config file: a claim’s own guest: false env first, the node’s second (below).

{
  "bridges": ["sbxbr0"],                    // egress-lane pools only; none-lane needs no attachment
  "secrets": [
    { "name": "gh", "header": "Authorization", "value_env": "GH_TOKEN" }
  ],
  "egress_ca": {                            // provisioned below
    "root_cert": "/etc/sandboxd/egress-ca/root.crt",
    "intermediate_cert": "/etc/sandboxd/egress-ca/node-a.crt",
    "intermediate_key": "/etc/sandboxd/egress-ca/node-a.key"
  },
  "pools": [
    { "template": "rt:24.04", "net": "none", "size": "small", "warm": 2,
      "egress": { "socks5": true, "allow": [
        { "host": "api.github.com", "methods": ["GET", "POST"], "secret": "gh", "intercept": true },
        { "host": "*.googleapis.com" }
      ] } },
    { "template": "rt:24.04", "net": "egress", "size": "small", "warm": 2,
      "egress": { "allow": [{ "host": "api.github.com", "secret": "gh", "intercept": true }] } }
  ],
  "egress_classes": [
    { "name": "desk", "egress": { "allow": [{ "host": "api.github.com" }] } }
  ]
}

A tenant joins the desk class with PUT /v1/tenants/acme {"token": "…", "egress_class": "desk"}.

Both pools serve the proxy the same way, and the first also opens the SOCKS5 door; the egress-lane pool additionally gets its NIC locked at claim, so its policy governs the only route out just like the none lane’s. sandboxd binds a warm clone’s doors when it joins the pool and starts serving them at claim, so the bind is off the claim path (a refill-time bind failure is logged and that clone binds at claim); a door the guest reached while the clone was warm is bound afresh at claim, which resets what waited in it, and an unpooled or woken sandbox binds at arm time.

When audit_log is on, each decision is written to audit.jsonl (op:"egress", dest, port, method, decision, and the secret name in secret). With egress_usage_events on (default off), it is also metered as an egress usage event, which names the injected secrets in secret as the audit line does. An intercepted CONNECT is recorded as a decision of its own before its inner requests. The HTTP and CONNECT doors deny a host longer than 253 bytes (the DNS limit); the audit record clips it. They also deny a host with a zone (%) or a non-ASCII byte, which the dialer would resolve to another target than the rule matched; the SOCKS5 door denies such a name the same way, and every door records the deny. A sandbox records at most 10 deny and rate_limited decisions per second after a burst of 100, dropping the rest, so a guest cannot flood the journals. rate_limited comes from egress_conn_rate.

Claim env and secrets

A secret’s value is an environment variable: the one its value_env names, or the secret’s own name when value_env is empty. The proxy reads it from the claim’s env first, from its guest: false entries only, and from the node’s environment second:

{ "secrets": [
    { "name": "gh", "header": "Authorization", "value_env": "GH_TOKEN" }, // a claim's GH_TOKEN, else the node's
    { "name": "GW_KEY", "header": "Authorization" }                       // only a claim's GW_KEY
] }

So each sandbox can carry its own credential for the same rule, such as one gateway key per user, while the guest holds a placeholder.

The proxy reads a snapshot of the claim’s env that each env write publishes, without a lock, once per intercepted request, once per request whose matched rule names a secret, and once per CONNECT an "inject" rule covers; other requests pay nothing.

Claim credentials

A guest: false env entry can also name where its value goes, so a control plane binds each sandbox’s own credential to a host without editing the node config:

{ "env": {
    "API_TOKEN": { "value": "Bearer …", "guest": false,
                   "inject": { "hosts": ["api.example.com", "*.git.example.com"], "header": "Authorization" } }
} }

On an intercepted request to one of hosts, the proxy sets header to the entry’s value, overwriting whatever the guest sent. With placeholder set, it does so only on requests whose header carries exactly that value and leaves every other request to those hosts untouched, so a tool opts in per request while a browser on the same host gets nothing:

"GH_GIT": { "value": "Basic …", "guest": false,
            "inject": { "hosts": ["github.com"], "header": "Authorization", "placeholder": "Basic @GH_GIT@" } }
git config --global http.https://github.com/.extraHeader "Authorization: Basic @GH_GIT@"

A key that a client sends as a query parameter takes query instead of header, and a placeholder is required. On an intercepted request, the first <query>=<placeholder> pair of the URL gets the value, URL-escaped; the placeholder may arrive percent-encoded, and every other byte of the query stays as sent. With body: true and no placeholder in the URL, the first such pair of an application/x-www-form-urlencoded body, or the first top-level string member of that name holding the placeholder in an application/json body, is filled instead, when the body has a known length of at most 64 KiB; any other body streams unchanged. One fill per credential bounds what a request can grow by:

"FB": { "value": "EAAB…", "guest": false,
        "inject": { "hosts": ["graph.facebook.com"], "query": "access_token", "body": true, "placeholder": "@FB@" } }
curl "https://graph.facebook.com/me?access_token=@FB@"
curl https://graph.facebook.com/me/feed -d access_token=@FB@ -d message=hi

Pair the credential with an "inject" pool rule, for example { "host": "*", "intercept": "inject" }, and only the claims that hold a credential for a host have it intercepted: the operator decides which hosts may be intercepted, the claim’s entries decide which of them are. The decision is taken per CONNECT, so a write that adds or removes an entry applies to the guest’s next connection; a tunnel the guest keeps open stays as it was.

The placeholder is not a secret: it stays in the guest and in GET /env. The entry rides the claim request or any /env write, and rotation is the same PATCH, accepted on a hibernated or archived claim without waking it. With archive_failover the write also replaces the row of an archived claim, or of a hibernated claim with a replica, so an adopter starts from the rotated value; a running replicated claim’s row takes it at the next replication round. While the meta store is down the write answers 503 with the claim’s own record already changed: resend it, and the resend replaces the row.

Upstream proxies

A claim can leave through its own upstream proxy instead of the node’s address: a residential or regional exit for one sandbox, the node’s own exit for the next, freely mixed on one pool. The guest sees nothing of it — every sandbox still talks to its local proxy doors; the choice lives host-side.

{ "egress_upstream": { "claim_env": "EGRESS_UPSTREAM", "allow": ["res.example.com", "10.1.0.0/16"] } }

A claim names its upstream in that env entry, which must be guest: false (it carries the upstream’s credentials): http://[user:pass@]host:port (an HTTP CONNECT tunnel for every destination, plain HTTP included) or socks5://[user:pass@]host:port. direct forces the node’s own exit, and an absent entry falls back to the claim’s egress class, then the pool’s default: egress_upstream_env on an egress class or pool entry names a node environment variable holding the URL, read at every config load, so no credential sits in the config file. A claim (a checkpoint branch included), PUT or PATCH naming an upstream outside allow (exact host names, IPs or CIDR prefixes), a malformed URL or a guest-visible entry answers

  1. PATCH /v1/sandboxes/{id}/env switches the upstream for new connections; open tunnels keep their path, and pooled keep-alive connections are never reused across a switch.

Policy order does not change: the domain rules, methods, secret injection and interception all run on the destination the guest asked for. The proxy then resolves that destination itself and applies the internal-address guard to every address: a blocked one fails before the upstream is contacted, an address egress_internal_allow re-admits dials direct (a node-local service an external exit cannot reach), and otherwise the tunnel is opened to the vetted IP, so the upstream never resolves a name the node did not check. The first IPv4 address is preferred, since many upstream proxies have no IPv6 exit. The upstream’s own address is dialed with the node’s plain dialer. An upstream that fails answers the guest 502 (SOCKS5: host unreachable) and never falls back to a direct dial. An allowed connection’s audit record, and its usage records when they are on, name the upstream’s host:port, never its credentials, and GET env blanks the value like every guest: false entry. Cost: a node without egress_upstream pays nothing; with it, each allowed request reads the claim’s env snapshot, without a lock.

Byte usage

egress_usage_bytes: true adds one egress_bytes usage event per allowed tunnel or request when it ends, with the payload bytes it moved (tx guest to destination, rx back) and its upstream, so per-exit traffic can be billed. The counts come from the copies the proxy already runs; the cost is one more usage-journal append per connection, which is why it is opt-in. A tunnel that a sandboxd stop cuts records no egress_bytes event. A flow of the advanced network mode records one too, with its IP bytes and upstream tun/<label>.

egress_usage_interval_seconds (default 0, needs egress_usage_bytes) also writes an egress_bytes event every interval for each allowed tunnel, request or tun flow that lasts longer, with the bytes since its previous event, and one with the rest, if any, when it ends. Sums per claim and per upstream match the single events an interval of 0 writes, so a control plane can apply a window rule to a long upload before it ends. A spliced tunnel counts its bytes when its copy returns: each interval the proxy sets a past read deadline on every open tunnel, so the copy returns, counts and resumes, and an event carries the bytes up to the previous interval. A sandboxd stop therefore loses up to two intervals of a spliced tunnel’s bytes and up to one of a limited tunnel or a request. Tun flows are checked every 10 s, so their events follow that grid; a TCP flow that closed and stayed quiet for 10 s writes its last bytes at the next check, while its entry stays two minutes for late segments.

Bandwidth limit

PUT /v1/sandboxes/{id}/egress-limit caps one claim’s bytes per second, tx_bytes_per_second (guest to destination) and rx_bytes_per_second apart; zero leaves a direction unlimited. The control plane sets every cap, for example from the egress_bytes events above; a claim starts without one. The cap covers HTTP CONNECT, SOCKS5, plain HTTP forwarding and intercepted requests, and counts payload bytes as egress_bytes does. Preview, port relay and exec traffic is not capped.

Each direction has one token bucket per claim, shared by all its connections, with a burst of the rate, at least 256 KiB and at most 1 GiB. A limited direction copies in user space, in reads of about a tenth of a second’s bytes (1 KiB to 256 KiB on a tunnel, at most 32 KiB on a request), each followed by a wait for its tokens, so its CPU cost follows the bytes the cap lets through. A limited tunnel direction holds a 32 KiB buffer, and a claim lends at most eight 256 KiB ones at a time to the busiest, so a claim’s open tunnels hold at most 512 × 2 × 32 KiB plus 2 MiB, about 34 MiB, idle or not. An unlimited direction keeps the kernel splice; a claim without a cap pays one map update per tunnel and an atomic load per request read. A changed rate starts with a full bucket, a direction whose rate did not change keeps its bucket, and a repeated cap changes nothing. A change reaches every open connection at once: the proxy sets a past read deadline on every open tunnel, whose copy resumes in the matching mode; an uncapped direction can pass one more splice, up to 1 MiB at the old, uncapped rate, when its data arrives together with the deadline, and a request reads the cap on every read. A claim in advanced network mode keeps separate buckets on its tun path, so its proxy and tun traffic are capped apart.

A limited download paces what the proxy reads, not what the far end sends: the upstream can run ahead by up to the node’s TCP receive buffer per connection (net.ipv4.tcp_rmem max), as it can today against a guest that reads slowly, and then sees its window close. The guest still receives at the cap, and egress_bytes counts only the bytes the proxy read.

HTTPS interception

A rule with intercept: true, or "inject" for a claim with a credential for the host, makes the proxy terminate that host’s TLS with a leaf it signs, so it sees the request’s method and path and can inject the secret into HTTPS — the same guarantees plaintext already has. Upstream is re-originated and verified against the host’s real root store; the proxy never trusts an unverified origin. A guest that caches TLS sessions resumes them on its next CONNECT through the same sandbox, which skips the leaf signature and chain check. A response that is an event stream or has no length reaches the guest as each chunk arrives, on intercepted and forward requests alike.

Trust is a two-tier PKI. One cell root CA is the trust anchor: its certificate (public) is baked into every interception guest’s store, so a leaf from any node validates. Each node signs leaves with its own intermediate CA, issued from the root, and presents [leaf, intermediate]; the guest builds leaf → intermediate → cell root. The root private key never reaches a node — a node holds only its intermediate.

Provisioning the cell PKI

Run the sandboxd ca tool on an operator machine, not a node — the root key never touches a worker. Mint the root once, then issue one intermediate per node:

# 1. Once per cell. root.key stays here, offline, forever.
sandboxd ca init -out ca -cn "acme sandbox root"
#   → ca/root.crt (public)   ca/root.key (0600, never leaves this host)

# 2. One intermediate per node, all signed by the same root.
for n in node-a node-b node-c; do
  sandboxd ca issue-intermediate \
    -root-cert ca/root.crt -root-key ca/root.key \
    -node "$n" -out "dist/$n"
done
#   dist/node-a/node-a.crt  dist/node-a/node-a.key   (0600)
#   dist/node-b/…  dist/node-c/…

Both subcommands refuse to overwrite an existing .crt/.key; pass -force to replace one deliberately.

That gives one root per cell and a per-node signing key:

ca/root.crt          → copied to every node of the cell (public trust anchor)
ca/root.key          → stays offline on the operator machine
dist/node-a/node-a.* → node-a only
dist/node-b/node-b.* → node-b only
dist/node-c/node-c.* → node-c only

Distribute to each node — the root cert plus that node’s intermediate, never another node’s key, never root.key:

scp ca/root.crt dist/node-a/node-a.crt dist/node-a/node-a.key \
    node-a:/etc/sandboxd/egress-ca/

then point that node’s config at the root cert and its own intermediate:

"egress_ca": {
  "root_cert":         "/etc/sandboxd/egress-ca/root.crt",
  "intermediate_cert": "/etc/sandboxd/egress-ca/node-a.crt",
  "intermediate_key":  "/etc/sandboxd/egress-ca/node-a.key"
}

openssl verify -CAfile ca/root.crt dist/node-a/node-a.crt confirms the chain. Adding a node later is step 2 for the new node only — the root is untouched, so existing guests keep validating. Losing a node’s intermediate key compromises only that node: re-issue it (step 2) and roll the node; the cell root, and every other node, is unaffected. Give each cell its own root, so a cell’s guests never trust another cell’s intermediates.

egress_ca is required whenever a pool has an intercept rule. The root cert is baked into a guest when the guest is created — at golden build, or at a pre-golden cold claim’s provision (both via silkd; only the golden-build install is off the claim path). It is not re-installed on re-claim: a clone, checkpoint restore, archive wake, or reconcile adopts the guest with whatever root it was born with. Each golden’s .stamp file records what it baked (the root fingerprint, the lane verdict, the image id, a sized pool’s disk size, no_balloon when set, and the warmup argv), and a golden is adopted only when its stamp exists and matches, so a changed root rebuilds goldens. Nothing rebuilds an existing checkpoint, archive, promoted template, or live/hibernated claim. Because the baked cert is the cell root, promote/checkpoint/archive carry no node-private material and stay unrestricted.

Intermediate rotation is seamless. Issue a fresh intermediate from the same root, point the node’s config at it, restart: leaves still chain to the root every guest already trusts, and the root fingerprint in each golden’s .stamp is unchanged, so no golden rebuilds.

Root rotation needs a drain, not a hot swap. A guest verifies leaves only against the root(s) it was born with, so the node’s leaves must chain to a root every live guest trusts. root_cert may bundle several CA certs (every block must be a CA cert; the fingerprint covers the exact file bytes, so any edit rebuilds goldens), and LoadCA validates the node’s intermediate against the first cert in the bundle. That fixes the order:

  1. Bundle old root, new root (old first) and keep the old intermediate. Rebuilt goldens bake both roots, so new guests trust old and new while the node still signs under the old root that every live guest trusts.
  2. Recreate or drain every old-root guest — checkpoints, archives, promoted templates, and live/hibernated claims.
  3. Atomically swap to new root first and the new intermediate.

Swapping the intermediate while old-root guests are alive breaks interception for them, with no re-claim path to fix it.

Limitations: interception is HTTP/1.1 only and breaks clients that pin certificates, so scope it to hosts you control, or use "inject" so only the claims that inject on a host are affected. A host that speaks a non-HTTP protocol over TLS must not be given an intercept rule.