Gateway

Security model

What the gateway trusts, what it checks on every request, what it records, and what an operator must put around it. Every statement here names the config or code path that enforces it; the review that produced this page and its findings are summarized at the end.

Trust boundaries

Party	Holds	May
API customer	an access key (access_keys[].ak, or one created through the admin API)	call the model, MCP and batch surfaces its key is entitled to, within its quotas
Tenant admin	the tenant’s admin_token_env bearer	read and change its own tenant’s keys, usage, content-safety events; nothing global
Operator	the global admin.token_env bearer, the process environment, the config source	everything: config publish and reload, every tenant, /internal/*, the upstream credentials
Upstream vendor / MCP server	its own endpoint	is trusted only to answer; its errors are mapped, its headers are filtered, its credentials never leave the process

The gateway never holds a customer’s vendor credentials: every upstream key, Bedrock token, MCP bearer and OAuth client secret is read from the environment variable the config names, at call time, and appears in no response, log line, audit row or trace.

Authentication

• Access keys ride as Authorization: Bearer <ak> or x-api-key: <ak>; a missing or unknown key is a 401. Keys have a lifecycle: expires_at_epoch_secs, banned, and an automatic abuse suspension (abuse.tiers) each fail authentication with a distinct 403, on every surface (REST, realtime — where the key is re-checked per turn and a turn denied for the key’s status ends the session — batches, where a running batch re-checks the key before every item and fails once it is no longer active — and MCP).
• A raw access key enters only through request authentication and the operator’s own inputs (the config document, admin calls). Everything the gateway stores or reports names the key by its ak_id (sha256: plus the first 16 bytes of the key’s SHA-256 in hex): the key table, the ledger, security-event, retained-content and audit rows, governance counters, alert subjects, key listings, logs and traces. The config document is kept and returned as written, so declare its keys by ak_id. An ak_id sent as a key never authenticates. The digest is unsalted, so it protects a high-entropy key only: let POST /admin/keys generate the key (omit ak). Rows written by a release that stored raw keys keep them until the operator rewrites or prunes them.
• Admin routes are absent (404) until an admin token is configured. Two tiers apply: the global token (admin.token_env) manages everything; a tenant’s admin_token_env token manages only that tenant. A tenant token on a global-only route (reload, config publish, /internal/*, the cross-tenant /admin/audit/ops trail) is a 403; a tenant token looking up a foreign key gets an empty page or a 404, never a confirmation the key exists.
• The web control plane keeps its own identity store and browser sessions and reaches the gateway only through the admin API with the tokens above.

Authorization and tenant scoping

• A key belongs to one tenant (the implicit default tenant when undeclared). Model entitlement (tenants[].models) is enforced before the cache and the engine, so a tenant cannot be served another tenant’s cached answer or reach a model it is not entitled to — including through a fallback chain, where unentitled entries are skipped.
• Files, batches and retained content are tenant-owned: a foreign id answers 404, not 403, so ids cannot be probed.
• MCP: a key reaches only the servers named in its mcp_servers entitlement (any other name answers 404), and only the tools in its mcp_tools allowlist; tools/list is filtered to the allowlist and a tools/call outside it is refused inside the gateway (JSON-RPC -32000). An MCP session id is bound to the key that first received it. Keys created through the admin API carry no MCP entitlement. Only a fixed set of MCP headers crosses the proxy in either direction; the caller’s Authorization never reaches the server and the server’s headers never reach the caller beyond Content-Type and Mcp-Session-Id. The proxy’s HTTP client follows no redirects, so a server or token endpoint cannot steer a credentialed request elsewhere. JSON-RPC batches are refused.

Content controls

• Per-tenant security policy: blocklist (block / flag / shadow), regex recognizers, secret detection, DLP redaction of emails and phone numbers in both directions (a redacted stream is buffered and replayed so no unmasked span leaves), and an external moderator (moderation:, AWS Bedrock Guardrails) over inbound text and — for the MCP proxy — over tool results (tools/call, resources/read, prompts/get), reviewed whatever the reply’s HTTP status and including a JSON-RPC error’s message; a tenant under review cannot open an MCP listen stream, whose server-pushed content the proxy cannot review. Signed thinking blocks are never rewritten; a mask that would land in one fails the request instead.
• A tenant admin may ban its own keys but can neither lift a ban nor change an abuse suspension: those are platform sanctions the global token owns.
• Automatic suspension (abuse.tiers) counts only rejections for the key’s own limits, so a sibling’s traffic on a pooled limit cannot suspend it (Governance).
• Every hit is a security event (/admin/audit/events) carrying the rule, action and hit count — never the prompt text. Admin mutations land in /admin/audit/ops with the source IP (the TCP peer; the rightmost x-forwarded-for hop only under trust_proxy_headers). Retained content (tenants[].retention) is sealed at rest with GW_CONTENT_KEY; without the key, full retention degrades to redacted text.

Failure postures

Stated so an operator can choose them deliberately:

• Redis unreachable: rate limits, quotas and budgets fail open and a warning is logged; account health treats every account as healthy.
• Moderator unreachable: security.moderation_fail_open decides — false (default) denies the request or tool result, true admits it.
• Upstream failure: account failover within the model, then the model’s fallback_models chain, only while nothing has been sent to the client. A terminal model failure then answers 424 with model_error_exception (see API); an MCP server or identity-provider failure answers a generic 500, and its endpoint and error text stay in the gateway log.
• Abuse of long-lived connections: realtime sessions and MCP listen streams are capped per key (max_live_streams_per_key); realtime turns and every MCP request spend the key’s QPS permits; request bodies are capped at listen.max_request_bytes (32 MiB by default); replies the proxy must buffer are capped per server (max_reply_bytes); x-gw-user is capped at 256 bytes, since it keys governance counters.
• Billing store unreachable: rows queue in a bounded repair queue that applies backpressure when full and retries each batch eight times with backoff (about 13 s); a batch the store still refuses after that is dropped and counted in gateway_ledger_write_failures_total.

Operating recommendations

• Terminate TLS in front of the gateway; it listens on plain HTTP.
• Keep /admin/*, /internal/* and /metrics off the public load balancer (the sample nginx config in multi-instance does this) and leave GW_ADMIN_TOKEN unset on instances that do not need the admin API.
• Put MCP servers on a private network the gateway alone can reach; the gateway is their only client and holds their credentials.
• Give each customer their own key with qps, daily_token_quota, tokens_per_minute and a cost budget, and route alerts (alerts.webhook_url_env) somewhere watched.
• Rotate upstream and MCP credentials by changing the environment and reloading; OAuth tokens the gateway fetched are dropped on reload.

Review record

A targeted adversarial review of the admin plane and the MCP proxy ran on 2026-09-09 against three lenses: authentication and tenant scoping; injection, smuggling and protocol edge cases; resource exhaustion and information leakage. Fixed in the same change: unbounded buffering of MCP replies, redirect following by the MCP client (credential replay and SSRF), upstream endpoints and identity-provider error text in customer-visible errors, resources/read and prompts/get results escaping review, multi-line SSE data and other unparsable replies passing through unreviewed, structured tool output escaping a mask, stream resumption replaying results past the review, unbound MCP session ids, tool calls audited only after the upstream answered, a tools/call without a string tool name skipping every gate, realtime sessions and listen streams with no per-key cap, in-process monthly counters keyed by x-gw-user never evicted, a tenant token lifting bans and suspensions — through the key-create route as well as PATCH — raw access keys in denial messages, admin listings without a page ceiling, and MCP server names probe-able through 404-vs-403. A second pass over the same surface fixed: results escaping review on a non-2xx status or when carried as a JSON-RPC error; prose in structured tool output under identifier-named keys, while base64 image and blob payloads are left untouched; the attribution hint uncapped on the realtime subprotocol path and in request and batch bodies; the whole fallback chain disabled for any reasoning request rather than only one replaying reasoning output; a listen stream handing unreviewed server-pushed content to a tenant under review; a zero max_reply_bytes refusing every reply and a zero token lifetime defeating the token cache; and one server’s slow token endpoint stalling another server’s first token fetch.

Accepted as is, with the reason: the listener has no header-read or idle timeout of its own (a proxy in front terminates slow clients, which the deployment guidance requires anyway); a video download is held in memory for the clip’s size; a config reload can race one in-flight token fetch for at most one token lifetime; duplicate JSON keys parse last-wins here and possibly first-wins on a non-compliant upstream (a compliant server behaves identically); the cross-tenant key guard reads the instance’s key cache before mutating (with Postgres a key write invalidates it fleet-wide by NOTIFY within about a second), which would need a key to change tenant inside that window; MCP session binding is enforced per instance, so a multi-instance deployment must route a session’s requests sticky by Mcp-Session-Id (the id-less initialize may land on another instance, where ownership binds only from a reply that carries the id, so a second key holding the id could use the session there until one does); a server that returns one constant session id locks its session to the first key to use it (a conformant server mints a unique id per session).

This site is open source. Improve this page.