Gateway

Observability

Metrics

GET /metrics renders the Prometheus registry.

Metric Type Labels
gateway_requests_total counter route, status
gateway_request_duration_seconds histogram route
gateway_node_duration_seconds histogram node (pipeline stage)
gateway_tokens_total counter kind (prompt/completion)
gateway_cache_hits_total counter
gateway_ledger_write_failures_total counter
gateway_upstream_connect_retries_total counter account
gateway_upstream_status_retries_total counter account, status
gateway_thinking_signature_review_total counter result (match/mismatch/miss/no_evidence)
gateway_thinking_signature_cache_events_total counter event

gateway_requests_total is recorded by router middleware, so every response — including error statuses and the realtime WebSocket upgrade — is counted, which makes error-rate dashboards possible. All labels are bounded (route templates, status codes, protocol/stage names) — no per-key or per-model cardinality.

Access log

One structured line per successfully served request goes to stdout (via tracing; control level with RUST_LOG), carrying surface, request_id, ak_id (a stable SHA-256 fingerprint, never the bearer credential), product, user_id, model, protocol, account, prompt_tokens, completion_tokens, total_tokens, latency_ms, and decisions — the pipeline’s routing trail for that request (quota fallback, variant pick, degrade, DLP/moderation outcomes). Errored requests are counted by gateway_requests_total{status} rather than logged. request_id joins the access log to the ledger row and the audit events for the same request.

Billing ledger

GET /internal/ledger?limit=N (global admin token) returns the most recent N billing records, oldest-first within the page; count is always the true total, independent of the page size. Records persist when a SQLite or Postgres store is configured and can be capped with storage.ledger_max_rows. Each record carries request_id, the access key, product, tenant, user_id (effective end user), the requested model and the served_model (differs after a quota fallback), protocol, account, token counts, billed_units (characters / seconds / search units on the unit-priced surfaces; also summed in /admin/usage), charged cost_micros and vendor_cost_micros, created_at_epoch_secs, the PTU-spillover flag, and an estimated flag (set when counts came from an aborted stream rather than a vendor usage payload). Writes are idempotent by request_id; a transient store failure is held in a bounded repair queue and retried until the durable backend accepts it. A full queue applies backpressure to settlement until the repair worker makes room, so an accepted billing row is not discarded. Per-user usage additionally rolls into durable minute buckets every minute, so GET /admin/usage/users stays correct after ledger_max_rows pruning (see Governance).

Audit trails

Three operator-facing audit surfaces, all under the gated /admin prefix and covered in Governance: GET /admin/audit/events (content-safety hits, no prompt text), GET /admin/audit/ops (admin-plane mutations with source IP), and GET /admin/usage/users (per-user cost). Content retention, when a tenant enables it, is read back via GET /admin/audit/content/{request_id}.