Gateway

Single-binary LLM access point in Rust (binary: gw): OpenAI- and Anthropic-compatible APIs in front of pluggable model providers, with key-based auth, quotas, rate limits, failover, and a billing ledger.

client ──► /v1/* (OpenAI + Anthropic surfaces, streaming SSE, realtime WS)
       ──► pipeline: resolve/quota/cache → account select (PTU, failover)
                     → rate limits → engine → usage → billing ledger
       ──► providers: real endpoints over HTTP · in-process mock for the rest

Guides

• Examples — copy-paste curl for every surface, going live
• API reference — every endpoint, auth, streaming, batch/files
• Providers — presets, going live, failover, SigV4
• Governance — access keys, rate limits, quotas, cache, safety
• Observability — /metrics, access logs, the ledger
• Deployment — binary, Docker, env vars, multi-replica
• Running a fleet — nginx LB, shared state, affinity
• Configuration — the full gateway.yaml reference
• Architecture — crate layout, pipeline, trait seams
• Development — build, test, workspace map, contributing

Repository

Source and issue tracker: github.com/cocoonstack/gateway.

This site is open source. Improve this page.