A JWT-authenticated FastAPI gateway that eliminated API key exposure across every frontend service — with circuit breaking, encrypted Redis sessions, and full Prometheus observability baked in from day one.
An AI automation agency running multiple client-facing products faced a structural problem: every dashboard, form, and internal tool was making direct API calls to AI providers and data backends. That meant AI provider keys lived in browser-accessible environment variables, and any compromised session meant a compromised key.
Worse, there was no centralized observability. When an AI call failed, the error vanished into the browser console. There was no circuit breaking — a single flapping AI provider endpoint could cascade into a full product outage. And because sessions were stateless, conversational AI features required the client to re-send full conversation history on every request.
The agency needed a single, hardened entry point that all internal services could trust — something that enforced authentication before a single byte reached any backend, encrypted every session at rest, and gave engineering a real-time view of what was happening across every AI call.
The bridge sits between every frontend client and every backend service. No direct connections. No exceptions.
Every security primitive has a cost. Here is the reasoning behind the three decisions that shaped the gateway's security posture.
HS256 uses a shared secret — any service that verifies tokens can also forge them. RS256 uses asymmetric keys: Keycloak signs with its private key, the bridge validates with the public key fetched from Keycloak's JWKS endpoint. No service except Keycloak can ever produce a valid token.
AI providers flap. When Gemini returns 5xx, a naive retry loop hammers the provider and burns tokens. The circuit breaker pattern opens after a configurable failure threshold — returning 503 immediately to clients while the backend recovers, then entering half-open state to probe recovery before fully closing.
Redis stores conversation history per session. If Redis is compromised, plain-text sessions expose full conversation history. AES-256-GCM provides both confidentiality (encryption) and integrity (authentication tag) — any tampering is detected on decrypt. Each session gets a unique IV, preventing nonce reuse.
A FastAPI dependency on every router validates the Bearer token against Keycloak's RS256 public key. Expired, malformed, or missing tokens return 401 before touching any business logic. The public key is fetched once on startup and cached — JWKS rotation forces a restart, which is intentional.
Every backend credential (AI provider keys, database tokens, webhook secrets) lives exclusively in the bridge's Coolify environment. Frontend services send only their JWT and request payload. The bridge resolves credentials server-side for every call. Zero secrets cross the network boundary to the browser.
Gemini's conversational mode requires history. Naive implementations re-send full history on every request — increasing payload size quadratically and exposing history to network interception. The bridge stores encrypted history in Redis keyed to the session ID. Clients send only the latest message; the bridge rehydrates context server-side.
structlog emits structured JSON on every request: method, path, user sub, latency, upstream status, circuit state. This feeds directly into observability dashboards without a log parsing step. Every AI failure produces a traceable structured event with correlation IDs, not a free-text stack dump.
Each route exposes request count, error rate, and latency histograms as Prometheus metrics. The circuit breaker state is also exported as a gauge. This means a Grafana dashboard can alert on elevated error rates or a tripped circuit before users file a support ticket.
Every metric here is a direct consequence of an architectural decision — not a coincidence.
Before the bridge, provider API keys were injected as Vite build-time environment variables — baked into the JavaScript bundle, inspectable by any user who opened DevTools. After: all secrets live in Coolify environment variables on the bridge service, never crossing the network boundary to the browser. A compromised session yields a short-lived JWT, not a provider key.
The circuit breaker means a flapping AI provider endpoint returns 503 immediately after the threshold — rather than hammering the upstream and burning API quota during an outage. Recovery is automatic via half-open probe.
Conversation history grows per message. Sending full history client-side grows payload quadratically. Server-side Redis session storage keeps every client request flat — one message, one response — regardless of conversation depth.
Every route emits latency histograms and error counters. Grafana alerts fire on elevated p95 latency or circuit open state — before users report degraded behavior. Engineering knows before the ticket is filed.
Rotating an AI provider key, a database token, or a webhook secret requires updating one Coolify environment variable and redeploying one service. No frontend redeploy. No rebuild. No downtime for credential hygiene.
I design and ship production-grade AI infrastructure — secure, observable, and built to last.
Let's Work TogetherAG Bridge runs 24/7 as an internal gateway. The /health endpoint surfaces real-time status of Redis and Gemini circuit breaker.