Deployment

Container image

ghcr.io/soulwhisper/mcp-guardrails:<version> (current: 0.4.0). Multi-stage build, runs as nonroot (uid 65532), pre-bakes the PromptGuard-2 ONNX model pinned via the PG2_REVISION build-arg. Version tags (x.y.z) are immutable; latest and x.y float.

Kubernetes manifests

deploy/k8s/ contains a ready-made set:

Key points:

• appProtocol: kubernetes.io/h2c on the Service port is what tells agentgateway this is plaintext-HTTP/2 gRPC; the sidecar binds an insecure port (no TLS) because the gateway handles transport security. Do not add TLS to the sidecar unless you also switch the Service appProtocol to kubernetes.io/grpc and provide a certificate.
• ConfigMap rule pack: mount at /etc/guardrails/rules.policy and set INVARIANT_RULES_PATH to match. Reload without restarting the Pod with kill -HUP 1 inside the container.
• failureMode: FailClosed in the policy: the only safe default for write-capable agents. If the sidecar Pod is unreachable, agentgateway returns JSON-RPC -32001 rather than forwarding unguarded.
• Readiness probe: the grpc.health.v1 check goes SERVING only after engine warmup (PromptGuard-2 model load), keeping the Pod out of the Service endpoints during cold start.
• Resources: a 2Gi memory budget covers onnxruntime CPU + the ~350MB PromptGuard-2 weights. Bump when enabling ENABLE_AGENT_ALIGNMENT=1.

Rollout and drain

Rolling updates and HPA scale-downs are aligned with the sidecar's shutdown sequence (R-5). On pod deletion, in order:

1. preStop sleep (10s). Kubernetes marks the pod for deletion and removes it from Service endpoints immediately, but kubelet holds the SIGTERM while the preStop hook sleeps. Endpoint removal propagates (kubelet → endpoints controller → agentgateway dataplane) during this window, so new exchanges stop arriving before the process even begins shutting down.
2. SIGTERM. The health service flips to NOT_SERVING first (the health watchdog is cancelled so it cannot flip back mid-drain) — a second propagation barrier for anything that missed step 1.
3. Drain wait (SHUTDOWN_DRAIN_S, default 5s). The sidecar waits for readiness propagation; in-flight exchanges keep running.
4. gRPC drain (grace 5s) + exit. server.stop(grace) lets in-flight RPCs complete, then the process exits.

terminationGracePeriodSeconds: 30 budgets preStop (10s) + drain wait (5s) + gRPC grace (5s) = 20s of scripted shutdown, plus 10s headroom for slow in-flight exchanges inside the drain grace. It must exceed preStop + SHUTDOWN_DRAIN_S — if kubelet's grace expires first it sends SIGKILL mid-drain, truncating in-flight exchanges and their audit lines. If you raise SHUTDOWN_DRAIN_S or the preStop sleep, raise the grace period to match.

The POD_NAME env (via fieldRef: metadata.name) is set so audit replica attribution (AUDIT_REPLICA_ID, default $POD_NAME) works out of the box — see Compliance §2.

Remaining validation step: a live rolling-update soak against a real cluster (watch endpoint churn, drain timing and audit-chain continuity across several rollouts under load) has not yet been run — the sequence above is derived from the shutdown implementation and probe timings, not soak-tested.

agentgateway wiring

Two supported shapes:

1. CRD (Kubernetes) — deploy/k8s/agentgateway-policy.yaml maps:
2. tools/call → Full (request + response double gate — the indirect-injection frontline lives on the response side);
3. tools/list, prompts/get, resources/read → Response;
4. ping / initialize → None.
5. Standalone — examples/agentgateway.standalone.yaml points a non-K8s agentgateway at localhost:9001 (the binds / mcpGuardrails schema; @PLACEHOLDER@ tokens are substituted by the e2e script).

Multi-replica guidance

Two sidecar subsystems are per-process by default:

• Invariant traces default to in-memory per replica (INVARIANT_STATE_BACKEND=memory). Have agentgateway inject a session header (e.g. x-session-id), set INVARIANT_TRACE_KEY_HEADERS to match, and configure sticky/session-affinity routing on the gateway side — sticky routing itself is the deployer's responsibility. Without it, a toxic flow whose calls land on different replicas cannot be detected. The stronger option is INVARIANT_STATE_BACKEND=redis with REDIS_URL pointing at a shared Redis: loop/rate/aggregate rules then evaluate the fleet-wide trace regardless of which replica a call lands on (appends are linearised by Redis; detection may lag one call under concurrent replicas; the sidecar fails closed at startup if Redis is unreachable). Sticky ToxicFlow prefix-progress remains per-replica in both modes.
• The audit hash chain is per-process: each replica maintains an independent chain. Do not point multiple replicas at one shared AUDIT_LOG_PATH file — use per-replica files or stdout shipping (see Auditing). Every chained line carries a replica field (from AUDIT_REPLICA_ID, default $POD_NAME via the fieldRef in deployment.yaml) inside the hashed payload, so exported per-replica chains stay attributable — see Compliance §2.

Configuration checklist

A complete multi-replica setup combines:

| Concern | Setting | Notes | | Shared trace state | INVARIANT_STATE_BACKEND=redis, REDIS_URL=redis://<host>:6379/0 | Point at your own Redis (none is shipped in deploy/k8s — treat it as infrastructure, like the gateway). The client is baked into the image; the sidecar fails closed at startup if unreachable. Optional — sticky routing alone is sufficient for single-tenant fleets. | | Replica count | deploy/k8s/deployment.yaml replicas + hpa.yaml | HPA and PDB are already shipped; keep minAvailable aligned with replica floor. | | Trace-key isolation | INVARIANT_TRACE_KEY_HEADERS=x-session-id (+ gateway injects the header) | Per-session traces; applies in both memory and redis modes. | | Sticky routing | agentgateway session affinity (e.g. CEL-derived key, v1.5.0+) | Mitigation for memory mode; unnecessary for correctness in redis mode, still useful for cache locality. | | Audit attribution | AUDIT_REPLICA_ID (default $POD_NAME, already wired via fieldRef) | Per-replica hash chains; export each replica's chain separately, never share one AUDIT_LOG_PATH file. | | Metrics scraping | PROMETHEUS_LISTEN_ADDR=:9464 + Prometheus scrape config per pod | Pull endpoint coexists with OTLP push; scrape per replica, label by POD_NAME. | | Telemetry TLS | OTEL_EXPORTER_OTLP_INSECURE=false (or an https:// OTLP endpoint) | Required once the collector crosses a network boundary. | | Safe rollouts | preStop 10s + terminationGracePeriodSeconds: 30 (already in deployment.yaml) | See Rollout and drain above. |

Network isolation

The sidecar authenticates nothing on :9001. Restrict access to agentgateway with a NetworkPolicy (or mesh mTLS) before treating audit verdicts as authoritative — see the security model.