Integration &
Deployment Guide
Complete deployment playbook covering the nine-stage filter pipeline, Agentic SOC (SOVA + MasterAgent), Post-Quantum Cryptography, Shadow AI Governance, Explainable AI 2.0, Sovereign Cloud routing, SEP Communities, Semantic Layer (Headless BI), Settings Hub, API integration, compliance, and high-availability configuration.
Executive Summary & Business Case
Shadow Warden AI is a zero-trust AI security gateway deployed inline between your application and every LLM call. It filters requests through a nine-stage pipeline, catches 36.2% of prompts in our own published adversarial corpus at HIGH or BLOCK with zero false positives on 35 benign prompts, strips secrets and PII before they reach third-party models, and self-improves through an automated Evolution Engine — all without sending sensitive data to external services.
Filter Pipeline Architecture
Every request passes through a deterministic, ordered pipeline. Each layer fails open on non-fatal errors except the auth guard (fail-closed). Latency per layer is tracked via OpenTelemetry spans.
Client Segment Playbooks
Six deployment profiles map organization type to recommended tier, configuration preset, and compliance posture.
- › PQC signing on all AI transactions
- › Sovereign routing — PHI stays US/EU only
- › SOC 2 Type II evidence vault
- › GDPR Art. 35 DPIA pre-filled
- › PHI data pod jurisdiction lock
- › HIPAA audit chain (STIX 2.1)
- › Secret redactor covers HL7/FHIR tokens
- › Causal Transfer Guard blocks exfiltration
- › Per-tenant API keys + rate limits
- › MasterAgent SOC coordinator
- › Shadow AI Discovery for rogue models
- › XAI audit reports per request
- › MASQUE H3 sovereign tunnels
- › 8-jurisdiction transfer matrix
- › Post-quantum key encapsulation (ML-KEM-768)
- › Air-gapped Evolution Engine mode
- › Single Docker Compose deploy
- › SMB Governance Suite add-on
- › AI Budget Dashboard + Vendor Registry
- › Obsidian Plugin for note security
- › OpenAI-compatible proxy endpoint
- › LangChain WardenCallback
- › SOVA scheduled threat sync (every 6h)
- › Evolution Engine with ArXiv auto-synthesis
Integration Patterns
Send every AI request through POST /filter before passing to your LLM. Synchronous — one inline call on your request path; each response returns its own processing_ms.
curl -X POST https://api.shadow-warden-ai.com/filter \
-H "X-API-Key: $WARDEN_KEY" \
-H "Content-Type: application/json" \
-d '{"content":"<user prompt>","tenant_id":"acme"}'
# → {"blocked":false,"risk_score":0.12,"redacted_content":"...","processing_ms":4.7} Drop-in replacement for the OpenAI base URL. Zero code changes. Supports streaming with 400-char fast-scan buffer.
from openai import OpenAI
client = OpenAI(
base_url="https://api.shadow-warden-ai.com/v1",
api_key="your-openai-key"
) Duck-typed callback handler. Attach to any LangChain chain or agent — no changes to existing code.
from warden.integrations.langchain_callback import WardenCallback
chain = LLMChain(
llm=chat,
callbacks=[WardenCallback(api_key="...", tenant_id="acme")]
) Trigger MasterAgent SOC investigation. Sub-agents run in parallel; high-impact actions require Slack approval.
# Trigger MasterAgent
curl -X POST https://api.shadow-warden-ai.com/agent/master \
-H "X-API-Key: $WARDEN_KEY" \
-d '{"query":"Investigate bypass spike last 2h"}'
# Approve pending high-impact action
curl -X POST https://api.shadow-warden-ai.com/agent/approve/{token}?action=approve \
-H "X-API-Key: $WARDEN_KEY" Deployment Architecture
Shadow Warden runs as 24 Docker services behind a Caddy v2 reverse proxy with QUIC/HTTP3 support. All services are stateless except Redis and PostgreSQL.
| Service | Port | Role |
|---|---|---|
| proxy (Caddy) | 80 / 443 | TLS termination, HTTPS/3 QUIC, vhost routing |
| warden | 8001 | Core FastAPI gateway — all /filter, /agent, /sep endpoints |
| app | 8000 | Streamlit analytics dashboard |
| analytics | 8002 | Analytics REST adapter |
| dashboard | 3002 | Next.js 14 SOC Dashboard |
| postgres | 5432 | Persistent event store (optional) |
| redis | 6379 | ERS, shadow-ban, cache, SOVA memory |
| prometheus | 9090 | Metrics scrape |
| grafana | 3000 | Dashboards + SLO alerts |
| minio | 9000 / 9001 | S3-compatible object store — evidence vault, patrol video |
| minio-init | — | One-shot bucket provisioning init container |
git clone https://github.com/shadow-warden-ai/shadow-warden.git
cp .env.example .env && nano .env
docker compose up --build -d
curl http://localhost:8001/health
# → {"status":"ok","version":"5.2.0","model_loaded":true} Dollar Impact Calculator
Financial impact is calculated across five independent cost models using IBM 2024 benchmarks.
Run python scripts/impact_analysis.py --live --industry fintech for real-time figures.
Agentic SOC — SOVA & MasterAgent
Shadow Warden AI ships two autonomous AI operators: SOVA on Claude Opus 4.8 with 92 tools, and the MasterAgent coordinator. SOVA handles scheduled threat intelligence, health monitoring, and visual patrol. MasterAgent decomposes complex SOC investigations into parallel sub-agent workstreams with HMAC task tokens and human approval gates.
HMAC-SHA256 task tokens prevent cross-agent injection. High-impact actions post to Slack and wait for approval before execution.
Every blocked or flagged request generates a 9-stage causal chain: which layer fired, the primary cause,
and counterfactual remediations. Available as HTML or PDF via GET /xai/report/{id}/pdf.
Deployment Sizing Guide
| Tier | Requests/day | CPU | RAM | Disk | Notes |
|---|---|---|---|---|---|
| Dev / Staging | < 5K | 2 vCPU | 4 GB | 20 GB | Single compose, no MinIO |
| Small Production | 5K – 50K | 4 vCPU | 8 GB | 60 GB | Add Redis persistence |
| Mid-Market | 50K – 500K | 8 vCPU | 16 GB | 200 GB | Horizontal warden scaling |
| Large | 500K+ | 16+ vCPU | 32 GB | 500 GB+ | K8s Helm chart, HA Redis Cluster |
GDPR & Compliance Checklist
Configuration Reference
All configuration is via environment variables. Copy .env.example to .env and override as needed.
| WARDEN_API_KEY | string | Primary API key. Fail-closed if unset and ALLOW_UNAUTHENTICATED != true. |
| WARDEN_API_KEYS_PATH | path | JSON file with per-tenant key list (multi-tenant mode). |
| SEMANTIC_THRESHOLD | 0.72 | MiniLM cosine similarity cutoff (0.0–1.0). Lower = stricter. |
| STRICT_MODE | false | true = block on any non-PASS stage. false = block only on explicit BLOCK verdict. |
| REDIS_URL | redis://redis:6379 | Redis connection string. Use memory:// for tests. |
| LOGS_PATH | /data/logs.json | NDJSON event log. Content never written — metadata only (GDPR). |
| ANTHROPIC_API_KEY | string | Required for SOVA, MasterAgent, Evolution Engine, WardenHealer. Omit to disable all agents. |
| ADMIN_KEY | string | X-Admin-Key header for billing admin endpoints (grant/revoke add-ons). |
| SLACK_WEBHOOK_URL | https://... | Slack incoming webhook for SOVA alerts and MasterAgent approval requests. |
| PATROL_URLS | comma-separated | Extra URLs for sova_visual_patrol in addition to defaults. |
| VAULT_MASTER_KEY | Fernet key | Base64 32-byte Fernet key. Required for community keypairs and sovereign pod encryption. |
| SOVEREIGN_ATTEST_KEY | string | HMAC key for sovereignty attestations. Falls back to VAULT_MASTER_KEY. |
| COMMUNITY_VAULT_KEY | string | Key for encrypting sovereign data pod secret keys. |
| TRANSFER_RISK_THRESHOLD | 0.70 | CausalArbiter P(exfiltration) threshold for SEP transfer blocking. |
| SHADOW_AI_CONCURRENCY | 50 | Max concurrent probes per subnet scan (max prefix /24). |
| SHADOW_AI_PROBE_TIMEOUT | 3 | Per-host timeout in seconds. |
| S3_ENABLED | false | true = enable MinIO/S3 evidence vault and patrol video upload. |
| S3_ENDPOINT_URL | http://minio:9000 | S3-compatible endpoint URL. |
| OTEL_ENABLED | false | Enable OpenTelemetry distributed tracing. Zero overhead when false. |
| OTEL_EXPORTER_OTLP_ENDPOINT | http://otel-collector:4317 | gRPC collector endpoint. |
| INTEL_OPS_ENABLED | false | Enable ArXiv + OSV threat intel background sync. |
| INTEL_BRIDGE_INTERVAL_HRS | 6 | Hours between ArXiv → Evolution Engine sync cycles. |
| SEMANTIC_DB_PATH | /tmp/warden_semantic.db | SQLite path for Self-Service Catalog tenant model registry. |
| SEMANTIC_CACHE_TTL | 600 | Redis query cache TTL in seconds for SemanticEngine.generate(). |
| COMMERCE_BUDGET_ENABLED | true | Enable Budget Guardian check on every AP2 payment flow. |
| M2M_STORE_DB_PATH | /tmp/warden_m2m.db | SQLite path for M2M Commerce Store catalog + inventory. |