STADIUM OS — GenAI Stadium Intelligence for FIFA World Cup 2026

One brain, two faces. A live digital twin of BC Place (Vancouver) where 4,800 simulated fans, real camera feeds, and generative AI meet: every fan gets a local friend who works at the stadium; every operator gets an analyst who never sleeps.

Single HTML file. No build step. No backend. Open it in Chrome and it runs.

---

Problem statement alignment

"Build a GenAI-enabled solution that enhances stadium operations and the overall tournament experience… navigation, crowd management, accessibility, transportation, sustainability, multilingual assistance, operational intelligence, or real-time decision support."

Stadium OS addresses seven of these categories with one coherent architecture:

Category	How
Navigation	AI fan agent answers in natural language and returns a destination; Dijkstra routes draw live on the 3D twin, avoiding congestion and blocked corridors
Crowd management	4,800-agent simulation with per-zone density heatmaps; ops tools (open gates, dispatch staff) visibly mutate crowd flow
Accessibility	Step-free (elevator-only) routing for wheelchair users; voice-first UI; spoken replies for low-vision fans; ARIA + reduced-motion support in the UI itself; every custom control is keyboard-operable (tabindex, roles, Enter/Space)
Transportation	Metro exit modeling; post-match egress simulation
Sustainability	Exit-stagger tool flattens the transit surge via fan-fest incentives instead of idling buses
Multilingual assistance	Personas in Arabic, Spanish, Hindi, English; per-language speech recognition + synthesis; ops broadcasts auto-translated per fan
Operational intelligence & real-time decision support	Ops chat agent grounded in live state JSON with executable tools; auto-alert watchdog generates human-in-the-loop recommendations (Approve to act)

The core idea: one brain, two faces

The crowd simulation emits a normalized state JSON (gate queues, zone densities, service waits, incidents) every tick. Both interfaces inject that same state into every LLM call:

• Fan Companion — voice-first, multilingual, persona-aware (seat, language, accessibility needs), grounded additionally in a stadium ops manual (RAG-style). Answers end in a "Take me there" button that draws the route on the twin and glides the camera.
• Ops Console — control-room chat with executable tools (open_gate, dispatch_staff, broadcast, stagger_exit, clear_incident) that actually change the simulation, plus an auto-alert watchdog with human approval.

Because the AI only ever reads the state JSON, it cannot hallucinate stadium facts — and production deployment means replacing what writes the state (turnstile scans, CV on CCTV, POS rates, GTFS-Realtime transit feeds), never the AI layer.

Real sensors, today: live CCTV vision

Click 📷 Connect live CCTV (Ops Console): your webcam becomes camera CCTV-01. Frames are analyzed by a vision LLM every 15s → people count + density → merged into the same state pipeline → heatmap, agents, and alerts react. Hold a crowd photo to the camera and watch the zone go critical. One camera → vision → normalized state → AI decisions: the production architecture in miniature.

Resilience (battle-tested during development)

• Dual-provider AI: auto-detects Anthropic (sk-ant…) or Google Gemini (AQ.… / AIza…) keys; one translation layer, zero vendor lock-in.
• Self-healing model selection: when Google retired gemini-2.0-flash mid-project (June 2026), we added automatic model discovery — on a 404 the app queries the live model list, selects the newest stable flash model, and retries. Survives provider deprecations without a code change.
• Graceful degradation: every AI feature has a deterministic local fallback (keyword routing, templated alerts), so navigation and ops keep working with no network at all.
• Rate-limit aware: 429s trigger polite pauses, vision cadence tuned for free-tier quotas.

Running it

1. Open stadium-os.html in Chrome (voice + webcam need Chrome/Edge).
2. Add an AI key (optional — app works in fallback mode without one):
   • Click the 🔑 button and paste a key, or hardcode it in the ANTHROPIC_API_KEY constant.
   • Free option: Google Gemini key from aistudio.google.com/apikey (starts with AQ.).
3. The app self-tests on boot and reports status in the AI: pill (click it to re-test anytime).

Suggested 3-minute demo

1. Watch pre-match arrivals flow through gates on the twin (try ☀ DAY / 🌙 NIGHT).
2. Fan Companion → Fatima (Arabic, wheelchair) → tap 🎤, ask for food → hear the Arabic reply → Take me there → step-free route draws, camera glides.
3. Inject Chaos → Spill blocks Corridor 12 → crowd reroutes; Ops Console raises an AI alert → Approve → gates open, staff dispatched, multilingual broadcast lands on the fan's phone.
4. Ops Console → 📷 Connect CCTV → hold a crowd photo to the webcam → the zone goes red from a physical camera.

Testing

Embedded self-test suite (Inject tab → 🧪, or append ?selftest to the URL): 32 tests across 9 suites — graph integrity and reachability, pathfinding, blocked-edge rerouting and restoration, step-free constraint enforcement, state schema, tool-call validation and clamping, XSS escaping, CSP hardening, and keyboard operability of custom controls. Results in console + on-screen summary.

Security

• All LLM/user-derived text is HTML-escaped before any innerHTML sink; chat uses textContent.
• Content-Security-Policy restricts script and connection origins, and hardens against plugin injection, base-tag hijacking, and clickjacking (object-src 'none', base-uri 'self', frame-ancestors 'none').
• AI tool calls are whitelisted and input-validated (clamped staff counts, validated gate IDs); watchdog actions require human approval.
• API keys: in-memory by default, never persisted; hardcoding is documented as local-demo-only. For public hosting, use a serverless proxy with the key in an environment variable.
• Privacy by design: crowd sensing uses aggregate counts only — no identity, no facial recognition; fan location is opt-in.

Tech

Three.js (r128) digital twin with instanced crowd rendering (one draw call for 4,800 agents) · Dijkstra routing on a 53-node navigation graph · Web Speech API for per-language voice in/out · Anthropic Messages API / Gemini generateContent with a unified adapter · zero dependencies beyond a single CDN script.

Deployment

Vercel (full AI, key protected): import this repo at vercel.com → add environment variable GEMINI_API_KEY (free key from aistudio.google.com/apikey) → Deploy. The app is served at / and api/ai.js proxies AI calls with the key held server-side — visitors get full AI without the key ever reaching the browser.

GitHub Pages (static demo): Settings → Pages → deploy main root. Serves the app at the repo's Pages URL; AI runs in local-fallback mode unless a visitor adds their own key via 🔑.