JailbreakOps

A prompt-injection CTF sandbox built for GDG VIT Chennai's TechnoVIT event (Task 1 — LLM Sandbox). Participants try to extract hidden secrets from an LLM across two difficulty levels, earning points for each flag they successfully extract and submit.

Live deployment: https://jailbreakops-api.onrender.com/ (Render free tier — the first request after idle time may take 30-50s while the container cold-starts. This is a hosting-tier characteristic, not an application bug — see Problems Encountered.)

Demo video: https://youtu.be/xTBOeXSJgUI covers the working product end-to-end. This README covers everything the video doesn't have time for — architecture, data flow, security design, scalability evidence, and the engineering decisions behind them — matched directly to the evaluation criteria below.

How It Works

A participant selects a level (1 or 2), submits a natural-language prompt, and the backend forwards it to Google's Gemini API alongside a level-specific system prompt containing a hidden secret. The participant's goal is to socially engineer the model into revealing that secret through the response. Once they believe they've extracted it, they submit it as a flag through a separate verification endpoint, which awards points only on a correct, first-time match — 50 points for Level 1, 100 for Level 2.

The brief explicitly asks for a system prompt that's "just resilient enough," not bulletproof — Level 1's guardrail is intentionally looser than Level 2's, so the round has a real difficulty curve rather than being either trivial or unsolvable. The actual engineering problem isn't "make an AI unbreakable" — it's building a backend around an intentionally-beatable AI that's fast, cheap, abuse-resistant, and horizontally scalable.

Architecture & Data Flow

Both deployment tracks, side by side:

Both tracks run from the same Docker image — one codebase, two deployment stories:

• Local Kind cluster — proves horizontal scaling. The FastAPI app runs as 2-6 replicated pods behind a Kubernetes Service + HPA, exercised with a k6 load test.
• Render cloud deployment — the actual live URL participants use during the event. Deliberately kept simple (single instance); Render's own infra handles availability, so Kubernetes isn't needed here. See Why we didn't run Kubernetes in the cloud.

Local Kubernetes setup, in detail:

Request flow for a single /prompt call, regardless of which track it's running on:

1. Rate limit check — Redis, keyed by session_id (fixed-window counter).
2. Cache lookup — Redis, keyed by a SHA-256 hash of the normalized prompt + level. A hit returns immediately, skipping the LLM call entirely.
3. System prompt build — the level's secret is injected via Gemini's native system_instruction parameter, not string-concatenated into the user message, keeping the injection surface clean.
4. Gemini API call — response length capped via max_output_tokens, with a timeout to protect against hangs.
5. Output guardrail — the raw response is checked for the secret (case-insensitive, whitespace/punctuation-normalized to catch spaced-out obfuscation) before the participant ever sees it. A caught leak is replaced with a generic withheld message.
6. Cache write — only the sanitized response is cached, never a leaked one.
7. Background logging — a non-blocking background task writes the attempt (session, level, prompt, sanitized response, was_cached, flagged, timestamp) to Postgres after the HTTP response has already been sent to the participant, so logging never adds latency.

POST /verify follows a parallel, simpler path: normalize and compare the submitted flag against the level's real secret, and if correct, idempotently award points via a Redis set (solved:{session_id}) and counter (score:{session_id}) — resubmitting a correct flag never double-awards.

Security & Access Control

There's no user-account system by design — this is a session-based CTF, not an authenticated app. What is enforced:

• Rate limiting per session_id on both /prompt and /verify (the latter more tightly, since flag-guessing is a more direct attack than prompt exploration) — protects against both brute-force and LLM-cost abuse.
• Output guardrail, described above — the single most important security property in this project, since the entire premise is that every user is actively trying to extract information from the system.
• No internal detail leakage — every error path returns a clean, generic JSON message. Raw exception text and SDK-internal errors are logged server-side only, never returned to the client.
• No hints on failure — an incorrect flag submission gives zero signal about why it was wrong, to avoid enabling brute-force narrowing.
• Fail-open vs. fail-closed, deliberately asymmetric: if Redis becomes unreachable, rate limiting fails closed (denies the request — an unreachable rate limiter is a cost/abuse risk) while caching fails open (skips the cache, calls the LLM directly — a miss is only a performance cost). This asymmetry is unit-tested, not just a code comment.
• Secrets never committed — all credentials are environment-variable driven. A pre-deployment audit specifically checked git history for accidentally committed keys, confirmed .env was never tracked, and caught (before commit) an unused legacy secret and placeholder flag values that needed sanitizing in the deploy checklist — see Problems Encountered.

Tech Stack & Why

Component	Choice	Reasoning
API framework	FastAPI, fully async	Every request blocks on an external LLM call — async I/O lets one worker handle many concurrent waiting requests instead of tying up a thread per request. Direct scalability win for an I/O-bound workload.
LLM provider	Google Gemini (gemini-3.6-flash)	Fast and inexpensive, available via an existing Google Pro subscription. System prompt passed via Gemini's native system_instruction parameter.
LLM provider abstraction	LLMProvider ABC, factory-selectable	Adding a new provider, or supporting participant-supplied (BYOK) credentials, means adding one class — zero route-logic changes.
Rate limit / cache	Redis	One store, two jobs: per-session rate limiting and prompt-response caching, so repeated prompts skip the paid LLM call entirely.
Persistence	PostgreSQL + async SQLAlchemy + Alembic	Structured attempt log (prompt_attempt, verify_attempt) with indexed (session_id, created_at) matching the real query pattern. Migrations are committed and auto-run on container start.
Containerization	Docker + Docker Compose	docker compose up brings up API + Postgres + Redis with zero manual setup beyond .env.
Local scaling proof	Kubernetes (Kind)	A local, zero-cost way to demonstrate horizontal scaling without provisioning a paid cloud cluster.
Cloud deployment	Render (Blueprint / IaC via render.yaml)	Free-tier web service + managed Postgres + managed Redis, all provisioned from one Blueprint file.
CI	GitHub Actions (lint → test → build)	Catches lint issues, test regressions, and Dockerfile drift on every push.
Testing	pytest + fakeredis + dependency-injected fakes	No real Postgres/Redis/Gemini needed in CI — fast, deterministic, still exercises the real security logic.

Scalability

Stateless API layer. No in-memory session, rate-limit, or cache state lives inside the FastAPI process — everything is externalized to Redis. This is what makes horizontal scaling possible with zero code changes: any number of replicas behave identically because they all coordinate through the same external state.

What the local Kubernetes demo proves (see diagrams above): 2-6 FastAPI replicas behind a Service + CPU-based HPA, load-tested with k6. Results:

• Scale-up 2 → 4 → 6 under load, scale-down back to 2 once load dropped, zero pod restarts throughout.
• Shared-state proof — the part that actually matters for a multi-replica claim: identical prompts hit from different pods resulted in a single shared Redis cache entry, not one per pod, and rate limits were enforced consistently regardless of which pod served a given request. This is direct evidence the replicas coordinate correctly rather than behaving as isolated instances.
• Postgres and Redis deliberately stay outside the Kubernetes cluster, on Docker Compose, reached via host.docker.internal — see Why we didn't run Kubernetes in the cloud for the reasoning.

Database design: composite indexes on (session_id, created_at) in both attempt tables match the actual query pattern (session timelines) rather than indexing speculatively.

Why we didn't run Kubernetes in the cloud

Evaluated and rejected running a managed cloud Kubernetes cluster (EKS/GKE) for the live deployment. A managed control plane isn't free, is more infrastructure than a 3-service project needs, and directly works against the "easily reproducible" goal — a judge would need to provision a cluster before reproducing anything, versus one docker compose up. Kubernetes capability is proven locally instead; the live deployment uses the simpler, genuinely free Render path. This mirrors a real production pattern: prove scalability capability without paying for infrastructure you don't yet need.

Cost Efficiency

• Redis-backed prompt caching — identical prompts skip the paid LLM call entirely.
• max_output_tokens cap — bounds the cost of every individual Gemini call.
• Rate limiting doubles as cost control — the same mechanism that stops abuse also stops runaway API spend; one feature, two purposes.
• Free tier throughout — Render's free web service, free managed Postgres, free managed Redis, and Gemini's low-cost tier. The entire stack runs at $0 for the duration of the event.
• No cloud Kubernetes control-plane cost — see above; this was a cost decision as much as a complexity one.

Reproducibility

• Local: docker compose up --build brings up the entire stack (API + Postgres + Redis) from a clean clone, with Alembic migrations running automatically via an entrypoint script — no separate manual migration step.
• Cloud: render.yaml defines the entire deployment as a single Blueprint — one action provisions the web service, managed Postgres, and managed Redis together. Real secrets are entered once through Render's dashboard, never committed.
• Configuration is entirely environment-variable driven (.env.example documents every variable with a one-line explanation) — reconfiguring limits, points, or credentials needs no code changes.
• CI on every push — GitHub Actions runs lint, test, and a Docker build check, so "it works" is continuously verified, not just asserted once.
• Local Kubernetes demo is itself reproducible: k8s/ contains the full manifest set (Deployment, Service, HPA, ConfigMap, Secret template) and a k8s/README.md with the exact kind create cluster → kubectl apply → k6 run sequence used to reproduce the scaling demo shown above.

Code Quality

• Modular, non-microservices-in-name-only structure: guardrail logic (guardrails.py), caching/rate-limiting (cache.py), and scoring (scoring.py) are pure-function modules with zero FastAPI imports — route handlers are thin pipelines that call into them, not where the logic lives. This keeps a clean seam if any of these ever need to become a separate service.
• Provider abstraction (LLMProvider ABC) decouples route logic from any specific LLM SDK.
• Consistent failure handling — every external dependency (LLM, Redis, Postgres) has an explicit, deliberate failure mode (timeout → clean error; Redis down → fail-open/fail-closed as appropriate; Postgres down → silently logged, never surfaces to the participant), rather than letting exceptions propagate unpredictably.
• 58 automated tests covering guardrail leak detection (verbatim, case-varied, whitespace-obfuscated, and negative/clean cases), scoring idempotency, rate-limit and cache behavior, the fail-open/fail-closed Redis paths, and full API-level request/response contracts via dependency-injected fakes.
• Linted via ruff, enforced in CI on every push.

Problems Encountered & Mitigations

1. CPU-based autoscaling didn't trigger under initial load testing.

At first, our Kubernetes autoscaling did not increase the number of pods during testing. This happened because the /prompt API mostly waits for Redis and Gemini responses, so it does not use much CPU. Our initial k6 test reached only around ~19% CPU against a 50% HPA target, so Kubernetes had no reason to scale up. We fixed this by adjusting the CPU request values and HPA target based on the actual workload. We also made sure the load-test prompts resulted in cache misses so the application performed the complete processing flow. For a larger production system, request-based or queue-based autoscaling could be a better option for this type of workload.

---

2. Cross-cluster networking between Kind and Docker Compose.

Redis and PostgreSQL were intentionally kept outside the Kubernetes cluster and run using Docker Compose. Because of this, the Kubernetes pods needed a way to communicate with them. We solved this using host.docker.internal, which works correctly with Kind and Docker Desktop on Windows and Mac — the environment used for this project. For Linux environments, a different networking configuration would be required.

---

3. Pre-deployment secret hygiene.

Before making the repository public, the project was checked for accidentally exposed credentials and secrets. During this check, we found two things that needed cleaning up: an old unused SYSTEM_PROMPT_SECRET and example flag values that should not expose real CTF answers in a public repository. These were cleaned up before pushing the final version of the project.

---

4. Free-tier cold starts.

Render's free web service can shut down after being inactive for some time. Because of this, the first request after inactivity can take around 30–50 seconds. We considered this when configuring the application's timeout settings so the application has enough time to start and process the request.

Edge Cases Handled

• Empty prompt, oversized prompt (>2000 chars), missing session_id, invalid level, malformed JSON → clean 422 responses.
• LLM provider timeout or non-200 response → clean 502/503, no internal detail leaked.
• Redis unavailable → rate limiting fails closed, caching fails open (see Security).
• Postgres unavailable → attempt logging fails silently server-side; the participant's response is entirely unaffected — verified by stopping Postgres mid-session and confirming zero latency change and zero failed requests.
• Resubmitting a correct flag → idempotent, no double-scoring.
• Incorrect flag → no partial-match feedback.

Running Locally

git clone <repo-url>
cd JailbreakOps
cp .env.example .env
# fill in GEMINI_API_KEY, LEVEL_1_SECRET, LEVEL_2_SECRET
docker compose up --build
# visit http://localhost:8000

Reproducing the Kubernetes Scaling Demo

kind create cluster --config k8s/kind-config.yaml --name jailbreakops
docker build -t jailbreakops-api:local .
kind load docker-image jailbreakops-api:local --name jailbreakops
kubectl apply -f k8s/configmap.yaml -f k8s/secret.yaml -f k8s/deployment.yaml -f k8s/service.yaml -f k8s/hpa.yaml
kubectl get pods -w
kubectl port-forward svc/jailbreakops-api-service 8080:80
# in another terminal:
k6 run k8s/load-test.js
kubectl get hpa -w

Full details, including the host.docker.internal networking setup, are in k8s/README.md.

Deployment

Live at https://jailbreakops-api.onrender.com/, provisioned via the render.yaml Blueprint (web service + managed Postgres + managed Redis). See DEPLOY_CHECKLIST.md for the full deploy and post-deploy verification steps.