Project Readiness Copilot

A multi-agent, MCP-backed AI copilot that turns "Are we ready to launch?" into a structured, evidence-based answer — with full visibility into how the answer was produced (which agents ran, what tools they called, how long it took, what it cost, and what security checks passed).

---

1. Problem Statement

Before any product or feature launch, someone — a PM, an EM, a director — has to answer a deceptively simple question: "Are we ready?"

Today that answer is assembled by hand: reading through scattered docs to understand scope, digging through a ticket tracker for open risks, checking a project plan for dependencies, and then writing up a coherent update for stakeholders. It is slow, inconsistent between people, and easy to miss a blocker buried in a ticket nobody re-read.

At the same time, leaders evaluating "agentic AI" are asking a second, harder question: can we trust what the agents did to get that answer? Most agent demos are black boxes — you see an input and an output, with no visibility into reasoning, cost, latency, or the tools the agents touched along the way.

This project solves both problems in one system.

2. Use Cases

#	Use case	Who benefits
1	"Are we ready for launch?" readiness check, on demand, in seconds instead of a status-gathering meeting	PMs, EMs, launch leads
2	Auto-drafted, executive-ready stakeholder communication that a human reviews before sending	PMs, Comms, leadership
3	Standing risk/blocker visibility pulled live from the tracker rather than a stale spreadsheet	Eng leads, program managers
4	A reference implementation for how to build observable, governed multi-agent systems — reusable for any other multi-agent workflow (not just launch readiness)	Platform / AI engineering teams
5	A live, presentable demo of agent-to-agent orchestration, MCP tool use, cost/latency accounting, and guardrails — for both technical and non-technical audiences	Solution architects, enterprise AI leads

3. Value Proposition

• Minutes, not meetings. A readiness check that used to require pulling three people into a status meeting now returns a structured answer in under a minute.
• Consistent, auditable reasoning. Every agent runs from a versioned prompt template — the same question is answered the same way every time, and every prompt is inspectable (/backend/app/prompts/).
• Governed by design, not by accident. Guardrails (prompt-injection scanning, PII detection, MCP tool allow-listing) run on every call and are visible in the UI — not hidden in logs.
• Full cost and latency accountability. Every LLM call is metered (tokens, cost in USD, latency in ms) and shown per-agent, so this pattern can be evaluated for production cost before it's scaled to other workflows.
• Vendor-neutral pattern. Swap OpenAI for another model provider, or the mock MCP servers for real MCP servers (Jira, Confluence, a real doc store) — the orchestrator, prompts, and UI don't change.

4. Solution — Architecture

┌───────────────────────────────────────────────────────────────────┐
│  React Frontend (white theme, Copilot-style)                       │
│  ├── /            Chat — ask a readiness question                  │
│  └── /flow         System & Data Flow — agent graph, latency, cost, │
│                     security panel, event timeline, LangSmith links │
└──────────────────────────┬───────────────────────────────────────┘
                            │ WebSocket (/ws/chat) — live TraceEvents
                            │ REST (/api/flow/*, /api/uploads)
┌──────────────────────────▼───────────────────────────────────────┐
│  FastAPI Backend (Python 3.12)                                      │
│  ├── Orchestrator "Brain" (LangGraph StateGraph)                    │
│  │     plan → (research_agent ∥ risk_agent) → planning_agent        │
│  │     → communication_agent → synthesize                            │
│  ├── Agents: Research / Risk / Planning / Communication              │
│  │     each = versioned prompt template + MCP tool bindings +        │
│  │     OpenAI call via LangChain (auto-traced to LangSmith)          │
│  ├── MCP layer: docs_mcp / tracker_mcp / timeline_mcp                │
│  │     (mock data by default; real uploads feed docs_mcp)            │
│  ├── Security: prompt-injection scan, PII scan/redaction,            │
│  │     MCP tool allow-listing — all streamed as guardrail events     │
│  └── Observability: per-call latency/tokens/cost, in-memory run       │
│        registry for replay, LangSmith deep-links                     │
└──────────────────────────┬───────────────────────────────────────┘
                            │
┌──────────────────────────▼───────────────────────────────────────┐
│  OpenAI API (LLM calls)  ·  LangSmith (trace storage)  ·  MCP tools │
└───────────────────────────────────────────────────────────────────┘

The "Brain" — why LangGraph

The Coordinator does not run a fixed pipeline. It:

1. Plans — reasons over the question and decides which specialist agents are needed (orchestrator.plan prompt).
2. Dispatches — Research and Risk agents run in parallel (true concurrency via LangGraph's superstep model, not just async syntax). Planning waits for both; Communication waits for Planning.
3. Synthesizes — merges every agent's output into one coherent, plain-language report (orchestrator.synthesize prompt).

Every step is a LangGraph node, so LangSmith automatically captures the full execution tree — the /flow page's "View in LangSmith" links point straight into that trace for anyone who wants to go deeper than the UI shows.

MCP (Model Context Protocol)

Agents never call tools directly — every tool call goes through MCPClient.call(server, tool, args). In this repo the three MCP "servers" (docs_mcp, tracker_mcp, timeline_mcp) run in-process with mock data by default, with real user uploads flowing into docs_mcp when provided. This keeps the demo self-contained while making the swap to a real MCP transport (stdio/SSE against Jira, Confluence, etc.) a one-file change (backend/app/mcp/servers.py), not a rewrite.

Prompt templates

Every agent action is a Markdown file under backend/app/prompts/, loaded and filled at runtime — not a string buried in Python. This is what makes the system auditable to a non-technical stakeholder: "here is exactly what we told the AI to do."

Observability page (/flow)

Built specifically to make multi-agent execution legible to a mixed technical / non-technical audience:

• Agent Graph — live node/edge diagram; edges light up as agents hand off to each other (the actual agent-to-agent communication, not a mockup)
• Metrics — per-agent latency, token count, and cost, plus running totals
• Security — every guardrail check (pass/flag) and every MCP tool call, with latency
• Timeline — full chronological waterfall of the run, each entry linking to its LangSmith trace when tracing is enabled

---

5. Engineering Runbook

Prerequisites

• Docker + Docker Compose (recommended path), or
• Python 3.12 and Node.js 20+ for running services locally without Docker
• An OpenAI API key
• (Optional but recommended for the observability demo) A LangSmith API key

5.1 Configure environment

cp .env.example .env
# then edit .env and set at minimum:
#   OPENAI_API_KEY=sk-...
# optionally set LANGCHAIN_TRACING_V2=true and LANGCHAIN_API_KEY=... for LangSmith

5.2 Run with Docker (recommended)

docker compose up --build

• Frontend: http://localhost:5173
• Backend API: http://localhost:8000 (docs at http://localhost:8000/docs)

5.3 Run locally without Docker

Backend:

cd backend
python3.12 -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
pip install -r requirements.txt
cp ../.env.example ../.env         # if not already created
uvicorn app.main:app --reload --port 8000

Frontend (separate terminal):

cd frontend
npm install
npm run dev

Frontend dev server runs at http://localhost:5173 and expects the backend at http://localhost:8000 (see VITE_API_BASE_URL / VITE_WS_BASE_URL in .env).

5.4 Using the demo

1. Open the Chat tab, ask (or click a suggested) readiness question.
2. Watch the Coordinator's reasoning appear, then agent status chips light up as Research and Risk run in parallel, followed by Planning and Communication.
3. Read the final readiness report (summary, risks, action plan, follow-ups, and a draft stakeholder update for human review — nothing is auto-sent).
4. Click "View system & data flow for this run" (or the System & Data Flow tab) to see:
   • the live agent-to-agent graph
   • latency / token / cost breakdown per agent
   • every guardrail check and MCP tool call
   • the full event timeline with LangSmith deep-links (if tracing is on)

5.5 Project structure

project-readiness-copilot/
├── README.md                  (this file)
├── .env.example
├── docker-compose.yml
├── backend/
│   ├── Dockerfile
│   ├── requirements.txt
│   └── app/
│       ├── main.py            FastAPI app entrypoint
│       ├── config.py          Settings from .env
│       ├── orchestrator/      LangGraph "brain" + tracing + state
│       ├── agents/            Research / Risk / Planning / Communication
│       ├── mcp/                Mock MCP tool servers + client
│       ├── prompts/            Versioned prompt templates (.md)
│       ├── security/           Guardrails (injection/PII/allow-list)
│       ├── observability/      Cost/latency accounting + run registry
│       ├── ws/                  WebSocket connection manager
│       ├── models/              Shared Pydantic schemas
│       └── routers/             chat (WS), flow (REST), uploads (REST)
└── frontend/
    ├── Dockerfile
    ├── package.json
    └── src/
        ├── pages/Chat.tsx       Main Copilot-style demo surface
        ├── pages/Flow.tsx       System & Data Flow observability page
        ├── components/          AgentGraph, MetricsPanel, SecurityPanel, ...
        ├── hooks/useChatRun.ts  WebSocket streaming hook
        └── styles/index.css     Design tokens (white theme)

5.6 Troubleshooting

Symptom	Likely cause / fix
Chat page shows a connection error	Backend not running or VITE_WS_BASE_URL doesn't match the backend's actual host/port
/flow page says "No runs yet"	You haven't run a chat query yet in this backend process — the run registry is in-memory and resets on backend restart
LangSmith links don't work / are absent	LANGCHAIN_TRACING_V2 is false or LANGCHAIN_API_KEY is unset in .env
401 from OpenAI	Check OPENAI_API_KEY in .env; container needs a restart after editing .env (docker compose up --build)
File upload rejected	Only .txt and .md files are accepted in this demo (backend/app/routers/uploads.py)

5.7 Extending the system

• Add a new agent: create backend/app/agents/<name>_agent.py + backend/app/prompts/<name>_agent.system.md, add a node + edges in backend/app/orchestrator/graph.py.
• Point at real data: replace the mock responses in backend/app/mcp/servers.py with real MCP client calls (Jira, Confluence, etc.) — the rest of the system is unaffected.
• Swap the LLM provider: change the ChatOpenAI construction in backend/app/agents/llm.py to any LangChain-compatible chat model.