Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
93fa217
feat(examples): add web UI + region metadata filtering to voice-agent
HarshaNalluru Jul 7, 2026
78886cc
examples(voice-agent): address review
HarshaNalluru Jul 9, 2026
3f9a979
examples(voice-agent): set tsconfig baseUrl for @/ paths; token route…
HarshaNalluru Jul 9, 2026
7d4cb55
examples(voice-agent): address review
HarshaNalluru Jul 9, 2026
23dc412
examples(voice-agent): neutral context label + no-match guardrail, ca…
HarshaNalluru Jul 9, 2026
784931b
examples(voice-agent): fix review issues for live retrieval UI
samanyugoyal2010 Jul 17, 2026
403de62
examples(voice-agent): address follow-up review findings
samanyugoyal2010 Jul 17, 2026
cb18de5
examples(voice-agent): harden token boundary, region races, and paylo…
samanyugoyal2010 Jul 17, 2026
aa2fc54
examples(voice-agent): close remaining token and region race gaps
samanyugoyal2010 Jul 17, 2026
8b42ad3
Update moss-live-labs/examples/voice-agent/web/app/api/token/route.ts
samanyugoyal2010 Jul 17, 2026
58b57d6
Update moss-live-labs/examples/voice-agent/web/app/api/token/route.ts
samanyugoyal2010 Jul 17, 2026
c8adde1
examples(voice-agent): fix Windows env scripts, prod tokens, proxy he…
samanyugoyal2010 Jul 17, 2026
1c811dc
examples(voice-agent): test token guard and drop dead region/session …
samanyugoyal2010 Jul 17, 2026
0305335
examples(voice-agent): harden tokenGuard against spoofed proxy and Ho…
samanyugoyal2010 Jul 17, 2026
75a9443
Update moss-live-labs/examples/voice-agent/web/lib/tokenGuard.ts
samanyugoyal2010 Jul 17, 2026
15987bc
examples(voice-agent): log TRUST_PROXY misconfig once at startup
samanyugoyal2010 Jul 17, 2026
cc8c782
Merge remote branch
samanyugoyal2010 Jul 17, 2026
a821b56
Potential fix for pull request finding
samanyugoyal2010 Jul 28, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion moss-live-labs/examples/voice-agent/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,11 @@ LIVEKIT_API_SECRET=secret
MOSS_PROJECT_ID=Your moss project id here
MOSS_PROJECT_KEY=Your moss project key here

# AI Provider Keys
# Initial region for console / no-UI runs only (US or EU). The web UI picker
# overrides this on connect — leave US unless you are using `agent.py console`.
MOSS_REGION=US

# AI Provider Keys (OpenAI = LLM, Deepgram = STT, Cartesia = TTS)
OPENAI_API_KEY=Your openai api key here
DEEPGRAM_API_KEY=Your deepgram api key here
CARTESIA_API_KEY=Your cartesia api key here
50 changes: 50 additions & 0 deletions moss-live-labs/examples/voice-agent/DEMO_SCRIPT_METADATA.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Demo Script — Metadata Filtering (region-scoped answers)

~90s. Shows the **same question** returning **different, correct answers** by region,
because Moss filters retrieval on `metadata.region` — one shared index, scoped per query.

## Before you record
- [ ] **Re-seed** (metadata changed): delete the configured index in the Moss portal
(`MOSS_INDEX_NAME`, default `demo-customer_faqs`), then `uv run python seed_index.py`.
- [ ] `uv run python agent.py dev` · `livekit-server --dev` · `cd web && npm run dev`
(UI at localhost:3000)
- [ ] Region is chosen with the **US / EU picker** in the panel (starts on US) — no restart.

## 1 · Frame it (0:00–0:12)
> "Same knowledge base, one index. This support panel has a region picker — right now it's
> set to the US, and Moss only retrieves policies that apply to that region. Watch."

**[Connect. Panel shows the picker on US and `region: US + global`.]**

## 2 · US answer (0:12–0:35)
> "What's your return window?"

**[Point at panel: the filter line + the retrieved US chunk.]**
> "Thirty days. The panel shows the filter — region is US or global — and it pulled the
> US returns policy. The EU policy is in the same index, but it was filtered out."

## 3 · Switch region (0:35–1:05)
**[Click EU in the picker — the filter line flips to `region: EU + global` instantly.]**
> "Now I switch the same agent to the EU — no restart, just the filter. Same question."

> "What's your return window?"

> "Fourteen days — the EU right of withdrawal. Same index, same question, different answer,
> because the metadata filter scoped retrieval to the right region."

## 4 · Why it matters (1:05–1:25)
> "That's one line of filter in the query. It's how you serve region-specific policies,
> or isolate tenants, or gate content by plan — all from a single Moss index, evaluated
> locally in milliseconds. Open source at github.com/usemoss/moss."

---

## The filter (for reference)
```python
QueryOptions(top_k=5, alpha=0.8,
filter={"field": "region", "condition": {"$in": ["EU", "all"]}})
```

## Try these too
- "How much is shipping?" → US quotes dollars, EU quotes euros (with VAT).
- A global question ("I forgot my password") → same answer in both regions (region = `all`).
96 changes: 62 additions & 34 deletions moss-live-labs/examples/voice-agent/README.md
Original file line number Diff line number Diff line change
@@ -1,58 +1,86 @@
# Voice Agent
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.

A voice assistant that answers questions using your Moss knowledge base. You talk to it, it searches your data, and responds out loud.
A customer-support **voice agent** grounded in a Moss knowledge base, with a brand-styled
web UI that shows Moss doing the retrieval live. You talk to it, it searches your data,
and a side panel surfaces the exact chunks Moss returned (with match scores and query
latency) for every turn.

```
Browser (web/) ⟷ LiveKit room ⟷ agent.py (STT → LLM → TTS) ⟷ Moss (RAG)
```

## What you need

- A [Moss](https://moss.dev) account with a project and an index loaded with your data
- A [LiveKit](https://livekit.io) account (or run it locally)
- An [OpenAI](https://platform.openai.com) API key (for the AI responses)
- A [Deepgram](https://deepgram.com) API key (for speech-to-text)
- A [Cartesia](https://play.cartesia.ai) API key (for text-to-speech)
- A [Moss](https://moss.dev) account (project ID + key)
- [LiveKit](https://livekit.io) — local (`livekit-server --dev`) or [LiveKit Cloud](https://cloud.livekit.io)
- [OpenAI](https://platform.openai.com) (LLM), [Deepgram](https://deepgram.com)
(speech-to-text), and [Cartesia](https://cartesia.ai) (text-to-speech) API keys
- Python 3.14+ (`uv`) and Node 18.18+ (`npm`) for the web UI

## Setup

1. Install dependencies:
```bash
uv sync
cp .env.example .env # fill in your Moss + provider keys
uv run python agent.py download-files
```

`.env` keys: `MOSS_PROJECT_ID`, `MOSS_PROJECT_KEY`, `OPENAI_API_KEY`, `DEEPGRAM_API_KEY`, `CARTESIA_API_KEY`.
For local LiveKit, leave the `LIVEKIT_*` values as-is; for LiveKit Cloud, set them to your
project's URL/key/secret and copy the same three into `web/.env.local`. The index name
defaults to `demo-customer_faqs` (override with `MOSS_INDEX_NAME`).

```bash
uv sync
```
## 1. Seed the knowledge base

2. Copy the env file and fill in your keys:
Loads the sample support FAQs in `data/faqs.json` into a Moss index:

```bash
cp .env.example .env
```
```bash
uv run python seed_index.py
```

Open `.env` and add:
## 2. Run it

```env
LIVEKIT_URL=ws://localhost:7880
LIVEKIT_API_KEY=devkey
LIVEKIT_API_SECRET=secret
Open three terminals (skip terminal **a** if you are on LiveKit Cloud):

MOSS_PROJECT_ID=your-project-id
MOSS_PROJECT_KEY=your-project-key
MOSS_INDEX_NAME=your-index-name
```bash
# a) LiveKit server — local-dev only (skip when using LiveKit Cloud)
livekit-server --dev

# b) the agent (joins rooms automatically)
uv run python agent.py dev

# c) the web UI
cd web
npm install
# create once; do not overwrite an existing Cloud-configured .env.local
[ -f .env.local ] || cp .env.local.example .env.local
npm run dev # → http://127.0.0.1:3000 (loopback-only; `npm run dev:lan` enables LAN + ALLOW_REMOTE_TOKEN)
```

OPENAI_API_KEY=...
DEEPGRAM_API_KEY=...
CARTESIA_API_KEY=...
```
Open http://127.0.0.1:3000, click **Start the demo**, and talk. The right-hand panel
shows what Moss retrieves on each turn.

3. Download the required model files:
> Prefer no UI? `uv run python agent.py console` still works for a mic-only, terminal session.
> For console-only demos you can set `MOSS_REGION=US` or `EU` in `.env`; the web UI picker
> is authoritative when using the browser and overrides that default on connect.

```bash
python agent.py download-files
```
## How the retrieval panel works

## Run
On each user turn, `agent.py` queries Moss and publishes the results to the LiveKit room
on the `moss.retrieval` data channel:

```bash
python agent.py console
```json
{
"query": "string",
"docs": [{ "id": "string | null (optional)", "text": "string", "score": 0.0 }],
"took_ms": 0.0,
"region": "US | EU"
}
```

Speak into your microphone and the agent responds out loud.
The web UI listens on that channel and renders them. The voice pipeline is otherwise untouched.

See [`DEMO_SCRIPT_METADATA.md`](./DEMO_SCRIPT_METADATA.md) for a ready-to-record walkthrough.

## Resources

Expand Down
Loading
Loading