-
Notifications
You must be signed in to change notification settings - Fork 498
feat: Add ride-hailing-analytics kit #353
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| LAMATIC_FLOW_ID="LAMATIC_FLOW_ID Flow ID" | ||
| LAMATIC_API_URL="LAMATIC_API_URL" | ||
| LAMATIC_PROJECT_ID="LAMATIC_PROJECT_ID" | ||
| LAMATIC_API_KEY="LAMATIC_API_KEY" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| .lamatic/ | ||
| node_modules/ | ||
| .next/ | ||
| .env | ||
| .env.local | ||
|
|
||
| # The repo root .gitignore has a bare "scripts" rule that unintentionally | ||
| # ignores this kit's own scripts/ directory. Un-ignore it here. | ||
| !scripts/ | ||
| !scripts/** |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,80 @@ | ||
| # Ride-Hailing Text-to-SQL Analytics Assistant | ||
|
|
||
| Ask questions about a ride-hailing operations dataset in plain English and get back a validated SQL query, the actual results, a natural-language answer, and a suggested chart type. Follow-up questions in the same session ("now break that down by pickup city") are understood in context — no need to restate the original question. | ||
|
|
||
| ## Why this is different from a single-shot text-to-SQL demo | ||
|
|
||
| Most text-to-SQL examples handle one isolated question well and stop there. This kit adds two things most demos skip: **conversational memory** (a session-scoped read/write pattern that lets follow-up questions build on the previous query) and a **dedicated safety layer** that independently re-validates every generated query is read-only before it's allowed to run — not just prompt instructions the model might ignore. | ||
|
|
||
| ## Architecture | ||
|
|
||
| ```text | ||
| API Request (question, sessionId) | ||
| → Session Memory (read) — prior question/sql for this session, if any | ||
| → Schema Context — column descriptions for the target table | ||
| → SQL Generator — writes a new query, or extends the prior one for follow-ups | ||
| → SQL Guardrail — independently verifies SELECT-only, blocks dangerous keywords, enforces LIMIT | ||
| → [only if valid] Execute Query — runs the SQL via a read-only-role-backed API | ||
| → Result Interpreter — natural-language answer + suggested chart type | ||
| → Session Memory (write) — upsert this session's question/sql/answer | ||
| → API Response (answer, chartType, sql, results) | ||
| ``` | ||
|
|
||
| ## Setup | ||
|
|
||
| ### 1. Build the flow in Lamatic Studio | ||
|
|
||
| The flow lives in [`flows/ride-hailing-text-to-sql.ts`](./flows/ride-hailing-text-to-sql.ts). Import it into [Lamatic Studio](https://studio.lamatic.ai), set a model on each of the two LLM nodes (SQL Generator, Result Interpreter), and point the **Execute Query** node's URL at your own deployed SQL-execution API (step 2 below). **Deploy** the flow and copy the deployed **Flow ID**. | ||
|
|
||
| This kit's SQL Guardrail step uses an empty string (`""`), not `null`, to represent "cannot answer" — Lamatic's Zod schema builder does not currently support nullable/union types, so downstream logic should check for an empty string rather than `null`/`undefined`. | ||
|
|
||
| ### 2. Deploy the SQL-execution API | ||
|
|
||
| Lamatic doesn't currently have a built-in node for executing arbitrary, dynamically-generated SQL against an external Postgres database synchronously. This kit ships a small Next.js API route that fills that gap — it accepts a SQL string, re-validates it's a SELECT statement, and runs it against your database using a **dedicated read-only Postgres role** (not just an application-layer check). | ||
|
|
||
| You'll need your own Postgres/Supabase database with a compatible schema (see `scripts/` for the expected `lamatic.trips`-style columns referenced in the Schema Context step), and a **read-only** database role/connection string — do not point this at a role with write access. | ||
|
|
||
| Deploy the API route (in `apps/`) to Vercel or any Node hosting provider, and set: | ||
|
|
||
| | Variable | Description | | ||
| |---|---| | ||
| | `READONLY_DB_URL` | Postgres connection string using a **read-only** role | | ||
| | `EXECUTE_SQL_SECRET` | A random shared secret; the route rejects requests without a matching `x-api-secret` header | | ||
|
|
||
| Then, in Lamatic Studio, store `EXECUTE_SQL_SECRET` under **Settings → Secrets** and reference it in the Execute Query node's headers as `{{secrets.project.EXECUTE_SQL_SECRET}}` rather than pasting the literal value — this keeps the secret out of flow exports. | ||
|
|
||
| ### 3. Run the chat app | ||
|
|
||
| ```bash | ||
| cd kits/ride-hailing-analytics/apps | ||
| cp .env.example .env.local # fill in the values below | ||
| npm install | ||
| npm run dev # http://localhost:3000 | ||
| ``` | ||
|
|
||
| ### Environment variables | ||
|
|
||
| | Variable | Where to find it | | ||
| |---|---| | ||
| | `LAMATIC_FLOW_ID` | Studio → deploy the flow → copy Flow ID | | ||
| | `LAMATIC_API_URL` | Studio → Settings → API Docs → Endpoint | | ||
| | `LAMATIC_PROJECT_ID` | Studio → Project settings | | ||
| | `LAMATIC_API_KEY` | Studio → Settings → API Keys | | ||
|
|
||
| ## Try it | ||
|
|
||
| 1. Ask a question: "How many trips happened this year?" | ||
| 2. Ask a follow-up in the same session: "Now break that down by pickup city." | ||
| 3. The second answer builds on the first query's filters automatically, without you needing to repeat "this year." | ||
|
|
||
| ## Design notes | ||
|
|
||
| - **Read-only enforcement is layered, not single-point.** The SQL Generator is prompted to only write SELECTs; the Guardrail step independently re-checks this; and the database connection itself uses a role with no write privileges. Any one layer failing doesn't expose write access. | ||
| - **Memory is session-scoped and explicit, not implicit.** The read-side lookup is a simple keyed table select, not a vector or fuzzy match — deterministic and easy to reason about. The prompt explicitly handles the empty-session case so a fresh conversation isn't contaminated by hallucinated "prior" context. | ||
| - **The SQL-execution API is a deliberate, documented external dependency**, not hidden platform magic — Lamatic doesn't yet have a synchronous "run this ad-hoc SQL string" node, so this kit is explicit about filling that gap rather than working around it silently. | ||
|
|
||
| ## Future improvements | ||
|
|
||
| - **Full chat history, not just one turn of memory.** `memory_table` currently stores only the *most recent* question/SQL/answer per session (an upsert target, not an append-only log) — enough for the SQL Generator to understand a single follow-up, but not enough to reconstruct or browse a full conversation. Supporting real chat history would mean changing the write-side to insert a new row per turn instead of updating one row per session, and adding a read endpoint the UI could page through. Left out of this submission to keep the scope focused on the core text-to-SQL + single-turn-follow-up problem. | ||
|
|
||
| Built on [Lamatic](https://lamatic.ai). |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| # Ride-Hailing Text-to-SQL Analytics Assistant | ||
|
|
||
| ## Overview | ||
|
|
||
| A conversational analytics assistant over a ride-hailing operations dataset. Ask a question in plain English — "How many trips happened this year?" — and get back a validated, read-only SQL query, the actual query results, a natural-language answer, and a suggested chart type. Follow-up questions in the same session ("now break that down by pickup city") are understood in context, without needing to restate the original question. | ||
|
|
||
| ## Purpose | ||
|
|
||
| Most text-to-SQL demos handle a single, isolated question well but fall apart on natural conversational follow-ups, and many skip query safety entirely. This kit addresses both: a session-scoped memory pattern lets the SQL Generator see the prior turn's question and query, and a dedicated guardrail step enforces SELECT-only, LIMIT-bounded queries before anything touches the database. | ||
|
|
||
| ## Flow Architecture | ||
|
|
||
| Single flow, sequential steps: | ||
|
|
||
| 1. **API Request Trigger** — accepts `{ question, sessionId }`. | ||
| 2. **Session Memory (read)** — looks up the most recent `question`/`sql`/`answer` for this `sessionId` from a `memory_table`, if one exists. | ||
| 3. **Schema Context** — returns a structured description of the target table's columns, so the SQL Generator doesn't need schema knowledge baked into its prompt. | ||
| 4. **SQL Generator** — an instructor LLM step that produces `{ sql, explanation }`. Given the schema, the current question, and the prior turn's question/SQL (if any), it either writes a new query or extends the previous one for follow-up questions. Outputs an empty `sql` string (never `null`) when a question can't be answered from the schema. | ||
| 5. **SQL Guardrail** — validates the generated SQL is a single SELECT statement, blocks dangerous keywords, and appends a LIMIT clause if missing. | ||
| 6. **Conditional routing** — only proceeds to execution if the guardrail marks the query valid. | ||
| 7. **Execute Query** — POSTs the validated SQL to a small external API route backed by a read-only Postgres role, which runs the query and returns rows. | ||
| 8. **Result Interpreter** — an instructor LLM step that turns the raw query rows into a natural-language `answer` and a suggested `chartType`. | ||
| 9. **Session Memory (write)** — inserts or updates the `memory_table` row for this `sessionId` with the latest `question`, `sql`, and `answer`, so the next turn in the same session has context. | ||
| 10. **API Response** — returns `{ answer, chartType, sql, results }`. | ||
|
|
||
| ## Guardrails | ||
|
|
||
| - The SQL Generator is instructed to only ever produce `SELECT` statements, to always include a `LIMIT` clause, and to never reference columns outside the provided schema. | ||
| - The SQL Guardrail step independently re-validates the query is SELECT-only and free of dangerous keywords before it's allowed to execute — the LLM's own instruction-following is not the only line of defense. | ||
| - SQL execution runs against a dedicated **read-only** database role at the connection level, not just an application-layer check, so even a guardrail bypass cannot mutate data. | ||
| - When a question can't be answered with the available schema, the SQL Generator returns an empty string rather than fabricating a plausible-looking but unanswerable query. | ||
| - The session memory read step is guarded in the prompt itself: if no prior question/SQL exists for a session, the model is explicitly instructed to treat the turn as a new conversation rather than inferring false context from empty fields. | ||
|
|
||
| ## Integration Reference | ||
|
|
||
| - **Trigger:** API Request (`question`, `sessionId`) | ||
| - **Output:** `{ answer, chartType, sql, results }` returned via API Response | ||
| - **External dependency:** a small SQL-execution API (see `apps/` and this kit's README for setup) that validates and runs the generated SQL against your Postgres/Supabase instance using a read-only role | ||
| - See `flows/ride-hailing-text-to-sql.ts` for the full node graph and the `prompts/`, `model-configs/`, and `scripts/` directories for prompt text, model selection, and guardrail code. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| LAMATIC_FLOW_ID="LAMATIC_FLOW_ID Flow ID" | ||
| LAMATIC_API_URL="LAMATIC_API_URL" | ||
| LAMATIC_PROJECT_ID="LAMATIC_PROJECT_ID" | ||
| LAMATIC_API_KEY="LAMATIC_API_KEY" | ||
|
Comment on lines
+1
to
+4
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- relevant files ---'
git ls-files | rg '(^|/)(README[^/]*|\.env\.example|package\.json|next\.config\.mjs|tsconfig\.json)$|ride-hailing-analytics'
printf '%s\n' '--- environment variable references ---'
rg -n --hidden --glob '!node_modules' --glob '!dist' --glob '!build' \
'READONLY_DB_URL|EXECUTE_SQL_SECRET|LAMATIC_FLOW_ID|LAMATIC_API_URL|LAMATIC_PROJECT_ID|LAMATIC_API_KEY' \
kits/ride-hailing-analytics README.md 2>/dev/null || true
printf '%s\n' '--- target template ---'
cat -n kits/ride-hailing-analytics/apps/.env.exampleRepository: Lamatic/AgentKit Length of output: 22173 🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- README setup and deployment instructions ---'
cat -n kits/ride-hailing-analytics/README.md | sed -n '1,110p'
printf '%s\n' '--- apps files and scripts ---'
git ls-files kits/ride-hailing-analytics/apps | sort
printf '%s\n' '--- flow sections around SQL execution ---'
cat -n kits/ride-hailing-analytics/flows/ride-hailing-text-to-sql.ts | sed -n '190,250p'
printf '%s\n' '--- package scripts and app configuration ---'
cat -n kits/ride-hailing-analytics/apps/package.json
cat -n kits/ride-hailing-analytics/apps/next.config.mjsRepository: Lamatic/AgentKit Length of output: 10727 🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- all execute-sql references ---'
rg -n --hidden --glob '!node_modules' --glob '!dist' --glob '!build' \
'execute-sql|READONLY_DB_URL|EXECUTE_SQL_SECRET' . 2>/dev/null || true
printf '%s\n' '--- application action and page ---'
cat -n kits/ride-hailing-analytics/apps/actions/orchestrate.ts
cat -n kits/ride-hailing-analytics/apps/app/page.tsx | sed -n '1,220p'
printf '%s\n' '--- route-like files in this app ---'
find kits/ride-hailing-analytics/apps -type f \( -path '*/api/*' -o -name 'route.ts' -o -name 'route.js' -o -name 'route.tsx' \) -printRepository: Lamatic/AgentKit Length of output: 12206 Deploy the SQL execution API The README and flow reference 🧰 Tools🪛 dotenv-linter (4.0.0)[warning] 2-2: [QuoteCharacter] The value has quote characters (', ") (QuoteCharacter) [warning] 2-2: [UnorderedKey] The LAMATIC_API_URL key should go before the LAMATIC_FLOW_ID key (UnorderedKey) [warning] 3-3: [QuoteCharacter] The value has quote characters (', ") (QuoteCharacter) [warning] 4-4: [QuoteCharacter] The value has quote characters (', ") (QuoteCharacter) [warning] 4-4: [UnorderedKey] The LAMATIC_API_KEY key should go before the LAMATIC_API_URL key (UnorderedKey) 🤖 Prompt for AI Agents |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| "use server" | ||
|
|
||
| import { lamaticClient } from "@/lib/lamatic-client" | ||
| import config from "../../lamatic.config" | ||
|
|
||
| export type QueryResultRow = Record<string, string | number | null> | ||
|
|
||
| export async function askQuestion( | ||
| question: string, | ||
| sessionId: string, | ||
| ): Promise<{ | ||
| success: boolean | ||
| data?: { | ||
| answer: string | ||
| chartType: string | ||
| sql: string | ||
| results: QueryResultRow[] | ||
| } | ||
| error?: string | ||
| }> { | ||
| try { | ||
| const flows = config.flows | ||
| const firstFlowKey = Object.keys(flows)[0] | ||
|
|
||
| if (!firstFlowKey) { | ||
| throw new Error("No workflows found in configuration") | ||
| } | ||
|
|
||
| const flow = flows[firstFlowKey as keyof typeof flows] as (typeof flows)[keyof typeof flows] | ||
|
|
||
| if (!flow.workflowId) { | ||
| throw new Error("Workflow not found in config.") | ||
| } | ||
|
|
||
| const inputs = { | ||
| question, | ||
| sessionId, | ||
| } | ||
|
|
||
| const resData = await lamaticClient.executeFlow(flow.workflowId, inputs) | ||
|
|
||
| const answer = resData?.result?.answer | ||
| const chartType = resData?.result?.chartType ?? "none" | ||
| const sql = resData?.result?.sql ?? "" | ||
| const results = resData?.result?.results ?? [] | ||
|
|
||
| if (!answer) { | ||
| throw new Error("No answer found in response") | ||
| } | ||
|
|
||
| return { | ||
| success: true, | ||
| data: { answer, chartType, sql, results }, | ||
| } | ||
| } catch (error) { | ||
| console.error("Query error:", error) | ||
|
|
||
| let errorMessage = "Unknown error occurred" | ||
| if (error instanceof Error) { | ||
| errorMessage = error.message | ||
| if (error.message.includes("fetch failed")) { | ||
| errorMessage = "Network error: Unable to connect to the service. Please check your internet connection and try again." | ||
| } else if (error.message.includes("API key")) { | ||
| errorMessage = "Authentication error: Please check your API configuration." | ||
| } | ||
| } | ||
|
|
||
| return { | ||
| success: false, | ||
| error: errorMessage, | ||
| } | ||
| } | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| @import "tailwindcss"; | ||
|
|
||
| :root { | ||
| --background: #f8fafc; | ||
| --foreground: #0f172a; | ||
| --card: #ffffff; | ||
| --card-foreground: #0f172a; | ||
| --border: #d1d5db; | ||
| --muted-foreground: #4b5563; | ||
| --primary: #2563eb; | ||
| --primary-foreground: #ffffff; | ||
| --primary-hover: #1d4ed8; | ||
| --destructive: #dc2626; | ||
| --destructive-foreground: #b91c1c; | ||
| --destructive-bg: #fef2f2; | ||
| --destructive-border: #fecaca; | ||
| --link: #2563eb; | ||
| --user-bubble: #2563eb; | ||
| --user-bubble-foreground: #ffffff; | ||
| --assistant-bubble: #ffffff; | ||
| --assistant-bubble-foreground: #0f172a; | ||
| } | ||
|
|
||
| @media (prefers-color-scheme: dark) { | ||
| :root { | ||
| --background: #030712; | ||
| --foreground: #f3f4f6; | ||
| --card: #111827; | ||
| --card-foreground: #f3f4f6; | ||
| --border: #374151; | ||
| --muted-foreground: #9ca3af; | ||
| --primary: #2563eb; | ||
| --primary-foreground: #ffffff; | ||
| --primary-hover: #1d4ed8; | ||
| --destructive: #f87171; | ||
| --destructive-foreground: #f87171; | ||
| --destructive-bg: rgba(127, 29, 29, 0.2); | ||
| --destructive-border: #991b1b; | ||
| --link: #60a5fa; | ||
| --user-bubble: #2563eb; | ||
| --user-bubble-foreground: #ffffff; | ||
| --assistant-bubble: #111827; | ||
| --assistant-bubble-foreground: #f3f4f6; | ||
| } | ||
| } | ||
|
|
||
| @theme inline { | ||
| --color-background: var(--background); | ||
| --color-foreground: var(--foreground); | ||
| --color-card: var(--card); | ||
| --color-card-foreground: var(--card-foreground); | ||
| --color-border: var(--border); | ||
| --color-muted-foreground: var(--muted-foreground); | ||
| --color-primary: var(--primary); | ||
| --color-primary-foreground: var(--primary-foreground); | ||
| --color-primary-hover: var(--primary-hover); | ||
| --color-destructive: var(--destructive); | ||
| --color-destructive-foreground: var(--destructive-foreground); | ||
| --color-destructive-bg: var(--destructive-bg); | ||
| --color-destructive-border: var(--destructive-border); | ||
| --color-link: var(--link); | ||
| --color-user-bubble: var(--user-bubble); | ||
| --color-user-bubble-foreground: var(--user-bubble-foreground); | ||
| --color-assistant-bubble: var(--assistant-bubble); | ||
| --color-assistant-bubble-foreground: var(--assistant-bubble-foreground); | ||
| } | ||
|
|
||
| body { | ||
| background: var(--background); | ||
| color: var(--foreground); | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| import type { Metadata } from "next" | ||
| import "./globals.css" | ||
|
|
||
| export const metadata: Metadata = { | ||
| title: "Ride-Hailing Analytics Assistant", | ||
| description: "Ask questions about ride-hailing trip data in plain English", | ||
| } | ||
|
|
||
| export default function RootLayout({ children }: { children: React.ReactNode }) { | ||
| return ( | ||
| <html lang="en"> | ||
| <body>{children}</body> | ||
| </html> | ||
| ) | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
Repository: Lamatic/AgentKit
Length of output: 22686
🏁 Script executed:
Repository: Lamatic/AgentKit
Length of output: 6648
Harden the SQL safety boundary or narrow the documentation claims.
The validator accepts multiple statements, treats
LIMITinside comments as active, and accepts limits above 500. No SQL execution API route exists underkits/ride-hailing-analytics/apps/, although the flow calls/api/execute-sql. Implement strict parsing and matching validation at the execution boundary, or updatekits/ride-hailing-analytics/agent.mdand the repeated claims inkits/ride-hailing-analytics/README.mdLines 7, 16-17, and 72.🤖 Prompt for AI Agents