Skip to content

feat(dashboards): SQL-defined datasets, publish gate, list/load tools - #346

Merged
8 commits merged into
stagingfrom
feat/dashboard-json-artifacts
Sep 9, 2026
Merged

feat(dashboards): SQL-defined datasets, publish gate, list/load tools#346
8 commits merged into
stagingfrom
feat/dashboard-json-artifacts

Conversation

@kiwi0401

@kiwi0401 kiwi0401 commented Sep 8, 2026

Copy link
Copy Markdown
Member

Follow-up to #344/#345 on the same branch.

  • Datasets are defined by SQL only ({connection, sql} or static rows); the snapshot is the cached result written by sp.dashboard_dataset. The sandbox check tools flag snapshot_missing / snapshot_stale.
  • Publish runs each dataset's SQL once (row_limit 1) and rejects with 422 not_repeatable / sql_failed when it cannot feed the charts. This is what the failed staging refresh needed.
  • New sandbox tools dashboard_list_published and dashboard_load_published so a fresh chat can edit a published dashboard; the user still publishes (dialog preselects the matching dashboard).
  • Security: the sandbox run token could reach dashboard write routes; all write routes now refuse execution identities.
  • Screenshot projection parses past the runtime's [image] marker.

Verified live on the local staging stack (Dumpsters project): build with the helper, publish through the gate, SQL refresh to v2 with rows; then a fresh chat loaded and edited that dashboard without publishing.

Deployment: no new env vars; the staging notebook image rebuilds automatically on merge (notebook-server changed).

🤖 Generated with Claude Code

https://claude.ai/code/session_01XfZZecsk7TeJvfaLxD2d3g

kiwi0401 and others added 3 commits September 8, 2026 15:35
…ed result

A refresh failed in staging because the agent recorded a base query and then
derived the chart columns in pandas, so the recorded SQL could not reproduce
the dataset. A dataset is now {connection, sql} (or static rows); the
snapshot at artifacts/datasets/<name>.csv is written only by the new
sp.dashboard_dataset helper, which runs the query through the governed path
and records a sidecar with the SQL hash and columns.

- notebook-server: sp.dashboard_dataset; the check tools report
  snapshot_missing and snapshot_stale (SQL or columns differ from the
  snapshot); CSV-only snapshots; skill rewritten around SQL-only datasets
  with a TTM window and CASE example.
- gateway: publish runs each dataset's SQL once (row_limit 1) and rejects
  with 422 not_repeatable / sql_failed when the result cannot feed the
  charts; refresh drops file carry-forward; one query seam for gate and
  refresh; edit chats no longer inline dataset bytes.
- web: renderer and publish dialog use the new dataset shape, show
  "SQL · <connection>" or "Static", resolve snapshots by convention, and
  render the per-dataset gate errors; fixtures converted.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XfZZecsk7TeJvfaLxD2d3g
A fresh chat can now find and edit a published dashboard. Two sandbox tools,
dashboard_list_published and dashboard_load_published, read the gallery with
the run's identity and materialize artifacts/<slug>.dashboard.json plus its
snapshots and sidecars, so the file passes the freshness checks and renders
at once. Publishing stays with the user: the publish dialog preselects the
matching dashboard by slug and the file view shows "Loaded from".

Also: the sandbox run token could reach the dashboard write routes; every
write route now refuses execution identities. Edit chats no longer inline
the spec; they ask the agent to load it. The screenshot projection parses
past the runtime's "[image]" marker so the card shows rendered charts.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XfZZecsk7TeJvfaLxD2d3g
@vercel

vercel Bot commented Sep 8, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
signal-pilot Ready Ready Preview Sep 9, 2026 4:07pm UTC
signal-pilot-docs Ready Ready Preview Sep 9, 2026 4:07pm UTC

Request Review

kiwi0401 and others added 2 commits September 9, 2026 07:17
…book skill

The SDK query route parsed SQL with the default Postgres dialect before it
knew the connection, so valid T-SQL such as `select top 1` was rejected with
"Query blocked" while the MCP query_database path accepted it. The governed
executor already validates with the connection's dialect; the pre-check is
removed and covered by regression tests.

The notebook-specific instructions move out of the chat system prompt into
a new `signalpilot-dbt:notebook` skill (tools, sp SDK, cell rules, dialect
rule, files, charts, troubleshooting). The prompt keeps the basics and
points at the skill.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XfZZecsk7TeJvfaLxD2d3g
…ilters

Three defects an agent hit while building a dashboard on SQL Server:

- Limit injection on a T-SQL UNION/EXCEPT/INTERSECT wrapped the query as a
  derived table with its trailing ORDER BY inside, which SQL Server rejects
  (error 1033). The ORDER BY, OFFSET, and FETCH now move to the outer
  select; the cap is TOP, or FETCH when an OFFSET exists; a branch's own
  TOP stays on its branch; CTEs are hoisted. Covered by an 18-case matrix.
- The sandbox SDK client made one attempt, so a transient 502/503/504 from
  the tunnel aborted a dataset build. Requests and downloads now retry
  three times with backoff; other HTTP errors stay final.
- A dashboard filter could bind to only one dataset. `dataset` is now
  optional: an unbound filter applies to every dataset that has the column,
  so one control drives all charts sharing it. Bound filters are unchanged.
  Mirrored in the renderer, the sandbox checks, and the publish gate.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XfZZecsk7TeJvfaLxD2d3g
…in batches

Dashboards:
- KPI tiles accept text cells: a value or comparison without a numeric
  format renders the cell as text, so a top-region name comes from SQL
  instead of a hard-coded description. Same rule in the renderer, the
  sandbox checks, and the publish gate.
- New chart types: combo (grouped or stacked bars with lines on a secondary
  axis) and heatmap (category axes with a continuous colour scale), in the
  schema, renderer, print path, checks, and the plugin example.
- Skill: dataset reuse guidance, TOP/LIMIT in SQL for small snapshots, and
  the text-KPI rule.

Chat streaming: the worker persisted every streamed chunk as its own run
event with six database round trips, which on a remote database delivered
about one chunk per second. Deltas now coalesce per run and flush every
150 ms, at 2,000 characters, or before any other event (including the
terminal status), and append_event allocates the sequence with one atomic
UPDATE ... RETURNING. Measured on the local staging stack: 5 chars/s and
~900 events per answer became 214 chars/s and 9 events.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XfZZecsk7TeJvfaLxD2d3g
…itle

The analysis notebook no longer starts with placeholder scaffold cells
("Analysis output / Pending governed notebook analysis"). Every notebook
seeds with the hidden setup cells and one empty visible cell; the notebook
skill tells the agent to put a short title there as the first visible cell.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XfZZecsk7TeJvfaLxD2d3g
@kiwi0401 kiwi0401 closed this pull request by merging all changes into staging in c5a827b Sep 9, 2026
@kiwi0401
kiwi0401 deleted the feat/dashboard-json-artifacts branch September 9, 2026 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant