v0.3.0: server-side runner profiles and multi-review fusion - #20
Merged
Conversation
- Add "runnerProfiles" table (runner/model/effort/apiProvider/apiBaseURL/ token/params/isDefault) with a single-default partial unique index; seed one default claude/opus/xhigh profile - Add nullable "projects"."runnerProfileId" FK (null falls back to default) and a "reviews"."runnerProfile" jsonb snapshot column - Add RunnerProfileParams and ReviewRunnerProfile JSON types; the review snapshot deliberately excludes the token (secrets never persisted) - Regenerate db models/repos/test helpers via mfd; restore the hand-tuned IssueStatusFilter in review.go that regeneration reverted - Keep PGD, reviewsrv.sql and init.sql in sync with the pgmigrator patch
- Add RunnerProfileService (Count/Get/GetByID/Add/Update/Delete/Validate) in the project namespace, registered as the "runnerProfile" RPC; regenerate vt_zenrpc - Mask the token in the admin API: never return it raw (masked display + hasToken flag), set-or-keep semantics on update - Enforce a single default profile (unset the previous default on add/update) and validate runner/apiProvider/effort enum values - Wire runnerProfileId onto the Project DTO/search/summary with an FK existence check and a runnerProfile relation - Refresh project/review .vt.xml metadata via mfd-vt-xml
- Add Runner Profiles list + form pages (enum selects for runner/effort/ apiProvider, conditional direct/opencode fields) with nav + routes - Mask the token in the form (write-only set-or-keep, shows tokenMasked) - Wire a Runner Profile FKSelect onto the project form (— None — = default) and mask the projectKey with copy-to-clipboard on the project list/form - Use a VT-local RunnerProfileParams type so the TS client generates a clean interface; regenerate the vt/factory TS clients (restore @ts-nocheck)
- Add /v1/reviewctl/rpc/ (JSON-RPC 2.0): ReviewConfig(projectKey) resolves the project's runner profile or the global default and returns it (with the real token, CI-internal path); Prompt(projectKey) assembles the prompt - Add ProjectManager.RunnerProfile resolving pinned profile or default - Duplicate the review upload endpoints under /v1/reviewctl/upload/ - Snapshot the resolved profile onto the review: ReviewDraft carries a runnerProfile block persisted to reviews.runnerProfile (token excluded)
- Fetch ReviewConfig over /v1/reviewctl/rpc/ before building the runner; the profile fills runner/model/effort/apiProvider/apiBaseURL/params and explicit flags still override it (CI needs only image + creds + key + URL) - Resolve credentials env-first, then the profile token: pass it to the direct provider and export it to ANTHROPIC_API_KEY/OPENAI_API_KEY for the claude/codex CLIs when unset - Fetch the prompt via the same RPC (Prompt) instead of GET /v1/prompt - Upload via /v1/reviewctl/upload/ and snapshot the resolved profile onto the review (token excluded) - Update client tests for the JSON-RPC endpoints
- Remove the standalone GET /v1/prompt/ endpoint and its handler; the prompt is now served by /v1/reviewctl/rpc/ (Prompt) - Keep /v1/upload/ as deprecated aliases for older CI images - Update README: architecture, endpoint table, runner profiles, thin CI (credentials depend on the profile's runner), and internal URL gating
Drops a goconst finding introduced with the rpc client.
Replace the hand-rolled JSON-RPC handler (pkg/rest/reviewctl.go) with a real zenrpc service in pkg/reviewctl, mounted as a separate internal server at /v1/reviewctl/rpc/ — kept off the public SMD/api.ts since ReviewConfig returns the real runner-profile token. reviewctl now calls the rpcgen-generated Go client in pkg/reviewer/ctl/reviewctlclient instead of marshalling JSON-RPC by hand. Adds the -go_client flag and the `make go-client` target mirroring -ts_client.
Enforce a single default runner profile purely in code: soft-delete keeps isDefault set, so a partial-unique index would count a deleted profile as a live default and block setting any new one. Removes UNQ_runnerProfiles_isDefault from the patch/schema/dump; unsetCurrentDefault is now the only guarantee. Validate runner/provider against the canonical sources (runner.Runner* constants and the new direct.IsValidProvider, which exports the provider ids) so they can't drift. Adds a test that a new default can be set after the old default was deleted.
Dedupe the projectKey maskKey helper (duplicated in the project list and form) into a shared frontend/src/vt/format.ts. Add 'openai' to the runner-profile provider options, matching the new direct.ProviderOpenAI.
Local golangci-lint (v2.12.2) flags goconst that CI (v2.5.0) does not. Clean
fixes where a constant is the right call: reuse the existing Severity* constants
in calcIssueStats, add TrafficLight{Red,Yellow,Green}, and extract the API-key
env var names in reviewctl. The rest are silenced with explained //nolint:goconst
— validator tag names that only coincidentally equal FieldError* values, pg
column names, provider/model ids and API field names where a shared const adds
no clarity.
The Model field gets runner-aware suggestions (and provider-aware for the direct runner) via a new ComboInput — a free-text combobox with a theme-styled dropdown, since the native <datalist> popup renders unthemed (light) in dark mode. Also makes the API Base URL placeholder provider-specific and drops the effort levels a runner/provider can't use (no max for codex; an 'ignored' hint for deepseek/ openai-compat/opencode). Suggestion lists track ResolveDefaults and the codex.go / provider_factory.go price tables (2026-06).
- Add per-project panel and judge: projects.runnerProfileIds jsonb and judgeRunnerProfileId fk to runnerProfiles - Add review roles and member linking: reviews.reviewRole plus parentReviewId self-fk (single/member/fusion) - Add per-issue provenance: issues.sources jsonb - Add ReviewRole enum and IssueSources/ProjectRunnerProfileIDs jsonb types with Value/Scan JSON round-trip and a contract unit test - Restore hand-tuned IssueStatusFilter that mfd regen reverted - Regenerate mfd model/repo/db-test; sync PGD and SQL dump; add patch 2026-06-18-multi-review.sql
- Add --multi flag (runner:model panel members) that runs each member in its own detached git worktree, sequentially and fail-fast - Upload members as reviewRole=member via a runner factory injected into the Controller; --multi bypasses server config and uses ambient credentials, leaving the single-review path unchanged - Add ReviewRole to the upload draft and map empty to single in ToModel so single uploads keep the canonical role - Test the --multi parser, git-worktree add/remove, and reviewRole persistence through CreateReview
- Add FusionPrompt: the judge reads the staged panel members, keeps consensus, verifies singletons against the code, normalizes severity and tags each kept issue with its source models - Add --judge runner:model; reviewPanel runs the judge over the member worktrees and uploads one fused review (reviewRole=fusion) that links the members as children - Add MemberReviewIDs and per-issue Sources to the upload draft and map sources to db.Issue.Sources - Link members to their fusion via ReviewManager.LinkMembers, which sets parentReviewId - Degrade to a single review when the judge fails after a retry or only one member ran - Test sourceLabels dedup, fusion sources persistence and member linking
- Run panel members concurrently, bounded by panelConcurrency, adding the worktrees sequentially first since git worktree add is not safe to run concurrently against one repo - Tolerate per-member failures (log and skip) and require only one success instead of failing the whole panel on the first error - Add --timeout / $REVIEW_TIMEOUT (default 30m) bounding each member and the judge run, plus an EnvDuration helper - Test failure tolerance and panel ordering under the race detector
- Exclude reviewRole=member from default review lists and aggregations (ListReviews/CountReviews via ReviewSearch, ProjectsStats, FillLastVersions) so the fusion is the primary review; members stay reachable by direct id and opt-in via IncludeMembers/ParentReviewID - Rewrite LinkMembers as one project- and role-scoped UPDATE with a RowsAffected guard (no foreign, non-member, or already-parented ids) and run it inside the fusion-create transaction via CreateReviewWithMembers so a bad member id rolls the fusion back - Validate reviewRole, memberReviewIds (require fusion), and per-issue sources in ReviewDraft.Validate - Return the fusion panel breakdown (member children + summed panel cost) and per-issue sources from ReviewService.GetByID via a new ListMembers and rpc Review/PanelMember/Issue fields - Regenerate rpc zenrpc and the public TS client for the new fields
- Add ProjectManager.ReviewProfiles resolving a project's primary
runner, panel members (runnerProfileIds, order preserved, disabled
ids skipped) and judge (judgeRunnerProfileId)
- Reshape the reviewctl ReviewConfig RPC to return {primary, panel,
judge} with real tokens and add a FusionPrompt method serving the
built-in judge prompt; regenerate zenrpc and the go client
- Wire reviewctl to build cfg.Multi (the [primary] + panel run set)
and cfg.Judge from the server config when a judge is set; the judge
now fetches FusionPrompt over the RPC; --multi/--judge stay a local
override
- Pass the runner-profile token per-process via cmd.Env instead of a
global os.Setenv, so concurrent panel members with different tokens
no longer race (ambient env still wins)
- Carry each member's resolved profile in MemberSpec so it runs and
snapshots with its own credentials and settings
- Add tests for ReviewProfiles, the ReviewConfig/FusionPrompt RPC,
credEnv injection and the [primary] + panel assembly
Phase 5 regenerated the public TS client from rpc/model.go but the new fields carried multi-line Go doc comments, which the rpcgen TS codegen emits as a trailing // comment and leaks the extra lines as bare code — a syntax error. The same regen also dropped the file's // @ts-nocheck header. Both went unnoticed because only Go tests ran. - Collapse the Review/Issue multi-review field comments to single lines so the generated TS stays valid - Restore // @ts-nocheck in factory.generated.ts (the generator omits it) - Regenerate rpc_zenrpc.go and factory.generated.ts
- Expose runnerProfileIds + judgeRunnerProfileId on the VT Project model, converter and ToDB, plus the judge summary relation - Validate the judge FK and each panel member FK in code, like the existing runnerProfileId - Add an ordered FKListEditor component (add/remove/reorder, duplicates allowed) and wire the project form: relabel the runner profile "Primary / fallback", add the panel members list and a nullable Judge select, with a hint when members are set but no judge is chosen - Regenerate vt_zenrpc and the VT TS client for the new fields - Add a VT validation test for the new panel FKs
- Fold the repeated per-member config setup (clone, runner/model, clear panel fields, profile overlay) into one Controller.memberConfig, replacing applyMemberProfile and its two duplicated call sites - Extract newReviewFileSummaries, shared by the review-list row and the fusion panel breakdown instead of two identical loops - Refresh the stale FusionPrompt comment: it is served via rpc and the CI client fetches it, so the constant is the single source of truth
- Add a PanelBreakdownCard on the fusion review page listing each member child (traffic light, model, issue count, cost, link to the member review) with panel/judge/total cost - Add a ProvenanceBadge and render an issue's sources in the issue detail — the model labels that flagged it (or "judge" for a verified net-new) plus an agreement count when more than one model concurred - Re-export the PanelMember type from the public api client
- Public review: show the fusion panel breakdown in a dedicated "Panel" tab (after Issues) instead of an always-expanded card, so it opens on demand and the header stays compact; the tab appears only for fusion reviews - VT project form: group Runner Profile / Panel / Judge into a new "Runner" tab between General and Instructions, decluttering General
- Clearer for an end user that the review was synthesized from several models; the inner "Panel breakdown" heading and the Panel/Judge cost split keep "Panel" where it precisely means the member set
- Replace the OS-native select arrow with a single themed chevron via a shared .app-select class (appearance:none + an SVG background that follows --color-fg-subtle in light/dark) - Apply it to every select: PSelect (public), VSelect, FillExampleSelect and the FKListEditor add dropdown (VT)
- Extract applyRunResult to set model/runner/timing metadata and the resolved profile snapshot, replacing the identical block that was duplicated across the single review, panel members and the judge - Remove the unused write-only memberOutput.spec field - Collapse the repeated server-profile flag overrides in applyReviewConfig into a serverDefault helper - Hide the local --multi/--judge debug flags so they keep working but do not become a stable CLI contract; multi-review is profile-driven
- Reframe reviewctl flags as local overrides; runner config comes from the project runner profile fetched at run time - Add the missing runner/effort/api/timeout/continue flags and fix the --author env and --model default in the reviewctl flag table - Mark the LLM API key optional in CI now the profile may carry a token - Document the panel + judge fusion multi-review flow - Note ReviewConfig returns the panel and add FusionPrompt
- Collapse the runner-specific tabs in the local-run dialog into one script; the runner and model now come from the project profile - Rebuild the CI Dockerfile helper to layer the latest reviewctl release on the vmkteam/claude-ci base instead of inlining settings
- Make PROJECT_KEY and REVIEWSRV_URL the only required CI vars; LLM credentials now come from the server-side runner profile or, as a fallback, REVIEW_API_KEY / provider-specific keys - Document the dockerimage as the claude-ci + reviewctl combined image and default it to a registry placeholder - Annotate the MR session cache as a Claude-runner prompt-cache reuse that is a safe no-op for the opencode/codex/direct runners
- Add gpt-5.6 family (sol/terra/luna + bare alias) to the codex price table for token-based cost estimation - Fix claude-fable pricing in the direct runner: split from the claude-opus case, $10/$50 per MTok with matching cache rates - Add claude-opus-5, claude-sonnet-5 and gpt-5.6 tiers to runner profile model suggestions in the VT admin form - Extend pricing tests to cover opus-5, sonnet-5 and gpt-5.6
- Add http_fetch tool locked to the task tracker origin and path:
GET-only, same-scope redirects, 100KB clip, JSON/text bodies only,
responses wrapped as untrusted data; the Authorization header is
derived from the token shape (email:secret → Basic for Jira Cloud,
anything else → Bearer, so YouTrack perm tokens stay intact)
- Keep the tracker token out of assembled prompts: {{TOKEN}} renders
as $REVIEW_TRACKER_TOKEN, reviewctl exports it to CLI runner
processes per-process (ambient env wins) and the REVIEW_TRACKER_TOKEN
CI variable overrides the server-stored token
- Serve tracker URL/token out-of-band via the ReviewConfig RPC;
ReviewProfiles now returns a domain ReviewSetup (profiles + tracker)
in a single project load; drop the unused RunnerProfile resolver
- Preserve backward compatibility with an optional tokenEnv flag on
the Prompt RPC: legacy reviewctl clients keep receiving the
old-style prompt with the real token substituted in
- Surface direct-runner tool errors in the runner log at warn level
- Document the runner-agnostic fetchPrompt example and tracker access
in VT.md, Model.md, README and the CI template; regenerate zenrpc
bindings and the reviewctl client
- Raise Anthropic output cap to 64k and handle truncated rounds:
notify the model, reject empty add_issues batches, abort after 3
truncated rounds in a row (after running surviving tool calls)
- Preserve the prompt cache: keep the compaction head byte-identical,
defer compaction on 1M-context providers (direct.DefaultCompactAt),
clip grep output and matched lines (rune-safe), emit compact events
- Hide reviewer artifacts from diffs, walks and preload via a canonical
list in pkg/reviewer; root-level only so the judge's staged
members/<label>/ files stay visible; clean review.html before runs
- Detect the diff base from origin/HEAD for local runs without CI vars
- Fix tracker API calls returning SPA HTML: trim trailing slash from
tracker URL on write and in {{URL}} substitution, normalize http_fetch
paths with path.Clean before the scope check (also blocks ".."
escapes); refuse direct reads of .claude/ local agent state
- Panel: treat an untouched-skeleton member review as failure, upload a
debug bundle before the worktree is cleaned, retry an empty judge
review; pin opencode to its worktree with --dir (it escaped to the
main repo via the shared git dir)
- Persist run metadata into review.json, clear placeholder fields and
fall back to git metadata for local runs
- vt admin: write-only secrets with set-or-keep (nil or blank keeps the
stored value), SecretInput wraps VInput and owns the "Saved" hint,
move @ts-nocheck from Makefile perl into the TS client generator
- Install the arch-aware static-pie ast-index release in the VT CI Setup Dockerfile so the direct runner's ast_* tools work in CI - Pin the version via ARG and verify the binary at build time
- Move go.mod, workflows and the Docker builder image to Go 1.26 - Pin golangci-lint-action and LINT_VERSION to v2.12.2: CI ran v2.5.0, which flags prealloc issues the current linter dropped, so PR lint failed while local runs were clean
- go get -u direct deps: anthropic-sdk-go v1.61.0, zenrpc v2.3.3, goldmark v1.8.5, echo v4.15.4, prometheus client v1.24.1, rpcgen v2.5.5, sentry-go v0.44.0 and transitive updates - Keep sentry-go at v0.44.0: v0.45+ moves the echo integration to echo/v5, which breaks appkit's echo/v4 middleware - Refresh vendor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
runner_profileentity (docs/patches/2026-06-13-runner-profiles.sql,pkg/db) holding runner, model, effort, API provider/base URL, token and params (allowDangerousPermissions); projects reference a profile. VT admin CRUD (pkg/vt/runner_profile*.go) with hardened validation, plus admin UI (RunnerProfilesPage/RunnerProfileFormPage) with model autocomplete (ComboInput) and masked token input (SecretInput,maskKeyinvt/format— now covers theopenaiprovider).pkg/reviewctl, mounted on the CI-only/v1/reviewctl/path, never on the public RPC server): returns the resolved runConfig(incl. the real profile token — env vars still win on the client),TrackerConfigand the assembled prompt.reviewctlis now driven by the server-side profile instead of CLI flags; typed client generated via rpcgen (pkg/reviewer/ctl/reviewctlclient/client.generated.go), debug flags hidden.docs/patches/2026-06-18-multi-review.sql), per-project panel editor in the admin (config and breakdown moved into tabs,FKListEditor).reviewctlfans the panel out into per-member git worktrees (pkg/reviewer/ctl/panel.go) — worktrees created sequentially, members reviewed in parallel (capped at 4), per-member failures tolerated; a judge stage fuses member outputs into one review (pkg/reviewer/fusion_prompt.go). Server filters/links member reviews to the fusion review; UI shows the panel breakdown (PanelBreakdownCard.vue) and per-issue provenance badges (ProvenanceBadge.vue); tab named Multi-review.http_fetchto the direct runner (pkg/reviewer/direct/tools_http.go): GET-only, scoped to the project task-tracker base URL, auth header inferred from token shape, token delivered out-of-band ({{TOKEN}}→$REVIEW_TRACKER_TOKEN) so it never appears in the prompt.pkg/vt/gitlab-review.yml.tmpl): the runner profile supplies the token;REVIEW_API_KEYor provider-specific vars are only needed when the profile has none, and reviewctl fails fast when no credential is available from either source.94136f9and friends): opencode worktree escape fixed via--dir, empty member output → review marked failed + debug bundle uploaded, direct-runner fixes; dropped the single-default profile index.Test plan
make fmt lint test— greenreviewctl reviewwith no runner/model flags — pulls its config from/v1/reviewctl/, runs with the profile's runner/model/efforthttp_fetchreads the MR's tracker issue; the tracker token is absent from the prompt and transcriptANTHROPIC_API_KEYset; fails fast when neither profile nor env provides a credential