Let creative commissions pin an image/video render backend (local vs. Grok vs. Codex)

[atomantic]

Summary

Which backend actually rendered a creative commission's image/video was an accidental side effect. The Creative Director's planner LLM authors media_enqueueImageJob/media_enqueueVideoJob params freehand, it never set params.mode, and mediaJobQueue/index.js#getGenModuleForJob dispatches the backend purely on that key — so every plan-driven commission silently rendered on local diffusion no matter what the user wanted.

A commission can now pin its render backend:

• Schema — ability-scoped generation.imageMode / videoMode (+ optional imageModelId / videoModelId) as GENERATION_KEY_DEFS descriptors, so bounds/defaults stay single-sourced and the existing per-ability superRefine rejects an off-type pin (videoMode on an image commission is a 400, not a silent drop). imageMode on image, videoMode on video, both on music-video; music/series carry neither. A new type: 'id' descriptor kind covers the nullable free-string model ids.
• Forcing site — the enqueue happens inside the planner's tool call, which only sees ctx.projectId and has no reference back to the commission. So the pin rides the CD project: buildRenderBackendPin → buildProjectParams → buildProjectRecord's new renderBackend field, and a new enforceRenderBackendPin step in creative/tools/media.js injects the resolved mode + provider knob bundle into the job params after the planner authored them. It runs parallel to enforceVideoRenderPreset and shares one project read (previously the video path read the project itself; that read is hoisted to loadOwningProject). Side benefit: any plan-driven CD project now has the same knob, not just commission-spawned ones.
• UI — a Render backend section in CommissionConfigForm.jsx (mode selector + conditional local-model picker), mirroring the VisualGenSettings.jsx UX. It renders only for abilities that declare backend fields, so a music/series commission never shows it and fetches no model catalogs.
• VIDEO_GEN_MODE formalized — new server/services/videoGen/modes.js, deriving its values from the IMAGE_GEN_MODE constants the video lane already discriminates on inline, plus isVideoModeUsable/resolveVideoMode. The issue left this as the implementer's call; an enumerable alphabet was cheaper than hand-copying ['local','grok'] into a schema, a picker and a resolver.

auto is a strict no-op

The hard acceptance criterion. An auto/absent pin means buildRenderBackendPin returns null, so enforceRenderBackendPin returns params untouched (not "auto-resolved to something") without even reading settings — existing commissions enqueue byte-identical params and fall through to the install-wide default exactly as before. Covered by tests at each layer. renderBackend is a new null field on newly created projects, the same additive shape this record already uses for musicBed/plan/directive (verbatim JSONB round-trip); existing projects are never rewritten and sanitizeProjectForSync fabricates nothing, which a test locks in.

The pin genuinely overrides the planner

Three things the first draft got wrong and review caught — each is the difference between a pin that holds and one that politely defers to the LLM it exists to override:

• A pinned local model beats a planner-authored one. Applying pin.modelId only when the planner named none meant the model the user picked in the form lost to the planner's guess. An unpinned model still leaves the planner's alone.
• A local pin strips any backend token from params.mode. That key is the t2v/i2v semantic on the local lane (fflf, a2v, extend, an IC-LoRA id), which a pin must not overwrite — but the backend discriminator shares it, so a planner-written 'grok' would dispatch to grok in defiance of the pin (and defeat the grok-disabled fallback), and 'local' is worse than useless (local.js reads an unrecognized non-empty mode as plain text-to-video and would ignore the step's sourceImagePath/keyframes). Dropping the key entirely repairs both: an unset mode makes local.js infer the semantic.
• A grok video pin translates the clip length. The local lane derives frames from durationSeconds; grok's worker reads duration and silently falls back to its 6s minimum for anything absent — so a 10s commission rendered a 6s clip. Adds nearestGrokDuration to lib/grokVideoClip.js, which rounds up to the shortest deliverable clip. Deliberately separate from the existing resolveGrokDuration: that one validates an already-grok-shaped request and defaults a third value to 6s, which is right at the route boundary but would truncate an 8s brief here.

Other deliberate decisions

• Video defaults to local, not "prefer an enabled cloud backend" (the image ladder's rule). A Grok clip spends remote quota and only delivers 6s or 10s, so enabling Grok for images must not silently redirect every unpinned video render to a paid backend.
• A pin is re-resolved against live settings each fire, never trusted verbatim. A cloud toggle the user flips off long after pinning degrades to a usable backend rather than failing every nightly fire.
• creativeCommissions 2 → 3. generation is a federated field and a v2 peer's ABILITY_GENERATION_KEYS doesn't list these keys, so its sanitizer would strip the user's pinned backend on receive and could push the un-pinned downgrade back via LWW — the exact failure mode Creative Commissions: let the user pick the creative output type and tailor params + directive per type #2769 bumped 1 → 2 for.
• No migration. Purely additive optional keys with an auto default that sanitizeCommission fills on every read, so no on-disk record needs rewriting. renderBackend is likewise additive on the CD project, so no creativeDirectorProjects bump.
• Out of scope, documented in-code: generation.model and the legacy sceneRunner.js scene flow are untouched, and the cleanC2PA/denoise post-processing settings stay unresolved on this path — resolving them only when a backend is pinned would make the same commission post-process differently depending on whether a pin exists, and fixing it for both paths would break the "auto is byte-identical" contract.

Test plan

• cd server && npx vitest run — 22630 passed. 2 failures (lib/dataRoot, services/installState), both pre-existing worktree-path artifacts that pass in the primary checkout; neither touches a file in this diff.
• cd client && npm test — 469 files / 5185 tests passed.
• New: server/services/creative/tools/media.test.js (23 tests) — the forcing step end to end via the real tool execute, asserting the params that reach enqueueJob: untouched for unpinned/no-project/unreadable-settings, codex/grok pins forced over a planner-authored mode, a disabled backend degrading (including when the planner already wrote 'grok'), both backend tokens stripped from a local video pin while a real semantic survives, the pinned model winning, grok duration translated from the step / project target / an explicit value, both forcing steps composing, an image pin not leaking into a video enqueue, and audio never reading the project or settings at all.
• New: server/services/videoGen/modes.test.js (12 tests) — enum derived from the queue discriminator, the strict-boolean grok gate, and the local-not-cloud default.
• Extended: lib/grokVideoClip.test.js (nearestGrokDuration rounding/clamping/fallbacks, and the deliberate divergence from resolveGrokDuration at 8s), abilityAdapters.test.js (pin builder incl. every no-pin path), store.test.js (sanitize round-trip, blank-model normalization, legacy record reads as auto, hand-edited garbage falls back), projectsLogic.test.js (normalizeRenderBackend, buildProjectRecord default, and no key fabricated onto an existing/synced record), routes/creativeCommissions.test.js (schema accept/reject incl. off-ability and unknown-backend 400s), commissionForm.test.js (per-ability field declaration, payload emission, stale-model-id nulling).

Closes #3135