Skip to content

Commit ce9ce18

Browse files
vtkovapiclaude
andcommitted
feat: managed variables in the state file
Add a `variables` section to `.vapi-state.<env>.json` for centralizing values that repeat across resources — callback URLs, a default model name, shared prompt fragments. Resource files reference them with whole-value `{{name}}` placeholders. - Forward (push): `resolveVariables` replaces `{{name}}` with the managed value, native type preserved, wired into `resolveReferences` BEFORE resourceId→UUID resolution so a variable can yield a reference. - Drift stays clean: `hashLocalResource` renders variables before hashing; the platform side is already rendered, so both hash in one basis (no phantom drift). `canonicalizeForHash` is intentionally unchanged. - Pull preserves templates: `restoreVariablePlaceholders` re-inserts a placeholder only where the local file already had it and the value still matches (guided reverse → zero false positives), on both the main and `--resolve=theirs` write paths. - `extractReferencedIds` is variable-aware so dependency auto-creation, orphan/delete protection, and ignored-reference validation see the real resourceId behind a `{{placeholder}}`. - Validation: undefined `{{name}}` is a blocking finding in push + validate. - State plumbing: `variables` is exempted from the migration guard and the `npm run migrate` slim-rewrite (or it would trip the legacy-format check and be dropped); loaded via `normalizeVariables`. `serializeState` preserves hand-authored object-value key order on save. - Whole-value only (no in-string interpolation), per design. Docs: docs/learnings/variables.md + index tables + improvements.md (#27). Tests: variables / state-variables / resolver-variables (39 specs). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent c9ffafb commit ce9ce18

23 files changed

Lines changed: 1107 additions & 23 deletions

AGENTS.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ This project manages **Vapi voice agent configurations** as code. All resources
3434
| Enforcing call time limits / graceful call ending | `docs/learnings/call-duration.md` |
3535
| Voice provider field cheat-sheet (Cartesia vs 11labs vs OpenAI etc.) | `docs/learnings/voice-providers.md` |
3636
| YAML authoring conventions, .vapi-ignore lifecycle | `docs/learnings/yaml-conventions.md` |
37+
| Centralizing repeated values with `{{variables}}` | `docs/learnings/variables.md` |
3738
| What pull/push/apply do in every drift & existence scenario | `docs/learnings/sync-behavior.md` |
3839

3940
**Where new knowledge goes:**
@@ -202,7 +203,9 @@ docs/
202203
├── voicemail-detection.md # Voicemail vs human classification
203204
├── call-duration.md # Call time limits and graceful end-of-call
204205
├── voice-providers.md # Per-provider voice block field cheat-sheet
205-
└── yaml-conventions.md # YAML authoring conventions, .vapi-ignore lifecycle
206+
├── yaml-conventions.md # YAML authoring conventions, .vapi-ignore lifecycle
207+
├── variables.md # Managed {{variables}} in the state file
208+
└── sync-behavior.md # pull/push/apply scenario matrix
206209
207210
resources/
208211
├── <org>/ # Org-scoped resources (npm run push -- <org> reads here)

CLAUDE.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,7 @@ See `docs/learnings/voice-providers.md` for related "property X should not exist
6565
- Call time limits / graceful ending → `docs/learnings/call-duration.md`
6666
- Voice provider field cheat-sheet → `docs/learnings/voice-providers.md`
6767
- YAML authoring conventions, .vapi-ignore lifecycle → `docs/learnings/yaml-conventions.md`
68+
- Managed `{{variables}}` in the state file → `docs/learnings/variables.md`
6869
- Pull/push/apply behavior per drift & existence scenario → `docs/learnings/sync-behavior.md`
6970

7071
This list mirrors the "Learnings & recipes" table in `AGENTS.md`. Keep both in sync — if you add a new learnings file, update both files plus `docs/learnings/README.md`.

docs/learnings/README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@ Each file targets a specific topic so you can load only the context you need.
2828
| Enforcing call time limits / graceful call ending | [call-duration.md](call-duration.md) |
2929
| Voice provider field cheat-sheet (Cartesia vs 11labs vs others) | [voice-providers.md](voice-providers.md) |
3030
| YAML authoring conventions, .vapi-ignore lifecycle | [yaml-conventions.md](yaml-conventions.md) |
31+
| Centralizing repeated values with `{{variables}}` | [variables.md](variables.md) |
3132
| What will pull/push/apply do in situation X? | [sync-behavior.md](sync-behavior.md) |
3233

3334
---
@@ -80,3 +81,4 @@ How the gitops sync engine itself behaves:
8081
|------|----------------|
8182
| [sync-behavior.md](sync-behavior.md) | The full pull/push/apply scenario matrix: state file vs hash-store baseline vs dashboard, drift directions (clean / local-ahead / dashboard-ahead / both-diverged), per-resource conflict prompt, existence cases (local-only file, dashboard-only resource, deletions either side, fresh clone, renames, legacy-state migration), `.bkp` backup copies, flag cheat sheet |
8283
| [yaml-conventions.md](yaml-conventions.md) | YAML authoring conventions, `.vapi-ignore` lifecycle |
84+
| [variables.md](variables.md) | Managed `{{variables}}` in the state file: whole-value substitution at push, placeholder restoration at pull, type preservation, drift-stays-clean, validation of undefined names |

docs/learnings/variables.md

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
# Managed variables
2+
3+
Centralize values that repeat across resources — callback URLs, a default model
4+
name, a shared prompt fragment, a brand name — in one place, and reference them
5+
from any resource file with a `{{name}}` placeholder. At push the placeholder is
6+
replaced with the managed value; at pull the placeholder is preserved.
7+
8+
---
9+
10+
## TL;DR
11+
12+
1. Add a `variables` section to the state file `.vapi-state.<env>.json`:
13+
```jsonc
14+
{
15+
"assistants": { "...": { "uuid": "..." } },
16+
"variables": {
17+
"callback_url": "https://acme.com/vapi/webhook",
18+
"default_model": "gpt-4.1",
19+
"max_tokens": 260
20+
}
21+
}
22+
```
23+
2. Reference a variable anywhere in a resource file with a **whole-value**
24+
placeholder:
25+
```yaml
26+
model:
27+
model: "{{default_model}}"
28+
maxTokens: "{{max_tokens}}" # becomes the NUMBER 260, not "260"
29+
server:
30+
url: "{{callback_url}}"
31+
```
32+
3. `npm run push -- <env>` substitutes the values. `npm run pull -- <env>`
33+
restores the `{{...}}` placeholders. Drift detection treats a templated file
34+
and its rendered platform resource as **in sync** — no phantom drift.
35+
36+
---
37+
38+
## Rules
39+
40+
- **Whole-value only.** A placeholder substitutes only when the *entire* value
41+
is a single `{{name}}`. Embedded placeholders inside a longer string
42+
(`"Call us at {{callback_url}}"`) are **not** substituted — they ship
43+
verbatim. In-string interpolation is intentionally unsupported because it
44+
cannot round-trip cleanly through pull/drift.
45+
- **Type is preserved.** The managed value keeps its JSON type. `"{{max_tokens}}"`
46+
with `max_tokens: 260` sends the number `260`; an object-valued variable sends
47+
the object. (YAML needs the placeholder quoted, but the result is the native
48+
type.)
49+
- **Variable names** are `[A-Za-z0-9_.-]+` — letters, digits, underscore, dot,
50+
hyphen. No spaces. Whitespace *inside* the braces is ignored: `{{ x }}` ==
51+
`{{x}}`.
52+
- **Undefined placeholders are a validation error.** `npm run validate`/`push`
53+
fail (or warn, without `--strict`) on any `{{name}}` with no matching entry in
54+
`variables`, so a typo can't silently ship the literal string `"{{name}}"`.
55+
- **Variables compose with references.** Substitution runs *before* resourceId →
56+
UUID resolution, so a variable whose value is a resourceId works:
57+
`toolIds: ["{{tool_ref}}"]` with `tool_ref: "my-tool"` resolves to the tool's
58+
UUID.
59+
60+
## How it round-trips (push / pull / drift)
61+
62+
The state file is the single source of values. push and pull never mutate the
63+
`variables` section — you hand-edit it; the engine preserves it verbatim.
64+
65+
- **push** — `{{name}}` → value, then the usual reference/credential resolution,
66+
then the API call.
67+
- **drift** — the local file is hashed *with variables rendered*, and the
68+
platform resource is already rendered, so both sides hash in one basis. A
69+
templated resource you haven't changed reads as `clean`, never `both-diverged`.
70+
- **pull** — the platform value is written back, but a `{{name}}` placeholder is
71+
restored at any path where **your local file already had that placeholder and
72+
the value still matches**. This is guided by your file, so a literal value that
73+
merely *happens* to equal a variable's value is never rewritten into a
74+
placeholder. If a field was changed on the dashboard so its value no longer
75+
matches the variable, pull writes the literal value (the dashboard is now the
76+
source of truth for that field) — re-apply the placeholder by hand if you want
77+
it back.
78+
79+
## Limitations (v1)
80+
81+
- **No in-string interpolation** — whole-value only (see Rules).
82+
- **No nested/recursive variables** — a variable whose value itself contains a
83+
`{{...}}` placeholder is not re-resolved. Single pass.
84+
- **No `${ENV_VAR}` expansion** — variable values are literal. For per-developer
85+
secrets use `.env.<env>` (loaded into `process.env`), not the committed state
86+
file.
87+
- **Do not store secrets in `variables`.** The state file is committed to git.
88+
Tokens, API keys, and signing secrets belong in credentials or `.env.<env>`.
89+
- **`.ts` resources** keep their authored value (placeholder restoration on pull
90+
only applies to `.yml`/`.yaml`/`.md`), consistent with how `.ts` files are
91+
hashed.
92+
- **Avoid `null`-valued variables.** Null leaves are dropped by the hash
93+
canonicalization, so a `null`-valued placeholder may not round-trip cleanly
94+
through pull/drift. Prefer a sentinel value, or omit the field.
95+
- **Templating an entire `.md` system prompt** as a single `{{var}}` works for
96+
the normal case (one system message). Placeholder restoration on pull matches
97+
model messages positionally, so if the platform returns the messages in a
98+
different order than your local file, the literal rendered prompt is written
99+
instead of the placeholder — re-apply by hand if that happens.
100+
101+
## Why the state file (and not a separate file)?
102+
103+
Variables are the value analogue of the `name → { uuid }` reference maps the
104+
resolver already consults. Keeping them in `.vapi-state.<env>.json` means one
105+
lookup table, one commit, one merge surface. The migration guard
106+
(`assertStateMigrated`) and `npm run migrate` explicitly exempt the `variables`
107+
key, so its raw values are never mistaken for the legacy fat-state shape and
108+
`migrate` never drops them.

improvements.md

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1298,6 +1298,53 @@ as a transactional deploy rollback until create/delete/state coverage exists.
12981298

12991299
---
13001300

1301+
## 27. Migration seam assumes every top-level state key is a `{ uuid }` section
1302+
1303+
### Problem
1304+
1305+
`assertStateMigrated` and `migrateOne` (`src/migrate-hash-store.ts`) treat
1306+
**every** top-level object in the state file as a `name → { uuid }` section.
1307+
Any section that legitimately holds non-uuid values trips this.
1308+
1309+
### Current behavior
1310+
1311+
The managed-variables feature added a `variables` section holding raw values.
1312+
Without special-casing, `assertStateMigrated` reads each variable value, finds
1313+
it isn't exactly `{ uuid }`, and throws "legacy format" — blocking every
1314+
push/pull. Worse, `migrateOne` would call `uuidOf()` on each value, find none,
1315+
and **drop the entire section** on the next `npm run migrate`. Both were fixed
1316+
by exempting the `variables` key by name (`src/migrate-hash-store.ts:99`,
1317+
`src/migrate-hash-store.ts:177`), and `loadState` loads it via
1318+
`normalizeVariables` instead of `migrateSection` (`src/state.ts`).
1319+
1320+
### Risk
1321+
1322+
The exemption is **by literal key name**. The next contributor who adds another
1323+
non-uuid top-level section (e.g. a future `settings` or `defaults` block) will
1324+
hit the exact same silent-drop / false-legacy trap unless they remember to add
1325+
another `=== "section-name"` guard. The failure mode for the drop is silent.
1326+
1327+
### Current mitigation
1328+
1329+
`variables` is exempted and covered by `tests/state-variables.test.ts` (guard
1330+
does not trip; `migrateAll` preserves the section). `docs/learnings/variables.md`
1331+
documents the constraint.
1332+
1333+
### Possible fix
1334+
1335+
Replace the by-name guards with a structural rule: a top-level value is a
1336+
"uuid-section" only if every entry is string-or-`{ uuid }`-shaped; otherwise
1337+
treat it as opaque and preserve verbatim. Or maintain an explicit
1338+
`NON_UUID_SECTIONS` allow-list in one place that both the guard and the
1339+
migration consult.
1340+
1341+
### Status
1342+
1343+
**Partially mitigated** (`variables` handled, 2026-06-24). The general
1344+
brittleness — by-name exemptions in two functions — remains open.
1345+
1346+
---
1347+
13011348
## Out of scope (intentionally not improvements)
13021349

13031350
- **State file is identity-only and not git-ignored.** It's intentionally

src/audit.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -377,7 +377,7 @@ function checkContentDrift(
377377
const entry = state[type][resourceId];
378378
if (!entry) continue;
379379

380-
const localHash = hashLocalResource(type, resourceId);
380+
const localHash = hashLocalResource(type, resourceId, state.variables);
381381
if (!localHash) continue;
382382

383383
const remoteResource = remoteByUuid.get(entry.uuid);

src/delete.ts

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ import type {
1010
ResourceState,
1111
ResourceType,
1212
StateFile,
13+
Variables,
1314
} from "./types.ts";
1415

1516
// ─────────────────────────────────────────────────────────────────────────────
@@ -79,11 +80,18 @@ export function findReferencingResources(
7980
targetId: string,
8081
targetType: ReferenceableType,
8182
allResources: LoadedResources,
83+
variables: Variables = {},
8284
): ResourceReference[] {
8385
const referencingResources: ResourceReference[] = [];
8486

8587
const checkResource = (resource: ResourceFile, resourceType: string) => {
86-
const refs = extractReferencedIds(resource.data as Record<string, unknown>);
88+
// Pass variables so a reference via `{{placeholder}}` is resolved to the
89+
// real resourceId — otherwise a still-referenced resource looks
90+
// unreferenced and becomes eligible for deletion.
91+
const refs = extractReferencedIds(
92+
resource.data as Record<string, unknown>,
93+
variables,
94+
);
8795

8896
if (targetType === "tools" && refs.tools.includes(targetId)) {
8997
referencingResources.push({
@@ -375,6 +383,7 @@ export async function deleteOrphanedResources(
375383
orphan.resourceId,
376384
refType,
377385
loadedResources,
386+
state.variables,
378387
);
379388
if (refs.length > 0) {
380389
blocked.push({ ...orphan, refs });

src/drift.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -154,7 +154,7 @@ export async function checkDriftForUpdate(options: {
154154
// (rare on an update path), fall back to the baseline so the direction is
155155
// dashboard-ahead rather than a phantom both-diverged.
156156
const localHash =
157-
hashLocalResource(resourceType, resourceId) ?? baseline;
157+
hashLocalResource(resourceType, resourceId, state.variables) ?? baseline;
158158

159159
// Local and platform are byte-identical → there is nothing to reconcile and
160160
// the PATCH is a no-op. NEVER block here, even if the baseline disagrees

src/migrate-hash-store.ts

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,13 @@ async function migrateOne(
9696
const slim: Record<string, Record<string, { uuid: string }>> = {};
9797

9898
for (const [sectionKey, section] of Object.entries(raw)) {
99+
// `variables` is a hand-authored name→value map, NOT a uuid-section.
100+
// Preserve it verbatim — treating it as a section would try to read a
101+
// `.uuid` off every value, find none, and DROP the whole section.
102+
if (sectionKey === "variables") {
103+
(slim as Record<string, unknown>)[sectionKey] = section;
104+
continue;
105+
}
99106
if (!isSection(section)) {
100107
// Preserve any non-section top-level value verbatim (none expected,
101108
// but don't silently drop unknown shapes).
@@ -172,7 +179,10 @@ export function assertStateMigrated(stateFilePath: string): void {
172179
return;
173180
}
174181

175-
for (const section of Object.values(raw)) {
182+
for (const [sectionKey, section] of Object.entries(raw)) {
183+
// `variables` holds raw values (not `{ uuid }`); its entries would all
184+
// read as "legacy" and falsely trip the guard. It is never legacy.
185+
if (sectionKey === "variables") continue;
176186
if (!isSection(section)) continue;
177187
for (const value of Object.values(section)) {
178188
if (isLegacyEntry(value)) {

src/pull.ts

Lines changed: 35 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -37,11 +37,13 @@ import {
3737
import {
3838
FOLDER_MAP,
3939
hashLocalResource,
40+
readLocalResourceData,
4041
resolvePullScopeFromFilePaths,
4142
} from "./resources.ts";
4243
import { extractBaseSlug, isBackupCopyFile, slugify } from "./slug-utils.ts";
4344
import { hashPayload, loadState, saveState, upsertState } from "./state.ts";
4445
import type { ResourceState, ResourceType, StateFile } from "./types.ts";
46+
import { restoreVariablePlaceholders } from "./variables.ts";
4547

4648
// Map resource types to their API endpoints
4749
const ENDPOINT_MAP: Record<ResourceType, string> = {
@@ -652,7 +654,11 @@ export async function pullResourceType(
652654
const localFile = findLocalResourcePath(folderPath, resourceId);
653655

654656
if (localFile && baseline) {
655-
const localHash = hashLocalResource(resourceType, resourceId);
657+
const localHash = hashLocalResource(
658+
resourceType,
659+
resourceId,
660+
state.variables,
661+
);
656662
if (localHash) {
657663
// Use canonicalizeForHash so platform-default mutation (_platformDefault)
658664
// and the 3-step pipeline are applied identically across pull-write,
@@ -820,6 +826,18 @@ export async function pullResourceType(
820826
// replace credential UUIDs) plus the _platformDefault marker.
821827
const withCredNames = canonicalizeForHash(resource, state, credReverse);
822828

829+
// Guided variable restoration: re-insert `{{name}}` placeholders ONLY where
830+
// the existing local file already had them and the platform value still
831+
// matches the managed value. Hashing is unaffected — hashLocalResource
832+
// renders these back to values — so this is purely about keeping the
833+
// author's templates intact instead of clobbering them with literals.
834+
// No-op when there are no variables or no local file.
835+
const toWrite = restoreVariablePlaceholders(
836+
withCredNames,
837+
readLocalResourceData(resourceType, resourceId),
838+
state.variables,
839+
) as Record<string, unknown>;
840+
823841
if (bootstrap) {
824842
const icon = isPlatformDefault ? "🔒" : isNew ? "✨" : "📝";
825843
console.log(
@@ -830,7 +848,7 @@ export async function pullResourceType(
830848
const filePath = await writeResourceFile(
831849
resourceType,
832850
resourceId,
833-
withCredNames,
851+
toWrite,
834852
);
835853
const icon = isPlatformDefault ? "🔒" : isNew ? "✨" : "📝";
836854
const relPath = relative(BASE_DIR, filePath);
@@ -861,7 +879,7 @@ export async function pullResourceType(
861879
// next operator to find.
862880
const diskHash = bootstrap
863881
? null
864-
: hashLocalResource(resourceType, resourceId);
882+
: hashLocalResource(resourceType, resourceId, state.variables);
865883
if (!bootstrap && diskHash === null) {
866884
console.warn(
867885
` ⚠️ ${resourceType}/${resourceId}: failed to hash post-write disk form; falling back to in-memory hash (may produce phantom drift on next pull)`,
@@ -960,16 +978,28 @@ async function resolveBothDivergedResources(options: {
960978

961979
const withCredNames = canonicalizeForHash(entry.resource, state, credReverse);
962980

981+
// Guided variable restoration — preserve the author's `{{name}}` templates
982+
// for unchanged fields (mirrors the normal pull-write path).
983+
const toWrite = restoreVariablePlaceholders(
984+
withCredNames,
985+
readLocalResourceData(entry.resourceType, entry.resourceId),
986+
state.variables,
987+
) as Record<string, unknown>;
988+
963989
await writeResourceFile(
964990
entry.resourceType,
965991
entry.resourceId,
966-
withCredNames,
992+
toWrite,
967993
);
968994
console.log(
969995
` ⬇️ ${entry.resourceId} (both diverged — resolving with --resolve=theirs, overwriting local with platform) ${formatDriftLabel("both-diverged")}`,
970996
);
971997
// Hash the post-write disk form (same invariant as the normal pull-write path).
972-
const diskHash = hashLocalResource(entry.resourceType, entry.resourceId);
998+
const diskHash = hashLocalResource(
999+
entry.resourceType,
1000+
entry.resourceId,
1001+
state.variables,
1002+
);
9731003
if (diskHash === null) {
9741004
console.warn(
9751005
` ⚠️ ${entry.resourceType}/${entry.resourceId}: failed to hash post-write disk form; falling back to in-memory hash (may produce phantom drift on next pull)`,

0 commit comments

Comments
 (0)