feat(interface): measure complete profile changes - #97
Conversation
tangletools
left a comment
There was a problem hiding this comment.
✅ Auto-approved drewstone PR — 02f7ee07
This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.
tangletools · auto-approval · reason: drewstone_author · 2026-07-29T21:08:00Z
tangletools
left a comment
There was a problem hiding this comment.
🟢 Value Audit — sound
| Verdict | sound |
| Concerns | 0 (none) |
| Heuristic | 0.0s |
| Duplication | 0.0s |
| Interrogation | 185.8s (2 bridge agents) |
| Total | 185.8s |
💰 Value — sound
Broadens the measured profile-improvement contract from prompt+skills-only to every AgentProfile axis, reusing the existing surface enum and axis-diff helper; coherent and in-grain, with two correct bug fixes bundled in.
- What it does: The agentProfileImprovementChangeStepSchema previously rejected any profile diff touching axes other than
promptand inlineresources.skills. This change (1) removes that allow-list, so a measured/activated improvement may now set or remove model, harness, permissions, tools, mcp, connections, subagents, hooks, modes, confidential, metadata, extensions, and all resource kinds (files/tools/agen - Goals it achieves: (1) Let the optimizer measure and ship improvements to ANY profile axis, not just prompt/skills — the prior restriction was self-described in code as 'the first product path', so this is the anticipated evolution. (2) Preserve byte-exact experiment/activation guarantees for ALL measured resources, closing a hole where a non-skill resource (e.g. a github-ref'd tool/file) could drift between signed
- Assessment: Sound and well-scoped. It extends existing primitives rather than forking them: the AgentImprovementSurface enum (agent-candidate.ts:818) already defined prompt/skills/tools/mcp/hooks/subagents/agent-profile, so the mapper consumes labels that already existed for other improvement types; changedAgentProfileAxes (profile-diff.ts:288) already enumerated every property axis exhaustively via agentProf
- Better / existing approach: none — this is the right approach. Searched for an existing axis→surface mapper (grep AgentImprovementSurface across packages/agent-interface/src: only agent-candidate.ts defines it and only this schema consumes it for profile improvements), an existing 'is this removal a no-op' predicate (grep hasRemovalOperation/isEmptyRemoval: none existed; this is the first), and any parallel profile-improveme
- Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 2
- Bridge warning: opencode/kimi-for-coding/k2p7: bridge stream ended without value-audit content
🎯 Usefulness — sound
Widens an existing, fully-wired profile-improvement contract to accept every AgentProfile axis (previously prompt+skills only), reusing the established surface vocabulary and inline-bytes rule with no new types or competing patterns.
- Integration: Live and reachable.
agentProfileImprovementChangeSchema(agent-profile-improvement-schema.ts:200) feedsagentProfileImprovementExperimentSchema(line 213) →agentProfileImprovementMeasuredComparisonSchema(line 462) → theevaluationdiscriminant ofagentImprovementProposalSchema(agent-candidate-promotion-schema.ts:449). The rewrittenchangedProfileImprovementSurfacesis called at agen - Fit with existing patterns: Follows the codebase grain precisely. Reuses the existing
AgentProfileDiff/agentProfileDiffSchemaas the patch base rather than inventing a new format; reuses the existingAgentImprovementSurfaceenum from agent-candidate.ts:818 (no new labels — prompt/skills/tools/mcp/hooks/subagents already existed,agent-profilewas already the catchall); reuseschangedAgentProfileAxesfrom profile-di - Real-world viability: Edge-case handling is careful and tested. Set-side resources map granular keys (skills/tools/agents→subagents) vs. residual→agent-profile (lines 518-525); remove-side distinguishes
resources: true(full removal → all granular + catchall, lines 529-533) from partial removals, filtering empty arrays so structurally empty removals don't emit false surfaces (lines 534-543). The empty-{}-resources - Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 1
No concerns — sound change, no better or existing approach found. ✅
What this audit checks
It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.
| Pass | What it asks |
|---|---|
| Heuristic | Vague title? Whitespace-only or cruft-bearing diff? (content signals only) |
| Duplication | Do added function/class names already exist elsewhere in the repo? |
| Value Audit | What does it do? What goal does it achieve? Is it good? Better architecture or already-exists? |
| Usefulness Audit | Does it integrate and fit? Will it hold up in real use and actually get used? |
Findings are concerns, not blocks — the human reviewer decides what to do with them.
What changed
AgentProfileDiffimprovement contract accept every currentAgentProfileaxisagent-profilecovering residual axesNo candidate type or parallel improvement record is added.
Proof
pnpm test— all workspace tests passed; Agent Interface 228/228pnpm --filter @tangle-network/agent-interface check-typespnpm buildgit diff --checkgit merge-tree --write-tree origin/main HEAD