Skip to content

feat(interface): measure complete profile changes - #97

Merged
drewstone merged 1 commit into
mainfrom
feat/profile-complete-improvement-v2
Jul 29, 2026
Merged

feat(interface): measure complete profile changes#97
drewstone merged 1 commit into
mainfrom
feat/profile-complete-improvement-v2

Conversation

@drewstone

Copy link
Copy Markdown
Contributor

What changed

  • let the existing AgentProfileDiff improvement contract accept every current AgentProfile axis
  • map prompt, skills, tools, MCP, hooks, and subagents to their existing granular surface labels, with agent-profile covering residual axes
  • require newly introduced measured resources to be inline so experiment and activation bytes stay exact
  • reject structurally empty removal steps that cannot change a profile
  • fix changed-axis detection for explicit empty identity values

No candidate type or parallel improvement record is added.

Proof

  • pnpm test — all workspace tests passed; Agent Interface 228/228
  • pnpm --filter @tangle-network/agent-interface check-types
  • pnpm build
  • git diff --check
  • git merge-tree --write-tree origin/main HEAD

@tangletools tangletools left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Auto-approved drewstone PR — 02f7ee07

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

tangletools · auto-approval · reason: drewstone_author · 2026-07-29T21:08:00Z

@drewstone
drewstone merged commit 7c68070 into main Jul 29, 2026
1 check passed

@tangletools tangletools left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Value Audit — sound

Verdict sound
Concerns 0 (none)
Heuristic 0.0s
Duplication 0.0s
Interrogation 185.8s (2 bridge agents)
Total 185.8s

💰 Value — sound

Broadens the measured profile-improvement contract from prompt+skills-only to every AgentProfile axis, reusing the existing surface enum and axis-diff helper; coherent and in-grain, with two correct bug fixes bundled in.

  • What it does: The agentProfileImprovementChangeStepSchema previously rejected any profile diff touching axes other than prompt and inline resources.skills. This change (1) removes that allow-list, so a measured/activated improvement may now set or remove model, harness, permissions, tools, mcp, connections, subagents, hooks, modes, confidential, metadata, extensions, and all resource kinds (files/tools/agen
  • Goals it achieves: (1) Let the optimizer measure and ship improvements to ANY profile axis, not just prompt/skills — the prior restriction was self-described in code as 'the first product path', so this is the anticipated evolution. (2) Preserve byte-exact experiment/activation guarantees for ALL measured resources, closing a hole where a non-skill resource (e.g. a github-ref'd tool/file) could drift between signed
  • Assessment: Sound and well-scoped. It extends existing primitives rather than forking them: the AgentImprovementSurface enum (agent-candidate.ts:818) already defined prompt/skills/tools/mcp/hooks/subagents/agent-profile, so the mapper consumes labels that already existed for other improvement types; changedAgentProfileAxes (profile-diff.ts:288) already enumerated every property axis exhaustively via agentProf
  • Better / existing approach: none — this is the right approach. Searched for an existing axis→surface mapper (grep AgentImprovementSurface across packages/agent-interface/src: only agent-candidate.ts defines it and only this schema consumes it for profile improvements), an existing 'is this removal a no-op' predicate (grep hasRemovalOperation/isEmptyRemoval: none existed; this is the first), and any parallel profile-improveme
  • Model: opencode/zai-coding-plan/glm-5.2
  • Bridge attempts: 2
  • Bridge warning: opencode/kimi-for-coding/k2p7: bridge stream ended without value-audit content

🎯 Usefulness — sound

Widens an existing, fully-wired profile-improvement contract to accept every AgentProfile axis (previously prompt+skills only), reusing the established surface vocabulary and inline-bytes rule with no new types or competing patterns.

  • Integration: Live and reachable. agentProfileImprovementChangeSchema (agent-profile-improvement-schema.ts:200) feeds agentProfileImprovementExperimentSchema (line 213) → agentProfileImprovementMeasuredComparisonSchema (line 462) → the evaluation discriminant of agentImprovementProposalSchema (agent-candidate-promotion-schema.ts:449). The rewritten changedProfileImprovementSurfaces is called at agen
  • Fit with existing patterns: Follows the codebase grain precisely. Reuses the existing AgentProfileDiff/agentProfileDiffSchema as the patch base rather than inventing a new format; reuses the existing AgentImprovementSurface enum from agent-candidate.ts:818 (no new labels — prompt/skills/tools/mcp/hooks/subagents already existed, agent-profile was already the catchall); reuses changedAgentProfileAxes from profile-di
  • Real-world viability: Edge-case handling is careful and tested. Set-side resources map granular keys (skills/tools/agents→subagents) vs. residual→agent-profile (lines 518-525); remove-side distinguishes resources: true (full removal → all granular + catchall, lines 529-533) from partial removals, filtering empty arrays so structurally empty removals don't emit false surfaces (lines 534-543). The empty-{}-resources
  • Model: opencode/zai-coding-plan/glm-5.2
  • Bridge attempts: 1

No concerns — sound change, no better or existing approach found. ✅


What this audit checks

It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.

Pass What it asks
Heuristic Vague title? Whitespace-only or cruft-bearing diff? (content signals only)
Duplication Do added function/class names already exist elsewhere in the repo?
Value Audit What does it do? What goal does it achieve? Is it good? Better architecture or already-exists?
Usefulness Audit Does it integrate and fit? Will it hold up in real use and actually get used?

Findings are concerns, not blocks — the human reviewer decides what to do with them.

value-audit · 20260729T211127Z

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants