Skip to content

test(coding-agent): cover provider-error goal recovery - #727

Open
realsigridjin wants to merge 1 commit into
code-yeongyu:mainfrom
realsigridjin:test/provider-error-goal-recovery
Open

test(coding-agent): cover provider-error goal recovery#727
realsigridjin wants to merge 1 commit into
code-yeongyu:mainfrom
realsigridjin:test/provider-error-goal-recovery

Conversation

@realsigridjin

@realsigridjin realsigridjin commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add a real AgentSession regression that joins terminal provider failure, persisted blocking, next-input reactivation before agent_start, and completion.
  • Prove deliberate model/user blocks remain blocked, including an unchanged numeric blockedAt.
  • Add a hermetic source-CLI RPC scenario using the existing localhost fake provider, auth guard, and sandbox helpers.

Context

The stale-goal behavior was reproduced on published Senpi 2026.8.3-3. Production code is already fixed by #688 and included in newer releases; this PR closes the remaining end-to-end coverage gap rather than duplicating that fix. Existing coverage manually invokes goal handlers or tests direct input separately, so it would not catch a break in the full provider-error -> input disposition -> agent_start ordering.

RED -> GREEN

  • Pre-fix classifier mutation (remove the provider-error reason): the next real agent_start observed [null, "blocked"] instead of [null, "active"].
  • Deliberate-block mutation (classify Waiting on a user decision as mechanical): the real direct prompt observed active instead of blocked.
  • Restored current main: goal-provider-error-recovery.test.ts passes 3/3.

Manual QA

node .agents/skills/senpi-qa/scripts/goal-provider-error-recovery.mjs \
  --self-test \
  --evidence goal-provider-error-recovery

Result: 14/14 checks passed. The real source CLI made exactly seven localhost provider requests, reactivated the mechanical block on the next direct prompt, preserved the intentional block on a later prompt, exited RPC with code 0, left real auth unchanged, stopped the fake server, and removed its sandbox.

Validation

  • npm --prefix packages/coding-agent test -- test/suite/goal-provider-error-recovery.test.ts test/suite/goal-abort-extension.test.ts test/suite/goal-extension.test.ts test/suite/goal-turn-usage.test.ts — 53/53 passed
  • npm run check — passed
  • node --check .agents/skills/senpi-qa/scripts/goal-provider-error-recovery.mjs — passed

Summary by cubic

Added an end-to-end test and a hermetic RPC QA script to cover goal recovery after a terminal provider error. Confirms the next direct prompt reactivates the goal before agent_start, while intentional user blocks remain with the same blockedAt; runs fully against a fake localhost provider with no real auth or traffic.

Written for commit 30fce90. Summary will update on new commits.

Review in cubic

Exercise the real AgentSession path from terminal provider failure through direct-input reactivation and completion while preserving deliberate blocks.

Add an isolated RPC QA scenario that proves the same lifecycle through the source CLI without real credentials or provider traffic.
@realsigridjin
realsigridjin marked this pull request as ready for review August 5, 2026 10:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant