Skip to content

router: grant an in-flight resume a bounded grace past the park budget when a worker is already assigned - #676

Open
yufan-su wants to merge 1 commit into
agent-substrate:mainfrom
yufan-su:resume-commit-grace
Open

router: grant an in-flight resume a bounded grace past the park budget when a worker is already assigned#676
yufan-su wants to merge 1 commit into
agent-substrate:mainfrom
yufan-su:resume-commit-grace

Conversation

@yufan-su

Copy link
Copy Markdown
Collaborator

#675

Context

In a flaky run of TestRequestParking E2E, it parks a request for actor B behind a deliberately 1-worker pool, frees the worker by suspending actor A, and expects a 200.

What we got instead was a 503 "no free workers available" at just over the 5s park budget, where Actor B was left in RESUMING with the worker already assigned. In this case, CI runs 7 e2e suites against a single 4-vCPU kind cluster; under that contention the gVisor snapshot restore routinely runs a few hundred milliseconds past the 5s budget. So the router cancelled a resume that was about to succeed.

Proposal

When the park budget elapses with a resume attempt still in flight, the router issues one GetActor:

  • Actor is RESUMING or RUNNING → a worker is assigned; the in-flight attempt gets a bounded 3s grace to land.
  • Anything else (including a failed or timed-out probe) → cancel and 503 exactly as before.

The retry loop always stops at the budget. The grace covers the single attempt already underway, never another round. The probe's 1s timeout is spent inside the grace, not on top of it, so the router's worst-case hold on a parked request is exactly budget + grace.

  • Tests pass
  • Appropriate changes to documentation are included in the PR

@yufan-su
yufan-su force-pushed the resume-commit-grace branch from 1c260d1 to d442b58 Compare July 31, 2026 22:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant