Skip to content

fix: enforce CLI readiness deadlines - #265

Open
StevenWang-CY wants to merge 1 commit into
vllm-project:mainfrom
StevenWang-CY:fix-cli-readiness-deadline
Open

fix: enforce CLI readiness deadlines#265
StevenWang-CY wants to merge 1 commit into
vllm-project:mainfrom
StevenWang-CY:fix-cli-readiness-deadline

Conversation

@StevenWang-CY

Copy link
Copy Markdown
Contributor

Summary

wait_for_gateway checked its deadline only between polling rounds. A stalled /health or /ready request, or a retry sleep longer than the remaining budget, could overrun the readiness timeout. Successful probes observed after the deadline could also allow the harness to launch.

Apply one deadline to the probes and retry sleeps, and reject late success. This covers the configurable agentic run budget and the fixed 30-second agentic harness budget. Existing skip-probe behavior, redacted errors, and caller cleanup are preserved.

Test Plan

  • On current main (474d154), the five regression cases fail for retry-sleep overrun, stalled health/readiness probes, a stalled health probe with upstream checking skipped, and late success. Three healthy/skip, redirect, and zero-timeout/redaction controls pass. All eight cases pass with this patch, including 50 fresh-process repetitions split between serial and concurrent execution.
  • Actual CLI reproduction with --llm-ready-timeout-s 0.25 --llm-ready-interval-s 60: the baseline exceeded a three-second watchdog; the patched command returned the readiness error in about 0.6 seconds, with no gateway child or temporary session remaining and no harness launched.
  • Clean committed checkout: workspace/all-target/all-feature cargo check and Clippy with warnings denied; full all-feature Rust workspace suite (907 passed, 9 ignored); rustfmt and all applicable pre-commit hooks. The ignored tests require PostgreSQL or are an existing doctest.
  • Python suite: 111 passed, 3 installation-only skips. Source-install CLI end-to-end checks pass. Recorded harness smoke tests pass with Codex 0.149.1 and Claude Code 2.1.245; cassette validation passes for all 108 recordings and 238 turns.

No new inference cassette is needed for the readiness failure: it occurs before model requests. The regression tests exercise real local health endpoints, and existing recorded harness replays cover successful startup. No live model or GPU is required.

Signed-off-by: Chuyue Wang <stevenwang0805@outlook.com>
@StevenWang-CY

Copy link
Copy Markdown
Contributor Author

@franciscojavierarceo, could you review this when you have a chance? It fixes a reproduced CLI readiness deadline bug. The full local suite and pinned harness replays pass; hosted CI is still queued.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants