Outcome and baseline
Provide xpool top with live KV Cache occupancy by GPU and model, showing pool
totals and each model's share. The KV Control Channel and daemon already track
capacity policy and physical completion, but those observations do not provide
a complete read-only live occupancy contract.
This issue covers source auditing, a minimal terminal prototype, and the
snapshot target design. Production telemetry and CLI implementation follow an
accepted plan.
Suggested implementation route
- Audit authoritative sources for pool budgets, authorized targets, active
capacity, physical backing, and logical used/free/cached/reclaimable slots.
Distinguish post-capture device-memory observations, conservative in-flight
charging, and terminal backing completion from live measurements. Record
units, geometry, timestamp/freshness, and the owner of every proposed field.
- Prototype a read-only terminal view over available data and the smallest
experimental instrumentation needed for missing counters. Show transient
capacity changes and mark unavailable or stale observations explicitly.
Use per-partition geometry for physical bytes; avoid double-counting shared
group logical capacity across TP ranks.
- Propose the live snapshot contract, generation and Instance identity,
collection consistency, update cadence, and lifetime. Measure collection
overhead and serving/capacity-coordination behavior, then write the scoped
target plan.
- After accepting the plan, add the required owner-side observations and
xpool top command using that read-only contract.
- Qualify pool and model accounting, transient resize states, stale or failed
participants, and collection during concurrent serving and reclamation.
Discovery completion evidence
- A source-and-units matrix identifying authoritative values and gaps.
- A minimal view and raw observations showing physical/logical distinctions,
TP accounting, and capacity transitions.
- A reviewed snapshot proposal and evidence about collection overhead and
interference, or an evidence-backed no-go.
Closing this issue does not establish a production telemetry API. Operational
logs are diagnostics and are not the live-view data source. Unified Timeline
Observability would improve analysis but does not block this workstream.
References
Outcome and baseline
Provide
xpool topwith live KV Cache occupancy by GPU and model, showing pooltotals and each model's share. The KV Control Channel and daemon already track
capacity policy and physical completion, but those observations do not provide
a complete read-only live occupancy contract.
This issue covers source auditing, a minimal terminal prototype, and the
snapshot target design. Production telemetry and CLI implementation follow an
accepted plan.
Suggested implementation route
capacity, physical backing, and logical used/free/cached/reclaimable slots.
Distinguish post-capture device-memory observations, conservative in-flight
charging, and terminal backing completion from live measurements. Record
units, geometry, timestamp/freshness, and the owner of every proposed field.
experimental instrumentation needed for missing counters. Show transient
capacity changes and mark unavailable or stale observations explicitly.
Use per-partition geometry for physical bytes; avoid double-counting shared
group logical capacity across TP ranks.
collection consistency, update cadence, and lifetime. Measure collection
overhead and serving/capacity-coordination behavior, then write the scoped
target plan.
xpool topcommand using that read-only contract.participants, and collection during concurrent serving and reclamation.
Discovery completion evidence
TP accounting, and capacity transitions.
interference, or an evidence-backed no-go.
Closing this issue does not establish a production telemetry API. Operational
logs are diagnostics and are not the live-view data source. Unified Timeline
Observability would improve analysis but does not block this workstream.
References