Ask the process, not the transcript, whether an agent is still running - #5564
Ask the process, not the transcript, whether an agent is still running#5564vivekchand wants to merge 11 commits into
Conversation
✅ Drift Bot (ClawMetry): no drift detectedDrift Bot analyzed the changed files against this project's blueprints and requirements and found no drift. |
Visual diffComparing 39 of 70 comparison(s) flagged (>1% pixel diff).
Folder: 3179c6020954. Full PNGs also attached as a workflow artefact. Generated by visual-diff bot. Pixel diffs >1% flagged; eyeball the table before merging. This check is non-blocking — fail = bot bug, not a code problem. |
|
🔧 auto-fixed: two MOAT Verifier failures
Generated by Claude Code |
|
![8090-software-factory[bot]](https://avatars.githubusercontent.com/in/1847834?s=60&v=4){kind=small}
| OUTCOME_ESCALATED = "escalated" | ||
| OUTCOME_COGNITIVE_LOOP = "cognitive_loop" | ||
| OUTCOME_ONGOING = "ongoing" | ||
| OUTCOME_WAITING = "waiting" |
There was a problem hiding this comment.
The code implements a new "waiting" outcome label (line 66) that represents sessions idle at prompt, but this outcome category and its semantics are not documented in the requirements. The requirement describes "ongoing" sessions but does not define "waiting" as a distinct outcome or explain when a process is considered "waiting" vs "busy".
| return out | ||
|
|
||
|
|
||
| # What a live process is actually doing, as the runtime itself reports it. |
There was a problem hiding this comment.
The code implements a new session_live_state() function (line 1800+) that probes actual process state (busy/idle/dead) instead of inferring from transcript recency. This is a new mechanism for determining session liveness not described in the blueprint, which focuses on session timelines and trace context but does not specify how to determine whether a session is actively running.
|
Blocked on required review — no code changes needed. All CI checks are green (MOAT Verifier 72 tests passing). Generated by Claude Code |
|
| OUTCOME_ESCALATED = "escalated" | ||
| OUTCOME_COGNITIVE_LOOP = "cognitive_loop" | ||
| OUTCOME_ONGOING = "ongoing" | ||
| OUTCOME_WAITING = "waiting" |
There was a problem hiding this comment.
The code implements a new "waiting" outcome label (line 66) representing sessions idle at their prompt, but this outcome category and its semantics are not documented in the requirements. The requirement describes "ongoing" sessions but does not define "waiting" as a distinct outcome or explain when a process is considered "waiting" vs "busy".
| return out | ||
|
|
||
|
|
||
| # What a live process is actually doing, as the runtime itself reports it. |
There was a problem hiding this comment.
The code implements a new session_live_state() function that probes actual process state (busy/idle/dead) instead of inferring from transcript recency, but this mechanism for determining session liveness is not described in the blueprint. The blueprint does not specify how to determine whether a session is actively running or waiting.
| ``escalated``. | ||
| now: clock override for tests. Defaults to ``time.time()``. | ||
| live: what the caller's process probe says about this session right | ||
| now — ``"busy"``, ``"idle"``, ``"dead"``, or ``None`` for "this node |
There was a problem hiding this comment.
The code introduces time-dependent outcome re-classification with a 10-second TTL for "ongoing" and "waiting" labels (implemented via _LIVE_LABEL_TTL_MS and _is_stale_classification), but this TTL-based staleness mechanism and the concept of outcomes that decay with clock time are not documented in the blueprint.
|
blocked on author decision — skipping (auto-mergeability sweep): Drift Bot found 3 blueprint/requirement gaps (new "waiting" outcome, Generated by Claude Code |
|
Automated PR sweep — 2026-09-05 Investigated this PR as part of a scheduled mergeability sweep. All CI checks are green. The sole blocker is:
Cannot be auto-fixed by code changes. Resolving requires either updating the product blueprints at factory.8090.ai to document these behaviors, or removing the undocumented behavior from the implementation. Widening the PR is out of scope for the automated sweep. Status: Generated by Claude Code |
|
Automated sweep — standing-down note This PR is blocked solely by Drift Bot ( Drift Bot found 3 blueprint-documentation findings on the current head commit — meaning this PR's diff introduces or modifies behaviour that isn't yet documented in the corresponding 8090 Software Factory blueprints. This is a documentation gap, not a code defect; no change to the PR's source files can clear it. What's needed to unblock:
No further automated action can be taken from this sweep — Factory access is required. Generated by Claude Code |
|
Automated maintenance sweep — human action required This PR is blocked by the To unblock:
Generated by Claude Code |
|
blocked on author decision — skipping (auto-mergeability sweep) All 11 non-Drift-Bot legs of the E2E Gate pass. The sole blocker is Generated by Claude Code |
|
blocked on author decision — skipping (auto-mergeability sweep) The Generated by Claude Code |
|
| OUTCOME_ESCALATED = "escalated" | ||
| OUTCOME_COGNITIVE_LOOP = "cognitive_loop" | ||
| OUTCOME_ONGOING = "ongoing" | ||
| OUTCOME_WAITING = "waiting" |
There was a problem hiding this comment.
The code implements a new "waiting" outcome label for sessions idle at their prompt, but the requirement does not document this outcome category, its semantics, or when a session should be classified as "waiting" versus "busy".
| return out | ||
|
|
||
|
|
||
| # What a live process is actually doing, as the runtime itself reports it. |
There was a problem hiding this comment.
The code implements a new session_live_state() function that probes actual process state (busy/idle/dead/unknown) to determine session liveness, but the blueprint does not describe this probe-based mechanism or how it determines whether a session is actively running versus waiting or dead.
| ``escalated``. | ||
| now: clock override for tests. Defaults to ``time.time()``. | ||
| live: what the caller's process probe says about this session right | ||
| now — ``"busy"``, ``"idle"``, ``"dead"``, or ``None`` for "this node |
There was a problem hiding this comment.
The code introduces TTL-based staleness checking for "ongoing" and "waiting" outcomes (10-second TTL via _LIVE_LABEL_TTL_MS and _is_stale_classification), but the blueprint does not document this mechanism for re-resolving time-dependent outcomes or explain that these labels decay with the clock.
|
E2E Gate rerun triggered (timing failure — gate checked required checks before individual jobs finished; all individual checks passed). CI now running. If still not green in 10 min, may need manual attention. Generated by Claude Code |
|
✨ auto-fixed: merged current main into the branch (was behind; fresh CI triggered) Generated by Claude Code |
|
| OUTCOME_ESCALATED = "escalated" | ||
| OUTCOME_COGNITIVE_LOOP = "cognitive_loop" | ||
| OUTCOME_ONGOING = "ongoing" | ||
| OUTCOME_WAITING = "waiting" |
There was a problem hiding this comment.
The code implements a new "waiting" outcome label for sessions idle at their prompt, but this outcome category, its semantics, and when a session should be classified as "waiting" versus "busy" are not documented in the requirements.
|
| return out | ||
|
|
||
|
|
||
| # What a live process is actually doing, as the runtime itself reports it. |
There was a problem hiding this comment.
The code implements a new session_live_state() function that probes Claude Code process state (busy/idle/dead/unknown) to determine session liveness, but the blueprint does not document this probe-based mechanism or explain how it determines whether a session is actively running versus waiting or dead.
| OUTCOME_ESCALATED = "escalated" | ||
| OUTCOME_COGNITIVE_LOOP = "cognitive_loop" | ||
| OUTCOME_ONGOING = "ongoing" | ||
| OUTCOME_WAITING = "waiting" |
There was a problem hiding this comment.
The code implements a new OUTCOME_WAITING label for sessions idle at their prompt, but the blueprint does not document this outcome category, its semantics, or the conditions for classifying a session as "waiting" versus "busy" or "ongoing".
| *, | ||
| approvals: list[dict[str, Any]] | None = None, | ||
| now: float | None = None, | ||
| live: str | None = None, |
There was a problem hiding this comment.
The classify_session() function now accepts a new "live" parameter that supplies the caller's process probe result (busy/idle/dead/None), fundamentally changing how "ongoing" and "waiting" outcomes are determined, but this new parameter and its role in decision-making are not documented in the blueprint.
|
| OUTCOME_ESCALATED = "escalated" | ||
| OUTCOME_COGNITIVE_LOOP = "cognitive_loop" | ||
| OUTCOME_ONGOING = "ongoing" | ||
| OUTCOME_WAITING = "waiting" |
There was a problem hiding this comment.
The code implements a new OUTCOME_WAITING label for sessions idle at their prompt (waiting on user input), but the blueprint does not document this outcome category, its semantics, or the conditions for classifying a session as waiting versus ongoing.
| return out | ||
|
|
||
|
|
||
| # What a live process is actually doing, as the runtime itself reports it. |
There was a problem hiding this comment.
The code implements a new session_live_state() function that probes Claude Code process state (busy/idle/dead/unknown) to determine session liveness in real-time, but the blueprint does not document this probe-based mechanism or explain how session status is determined.
| *, | ||
| approvals: list[dict[str, Any]] | None = None, | ||
| now: float | None = None, | ||
| live: str | None = None, |
There was a problem hiding this comment.
The classify_session() function now accepts a new live parameter that supplies the caller's process probe result (busy/idle/dead/None), fundamentally changing how ongoing and waiting outcomes are determined, but this parameter and its role in decision-making are not documented in the blueprint.
|
✨ auto-fixed: merged latest main into branch (was BEHIND; no conflicts) Generated by Claude Code |
|
| return out | ||
|
|
||
|
|
||
| # What a live process is actually doing, as the runtime itself reports it. |
There was a problem hiding this comment.
The code implements a new session_live_state() function that directly probes Claude Code process state (busy/idle/dead/unknown) to determine session liveness in real-time, but the blueprint does not document this probe-based mechanism or explain how session status is determined from process state.
| OUTCOME_ESCALATED = "escalated" | ||
| OUTCOME_COGNITIVE_LOOP = "cognitive_loop" | ||
| OUTCOME_ONGOING = "ongoing" | ||
| OUTCOME_WAITING = "waiting" |
There was a problem hiding this comment.
The code implements a new OUTCOME_WAITING label for sessions idle at their prompt (waiting on user input), but the blueprint does not document this outcome category, its semantics, or the conditions for classifying a session as waiting versus busy or ongoing.
| *, | ||
| approvals: list[dict[str, Any]] | None = None, | ||
| now: float | None = None, | ||
| live: str | None = None, |
There was a problem hiding this comment.
The classify_session() function now accepts a new live parameter that supplies the caller's process probe result (busy/idle/dead/None), fundamentally changing how ongoing and waiting outcomes are determined, but this parameter and its role in decision-making are not documented in the blueprint.
vivekchand
left a comment
vivekchand
left a comment
There was a problem hiding this comment.
Test plan & review notes
Repo: vivekchand/clawmetry
What changed
process_control.session_live_state()probes the per-pid record Claude Code publishes (~/.claude/sessions/<pid>.json) and returnsbusy / idle / dead / unknownin ~4.5µs (memoised 2s).- A new
waitingoutcome (idle process, parked at prompt) is added toOUTCOME_KEYSso "Waiting on you" shows instead of "Still running" for parked sessions. _is_stale_classificationre-resolves the two live-process labels on a 10s TTL under the 20s read cache.- Four test files (
test_outcome_classifier.py,test_session_live_state.py,test_outcome_trend.py,test_otel_export_sessions_shape.py) are added to the CI file list inci.yml— they were previously not wired to any job.
Current status
mergeable_state: blocked — needs approval. 65 comments; from prior bot sweeps this appears to be a Drift Bot / E2E Gate issue, not a code failure. The PR body states 23 new tests, all red-before-green, and no new lint findings.
Smoke commands
# Fast: just the new test files
python3 -m pytest tests/test_outcome_classifier.py \
tests/test_session_live_state.py \
tests/test_outcome_trend.py \
tests/test_otel_export_sessions_shape.py -v
# Check the liveness probe against live processes
python3 - <<'EOF'
from clawmetry.process_control import session_live_state
import os, glob
# Point at real sessions if available
for f in glob.glob(os.path.expanduser("~/.claude/sessions/*.json"))[:3]:
sid = "claude_code:" + os.path.basename(f).replace(".json","")
print(sid, "->", session_live_state(sid))
EOF
# API spot-check: sessions list should show "Waiting on you" label for idle ones
curl -sS http://localhost:8900/api/sessions | python3 -m json.tool | grep -A2 '"outcome"'What to look at visually
- Sessions list (
http://localhost:8900/api/sessions) — idle Claude Code processes should now showwaitingoutcome, notongoing. - Trail page for a recently-finished session — should show
Finished, notStill running. - Guard tab — session ranking by
spend_at_riskshould still sort correctly withwaitingadded to non-finished set.
Likely failure modes from the diff
- "Cannot see" vs "nothing running": the PR body flags that an absent
~/.claude/sessionsdir returns{}for both mounted and unmounted cases — in a hosted/containerised environment every Claude Code session could wrongly flip to dead. Check the guard inclaude_code_session_maphandles an unmounted home gracefully. agent_typevs id-prefix: the id prefix (claude_code:<uuid>) must win over theagent_typecolumn when dispatching the probe. Worth a grep:grep -n "agent_type" clawmetry/process_control.py.waitingexcluded fromfinished:cohort_compare.py,harness_bench.py,insights.pyall updated — check that the cost-outcomes endpoint (/api/usage/outcomes) still reports correctly when some sessions arewaiting.- Pre-existing 31 failures on main: the PR body notes they exist on
origin/maintoo, so a clean run against main is the baseline, not zero.
Issue link
- No issue number in the PR body (
No-PRD: field-reported UI accuracy bug). If there's an issue for the "43 sessions labelled ongoing, 3 actually were" report, linking it (Closes #N) would auto-close on merge.
Generated by Claude Code
|
| return out | ||
|
|
||
|
|
||
| # What a live process is actually doing, as the runtime itself reports it. |
There was a problem hiding this comment.
The code implements a new session_live_state() function that directly probes Claude Code process state (busy/idle/dead/unknown) to determine session liveness in real-time, but the blueprint does not document this probe-based mechanism, the three process states it returns, or how session status is determined from live process state rather than transcript activity.
| OUTCOME_ESCALATED = "escalated" | ||
| OUTCOME_COGNITIVE_LOOP = "cognitive_loop" | ||
| OUTCOME_ONGOING = "ongoing" | ||
| OUTCOME_WAITING = "waiting" |
There was a problem hiding this comment.
The code implements a new OUTCOME_WAITING label for sessions idle at their prompt (waiting on user input), but the blueprint does not document this outcome category, its semantics, the conditions for classifying a session as waiting versus busy, or that waiting is displayed as "Waiting on you" in the UI.
|
| return out | ||
|
|
||
|
|
||
| # What a live process is actually doing, as the runtime itself reports it. |
There was a problem hiding this comment.
The code implements a new session_live_state() function that directly probes Claude Code process state (busy/idle/dead/unknown) to determine session liveness in real-time, but the blueprint does not document this probe-based mechanism, the three process states it returns, or how session status is determined from live process state rather than transcript activity.
| OUTCOME_ESCALATED = "escalated" | ||
| OUTCOME_COGNITIVE_LOOP = "cognitive_loop" | ||
| OUTCOME_ONGOING = "ongoing" | ||
| OUTCOME_WAITING = "waiting" |
There was a problem hiding this comment.
The code implements a new OUTCOME_WAITING label for sessions idle at their prompt (waiting on user input), but the blueprint does not document this outcome category, its semantics, the conditions for classifying a session as waiting versus busy, or that waiting is displayed as "Waiting on you" in the UI.
| *, | ||
| approvals: list[dict[str, Any]] | None = None, | ||
| now: float | None = None, | ||
| live: str | None = None, |
There was a problem hiding this comment.
The classify_session() function now accepts a new live parameter that supplies the caller's process probe result (busy/idle/dead/None), fundamentally changing how ongoing and waiting outcomes are determined and their confidence scores, but this parameter, its role in decision-making, and the confidence thresholds (0.95 for probed states vs 0.6 for time-based) are not documented in the blueprint.
2 participants
No-PRD: field-reported UI accuracy bug, found and reproduced on a live node.
The report
A user opened the Trail page for
claude_code:9a3e3302-6600-4245-9334-84909cd9c3d9and asked: are we really accurate in saying still running?No.
~/.claude/sessions/<pid>.jsonMeasured across that node's store at the time: 43 sessions were labelled
ongoingand 3 of them were. 21 were finished sub-agents, 6 were dead terminals, 13 were parked at a prompt. It failed in the other direction too — 12 liveclaudeprocesses carried a "Finished" badge, 10 with a non-nullended_at.Why
The label was decided by recency.
classify_sessionreturnedongoingwhen no terminal marker existed and an event had landed within 5 minutes. Recency is not liveness: it cannot separate an agent mid-turn from a terminal closed a second after the last reply, and it reads a long tool call as an ending.The label was stamped once and never revisited.
ongoingis the only label whose truth decays, and it was the one label nothing re-ran —reclassify_session_outcomefires on asession.endedevent, which the family runtimes never emit. The first label a Claude Code session received was also its last.The fix
process_control.session_live_state()reads the per-pid record Claude Code already publishes and returns busy / idle / dead / unknown in 4.5µs, off the existing 2s memo.unknownstays distinct fromdead— a runtime we cannot probe keeps the old heuristic rather than being retired.idlegets its own verdict instead of being folded into "Still running". A process parked at its prompt is waiting for a person; saying "running" sends someone to check on an agent that is waiting on them._is_stale_classificationnow re-resolves the two process-shaped labels on a 10s TTL, under the 20s read cache that bounds the cost. Settled labels still never re-run — only live sessions keep re-resolving, and they are few by definition.Two traps found while verifying
~/.claude/sessionsis "cannot see", not "nothing is running".claude_code_session_mapreturns{}for both. Caught by pointing the probe at an empty HOME and watching five busy sessions flip to Finished — which is what a container with no mount, or the hosted dashboard, would have done to every session.agent_type. The sessions table stampsagent_type"openclaw" on rows whose id isclaude_code:<uuid>— 1849 of them on this node. Trusting the argument would have left every family row unprobeable and fixed nothing.Verified
Booted from this branch against the real machine's live processes:
9a3e3302(dead, the reported session) → FinishedTests
tests/test_outcome_classifier.pyran in no CI job — which is how its seeds came to be hard-coded2026-05-17, inside the endpoint's?window=30dwhen written and outside it three months later, asserting0 == 3for reasons unrelated to outcomes. Seeds are anchored to now; that file plustest_outcome_trend,test_otel_export_sessions_shapeand the newtest_session_live_stateare added to the CI file list.23 new tests. Every one was checked red against the unfixed code and green after. Broad suite matches baseline exactly (31 pre-existing failures on
origin/main, 31 here). Zero new lint findings.Scope
Verdict accuracy only. No new way to affect an agent — the CLAUDE.md intervention list is unchanged.
🤖 Generated with Claude Code
https://claude.ai/code/session_0154kqdA7JGLTbfsFSM3u4fB