Skip to content

feat(daemon): the daemon reports the failures that stop it working - #5752

Merged
vivekchand merged 13 commits into
mainfrom
feat/daemon-field-report
Sep 10, 2026
Merged

feat(daemon): the daemon reports the failures that stop it working#5752
vivekchand merged 13 commits into
mainfrom
feat/daemon-field-report

Conversation

@vivekchand

@vivekchand vivekchand commented Sep 9, 2026

Copy link
Copy Markdown
Owner

Requirement: Daemon Field-Failure Reporting (AC-FFR-005), a live child of Field Failure Reporting and Auto-Triage.

Needs clawmetry-cloud PR (linked below) to reach the issue filer.

The gap

We already run this loop end to end, for exactly one class of failure:

desktop shell classifies a failed bootstrap
  -> POST /api/desktop/open (stage bootstrap_failed)
  -> cloud sink counts signatures at /api/desktop/_failures
  -> field-failure-issues.yml files ONE deduped issue per signature
  -> the hourly fixer routine picks it up

It is live and it works. Right now it is showing no_distribution / Windows / 3.11, a machine nobody reported.

It has one producer, and it is the installer. The daemon, which runs on every node and is the single point of failure for all of a customer's data, produced nothing when it could not run.

On 2026-09-08 a Pro node's daemon was locked out of its own pid lock and retried every 30 seconds for twelve hours. It reached nobody, by construction:

layer why it saw nothing
launchd the daemon exits 0, so KeepAlive recorded ~1,400 successful runs
DuckDB error events install_daemon_error_event_handler() is at sync.py:23972; the gate is at 23832
auto-update start_update_check_thread(role="daemon") is at sync.py:24459, 627 lines past the gate
cloud no daemon means no heartbeat, and quiet is not an event
clawmetry status read launchctl list's exit code, which is 0 for a merely registered job (fixed in #5745)

Every self-repair mechanism we ship starts after the gate that failed. The failure existed in one place on Earth: a log file on the customer's laptop. He emailed us a screenshot.

What this adds

clawmetry/field_report.py, the daemon's producer for that same pipeline. Two classes:

  • daemon_lock_refused — reported from ABOVE the gate, and only when ingest is actually stale. Refusing the lock is normally correct and frequent (a double start, a manual python -m clawmetry.sync); refusing it while nothing is moving is the field failure. Sent inline rather than on a thread, because the process exits on the next line and would take a background thread with it, which would have made the report as silent as the failure.
  • daemon_ingest_stalled — from the watchdog thread, which keeps running when the ingest loop does not. This is detectors.py's no_progress question, asked about ourselves for the first time. We ship stuck-detection for our customers' agents and had never once pointed it at our own daemon.

Privacy and egress

Same contract as the shell's, asserted key by key over the serialised body so nothing smuggles a path through a nested value: a closed enum, platform names, a version. No paths, usernames, hostnames, node id, or log text.

The gate is endpoints.egress_suppressed(), deliberately not the narrower is_custom_endpoint() I reached for first. That one misses both cases that matter: an air-gapped node sets no endpoint, and a self-hosted server IS the endpoint so it never sets one either. A report about a broken daemon is exactly the kind of well-meant call that gets shipped past an air gap. Verified live: CLAWMETRY_OFFLINE=1 in the same broken state sends nothing.

docs/EGRESS.md carries the new row.

Two things that would have made it useless

  • A restart loop is a new process every 30 seconds, so in-memory throttling throttles nothing. The stamp is on disk, one report per class per 6h (matching the filer's cron).
  • The sink is keyed UNIQUE (install_id, session_id, stage) with ON CONFLICT DO NOTHING. A constant session id would have recorded the first daemon failure an install ever had and silently discarded every one after it, including a different failure class: last_seen frozen forever. The id is derived as <class>-<UTC date>, which makes the server key "one row per class per install per day" and holds even when the local stamp is gone.

Verified end to end, not just unit-tested

A local stand-in for the cloud sink, and the customer's machine reproduced (a lock held by a live daemon, ingest stale since 19:32):

--- 0.12.748 (before): the failure that reached nobody ---
[clawmetry-sync] Another instance is already running. Exiting.
                                                     (nothing on the wire)

--- this branch ---
[clawmetry-sync] Another instance is already running. Exiting.
WARNING reporting field failure: daemon_lock_refused (aggregate only, no paths or ids)

RECEIVED /api/desktop/open
{ "arch": "arm64", "bootstrap_python": "3.14", "desktop_version": "0.12.748",
  "failure_class": "daemon_lock_refused", "install_id": "ea5d…", "os": "Darwin",
  "os_version": "25.3.0", "session_id": "daemon_lock_refused-20260909",
  "stage": "daemon_failed" }

Three further restarts into the same failure: zero additional posts. Air-gapped: zero.

tests/test_daemon_field_report.py, 24 tests, named in ci.yml. The workflow body now says which stage a signature came from, because "bootstrap" over a runtime failure sends the fixer to the wrong file.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CYQo5mXvPp3zT9tjq5mEaj


Update after merging main: a collision this PR introduced

main now carries #5790, whose reconcile step closes a field-failure issue when its signature stops being reported. Both branches touch field-failure-issues.yml. The merge was textually clean and the autoclose tests still pass — but a clean textual merge is not a working one, and checking turned up a real collision.

/api/desktop/_failures groups by stage × class × OS × Python. The issue title is the dedupe key, and it carried only class/OS/Python. So:

stage=bootstrap_failed  ->  [field-failure] store_unwritable on Darwin (py 3.12)
stage=daemon_failed     ->  [field-failure] store_unwritable on Darwin (py 3.12)

Two distinct signatures, one key. Whichever arrived second would merely refresh the first, under a lede describing the wrong failure and pointing the fixer at the wrong file — precisely the confusion this PR's per-stage lede exists to prevent.

Fixed: a stage other than bootstrap_failed prefixes the title.

bootstrap_failed  ->  [field-failure] no_distribution on Windows (py 3.11)
daemon_failed     ->  [field-failure] daemon: store_unwritable on Darwin (py 3.12)

bootstrap_failed keeps the historical shape byte for byte — that first line is #5739's actual title — because renaming it orphans every open issue, which the title contract explicitly forbids. A new stage takes a new prefix rather than reshaping the existing key.

Three tests pin it, computing the title with the workflow's own shell rather than a reimplementation (a reimplementation would drift from the thing it claims to check). Proven red against the colliding version:

AssertionError: ('[field-failure] store_unwritable on Darwin (py 3.12)',
                 '[field-failure] store_unwritable on Darwin (py 3.12)')

Recorded on the Field Failure Reporting and Auto-Triage blueprint as a contract on the key: the dedupe key must carry every dimension the aggregate groups by.

Also fixed

docs/MODULE_MAP.md had gone stale as main moved under the branch. That single cause produced both red signals: Syntax & Lint failing on gen_module_map.py --check, and Drift Bot's only finding ("field_report.py is missing from MODULE_MAP.md"). Regenerated — field_report.py is now at docs/MODULE_MAP.md:167 and the map is in sync.

837 tests pass locally across the field-report, autoclose, module-map, workflow-YAML and workflow-integrity suites.

We already run this loop end to end for ONE class of failure: the desktop
shell classifies a bootstrap that never completed, the cloud sink counts
signatures, field-failure-issues.yml files a deduped issue, and the hourly
fixer routine picks it up. It is live and it works (a Windows machine on a
blocked index surfaced as no_distribution/Windows/3.11 with nobody reporting
it).

That pipeline had one producer, and it was the installer. The daemon, which
runs on every node and is the single point of failure for all customer data,
reported nothing when it could not run. On 2026-09-08 a Pro node's daemon was
locked out of its own pid lock and retried every 30s for twelve hours: it
exits 0 so launchd recorded successes, install_daemon_error_event_handler
(sync.py:23972) and start_update_check_thread (sync.py:24459) both initialise
AFTER the gate at 23832 so neither ever ran, and with no daemon there was no
heartbeat, so the node just went quiet. Quiet is not an event. The failure
existed in one place: a log file on the customer's laptop.

clawmetry/field_report.py makes the daemon the second producer:

- daemon_lock_refused, reported from ABOVE the gate (inline, not a thread:
  the process exits on the next line and would take a background thread with
  it) and only when ingest is actually stale, because refusing the lock is
  normally correct and frequent.
- daemon_ingest_stalled, from the watchdog thread that keeps running when the
  ingest loop does not. This is detectors.py's no_progress question, asked
  about ourselves for the first time.

Same privacy contract as the shell's, asserted key by key over the serialised
body: a closed enum, platform names, a version. Same opt-outs, and the
CANONICAL egress gate: endpoints.egress_suppressed(), not the narrower
is_custom_endpoint(), because an air-gapped node sets no endpoint and a
self-hosted server IS the endpoint. Verified live: air-gapped sends nothing.

Throttled by an on-disk stamp, since a restart loop is a new process every 30
seconds and in-memory throttling would throttle nothing. The session id is
derived as <class>-<UTC date> because the sink is keyed UNIQUE (install_id,
session_id, stage) with ON CONFLICT DO NOTHING: a constant would have recorded
the first daemon failure an install ever had and muted every one after it.

Verified end to end against a local stand-in for the sink, reproducing the
customer's machine: before, the failure reached nobody; after, one classified
aggregate arrives, and three more restarts into the same failure add none.
Needs clawmetry-cloud PR (new stage + enum entries + the aggregate view) to
reach the issue filer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CYQo5mXvPp3zT9tjq5mEaj
@8090-software-factory

Copy link
Copy Markdown

⚠️ Drift Bot (ClawMetry): 3 potential drift finding(s)

1. Requirement: Field Failure Reporting and Auto-Triage

File: clawmetry/field_report.py:1

The requirement specifies field failure reporting for "desktop shell bootstrap failures" (AC-FFR-001.1), but this PR implements daemon-specific failure reporting (daemon_lock_refused, daemon_ingest_stalled) that extends the scope significantly beyond what is documented in the acceptance criteria.

2. Blueprint: Field Failure Reporting and Auto-Triage

File: clawmetry/field_report.py

The blueprint for Field Failure Reporting and Auto-Triage is empty/template only, with no documented architecture, contracts, or specifications, while the implementation adds a new field_report.py module with specific failure classes and reporting logic.

3. Requirement: Field Failure Reporting and Auto-Triage

File: clawmetry/sync.py:399-422

The requirement's delivery status indicates daemon failure reporting is "NOT BUILT" and scoped to the clawmetry-cloud repo, but this PR implements _report_if_ingest_stalled() in the daemon's sync.py with a configurable stall detection threshold (_STALLED_INGEST_SECS).

Comment thread clawmetry/field_report.py Outdated
@@ -0,0 +1,265 @@
"""Field-failure reports from the sync daemon (Requirement: Field Failure

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Drift Bot (ClawMetry) — Requirement: Field Failure Reporting and Auto-Triage

The requirement specifies field failure reporting for "desktop shell bootstrap failures" (AC-FFR-001.1), but this PR implements daemon-specific failure reporting (daemon_lock_refused, daemon_ingest_stalled) that extends the scope significantly beyond what is documented in the acceptance criteria.

Comment thread clawmetry/sync.py
Comment on lines +399 to +422
# A sync cycle is about a minute. An hour without one completing is not a slow
# machine, it is a daemon that is alive and not working: the failure mode that
# has no supervisor at all, because the process is up and every liveness probe
# in the stack says so.
_STALLED_INGEST_SECS = float(os.environ.get("CLAWMETRY_STALLED_INGEST_SECS", "") or 3600)


def _report_if_ingest_stalled() -> None:
"""detectors.py ships ``no_progress`` to tell a customer their agent has
stopped getting anywhere. This is the same question asked about ourselves,
from the watchdog thread, which keeps running when the ingest loop does
not. Throttled to one report per six hours by the field-report stamp.
"""
try:
from clawmetry import field_report as _fr

age = _fr.last_sync_age_secs()
if age is not None and age > _STALLED_INGEST_SECS:
_fr.report_daemon_failure("daemon_ingest_stalled",
version=_get_version())
except Exception as e: # noqa: BLE001 - the watchdog must never die
log.debug("stall check skipped: %s", e)


Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Drift Bot (ClawMetry) — Requirement: Field Failure Reporting and Auto-Triage

The requirement's delivery status indicates daemon failure reporting is "NOT BUILT" and scoped to the clawmetry-cloud repo, but this PR implements _report_if_ingest_stalled() in the daemon's sync.py with a configurable stall detection threshold (_STALLED_INGEST_SECS).

vivekchand added a commit that referenced this pull request Sep 9, 2026
The lock fix stops this node breaking again; it does nothing about the fact
that a node in this state reaches nobody. #5752 makes the daemon a producer
on the field-failure pipeline. Also re-triggers the CodeQL analyses, which
queue-priority cancelled when main moved and which cannot be rerun in place.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CYQo5mXvPp3zT9tjq5mEaj
Drift Bot was right: this extends "Field Failure Reporting and Auto-Triage"
from install-time to run-time failures, and that scope change existed only in
the code. The parent requirement scopes AC-FFR-001 to the desktop shell's
bootstrap and lists daemon reporting as NOT BUILT.

Written as a LIVE child requirement rather than an edit to the parent, since
a pending suggestion is not a record: "Daemon Field-Failure Reporting"
(AC-FFR-005.1 through .9), carrying the reason the daemon reached nobody --
every self-repair mechanism it has initialises after the gate that failed, it
exits 0 so the supervisor counts successes, and a dead daemon sends no
heartbeat -- and the boundary that this bounds DETECTION latency and repairs
nothing.

Tests now name the criteria they cover.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CYQo5mXvPp3zT9tjq5mEaj
@8090-software-factory

Copy link
Copy Markdown

⚠️ Drift Bot (ClawMetry): 3 potential drift finding(s)

1. Blueprint: Daemon Field-Failure Reporting

File: clawmetry/field_report.py:1

The blueprint is a template placeholder with no architecture, contracts, or specifications, while the implementation in clawmetry/field_report.py delivers a complete field-failure reporting system with specific failure classes (daemon_lock_refused, daemon_ingest_stalled), privacy contracts, session ID strategies, and throttling mechanisms that must be documented.

2. Blueprint: Field Failure Reporting and Auto-Triage

File: clawmetry/field_report.py:1

The parent blueprint is a template placeholder with no documented architecture or contracts. The implementation includes two distinct failure classes (daemon_lock_refused from sync.py:23858 and daemon_ingest_stalled from sync.py:433) with different triggering mechanisms (inline vs. watchdog thread) that should be specified in the architecture documentation.

3. Blueprint: Daemon Field-Failure Reporting

File: docs/MODULE_MAP.md:166

The MODULE_MAP.md entry for clawmetry/field_report.py appears incomplete or truncated in the diff, showing only "Field-failure reports from the sync daemon (Requirement: Field Failure |" without proper markdown table closure.

Comment thread clawmetry/field_report.py Outdated
@@ -0,0 +1,268 @@
"""Field-failure reports from the sync daemon.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Drift Bot (ClawMetry) — Blueprint: Daemon Field-Failure Reporting

The blueprint is a template placeholder with no architecture, contracts, or specifications, while the implementation in clawmetry/field_report.py delivers a complete field-failure reporting system with specific failure classes (daemon_lock_refused, daemon_ingest_stalled), privacy contracts, session ID strategies, and throttling mechanisms that must be documented.

Comment thread clawmetry/field_report.py Outdated
@@ -0,0 +1,268 @@
"""Field-failure reports from the sync daemon.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Drift Bot (ClawMetry) — Blueprint: Field Failure Reporting and Auto-Triage

The parent blueprint is a template placeholder with no documented architecture or contracts. The implementation includes two distinct failure classes (daemon_lock_refused from sync.py:23858 and daemon_ingest_stalled from sync.py:433) with different triggering mechanisms (inline vs. watchdog thread) that should be specified in the architecture documentation.

The map carries each module's first docstring line, so rewording one drifts
it. Caught by CI's own guard, which is the guard working.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CYQo5mXvPp3zT9tjq5mEaj
@8090-software-factory

Copy link
Copy Markdown

⚠️ Drift Bot (ClawMetry): 2 potential drift finding(s)

1. Blueprint: Daemon Field-Failure Reporting

File: clawmetry/field_report.py:1

The blueprint is a template placeholder with no architecture or specifications, while the implementation delivers a complete field-failure reporting system with specific failure classes, privacy contracts, session ID strategies, and throttling mechanisms that should be documented in the blueprint to maintain design-code alignment.

2. Blueprint: Field Failure Reporting and Auto-Triage

File: clawmetry/field_report.py:1

The parent blueprint is a template placeholder with no documented architecture or contracts. The implementation includes two distinct failure classes with different triggering mechanisms (daemon_lock_refused inline vs. daemon_ingest_stalled from watchdog thread) that should be specified in the blueprint's architecture documentation.

Comment thread clawmetry/field_report.py Outdated
@@ -0,0 +1,268 @@
"""Field-failure reports from the sync daemon.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Drift Bot (ClawMetry) — Blueprint: Daemon Field-Failure Reporting

The blueprint is a template placeholder with no architecture or specifications, while the implementation delivers a complete field-failure reporting system with specific failure classes, privacy contracts, session ID strategies, and throttling mechanisms that should be documented in the blueprint to maintain design-code alignment.

Comment thread clawmetry/field_report.py Outdated
@@ -0,0 +1,268 @@
"""Field-failure reports from the sync daemon.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Drift Bot (ClawMetry) — Blueprint: Field Failure Reporting and Auto-Triage

The parent blueprint is a template placeholder with no documented architecture or contracts. The implementation includes two distinct failure classes with different triggering mechanisms (daemon_lock_refused inline vs. daemon_ingest_stalled from watchdog thread) that should be specified in the blueprint's architecture documentation.

Drift Bot's remaining objection was that both blueprints were still template
placeholders while the code carried real architecture. Both are now written:
the child documents this feature (five ADRs), and the parent documents the
pipeline it joins, which had shipped and been live for weeks against an empty
blueprint.

The three choices here that look arbitrary and are not now name their ADR at
the call site, so the next reader finds the reasoning rather than rediscovering
it: report from above the gate, send inline on a path that exits, derive the
session id rather than fix it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CYQo5mXvPp3zT9tjq5mEaj
@8090-software-factory

Copy link
Copy Markdown

⚠️ Drift Bot (ClawMetry): 1 potential drift finding(s)

1. Blueprint: Daemon Field-Failure Reporting

File: docs/MODULE_MAP.md

The new clawmetry/field_report.py module is implemented but missing from the generated MODULE_MAP.md documentation. The MODULE_MAP file is auto-generated and should include an entry documenting the field_report module's purpose and relationship to the requirements.

Drift Bot reported clawmetry/field_report.py as missing from MODULE_MAP.md.
It was not missing: it was at line 166, cut mid-clause, because the generator
took the first LINE of the docstring and that sentence wrapped.

Thirteen other modules were already in that state, so the table has been
quietly describing detectors.py as "research-backed, judge-free, CPU-cheap
trajec" for as long as anyone has been reading it. Nothing failed, because the
generator and its --check gate agreed with each other perfectly on a truncated
string.

_summary() now joins the first paragraph and cuts at the first sentence end,
ignoring the two things that look like one and are not (a version or section
number, an initial), bounded at 200 chars. Wrapping a line is a formatting
choice and must not change generated documentation. 17 rows become whole
sentences; no row loses information.

tests/test_module_map_summaries_are_whole.py: an auto-discovering check that
no row ends mid-sentence, plus the generator's behaviour pinned directly.
Proven red against origin/main's generator (5 failures). Named in ci.yml.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CYQo5mXvPp3zT9tjq5mEaj
@8090-software-factory

Copy link
Copy Markdown

⚠️ Drift Bot (ClawMetry): 1 potential drift finding(s)

1. Blueprint: Daemon Field-Failure Reporting

File: docs/MODULE_MAP.md

The new clawmetry/field_report.py module is implemented but missing from the auto-generated docs/MODULE_MAP.md file. The MODULE_MAP should include an entry documenting the module's purpose to maintain documentation-code alignment.

vivekchand added a commit that referenced this pull request Sep 9, 2026
The lock fix stops this node breaking again; it does nothing about the fact
that a node in this state reaches nobody. #5752 makes the daemon a producer
on the field-failure pipeline. Also re-triggers the CodeQL analyses, which
queue-priority cancelled when main moved and which cannot be rerun in place.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CYQo5mXvPp3zT9tjq5mEaj
@8090-software-factory

Copy link
Copy Markdown

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

Copy link
Copy Markdown
Owner Author

✨ auto-fixed: merged latest main into branch to keep it up to date


Generated by Claude Code

@8090-software-factory

Copy link
Copy Markdown

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

Copy link
Copy Markdown
Owner Author

✨ auto-fixed: merged latest main into branch (was BEHIND; no conflicts)


Generated by Claude Code

vivekchand added a commit that referenced this pull request Sep 9, 2026
#5745) (#5751)

* [RELEASE] daemon lock self-heal + honest sync status (carries #5745)

Publishes the fix for the field failure reported 2026-09-09: a Pro node whose
sync daemon had been locked out of its own pid lock since the previous
evening, behind a status screen of green ticks.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CYQo5mXvPp3zT9tjq5mEaj

* Note the follow-up in the CHANGELOG entry

The lock fix stops this node breaking again; it does nothing about the fact
that a node in this state reaches nobody. #5752 makes the daemon a producer
on the field-failure pipeline. Also re-triggers the CodeQL analyses, which
queue-priority cancelled when main moved and which cannot be rerun in place.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CYQo5mXvPp3zT9tjq5mEaj

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Resolves conflict in docs/MODULE_MAP.md: take main's addition of
openclaw_share.py adapter row alongside the existing openclaw.py entry.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UNcAS3GeEW3FkQyZ2vcsrF
@8090-software-factory

Copy link
Copy Markdown

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

Fixes the Syntax & Lint CI failure:
  docs/MODULE_MAP.md is out of date.
  Regenerate it with: python3 scripts/gen_module_map.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBmS1miHmw8rA5a8jny1Qr
@8090-software-factory

Copy link
Copy Markdown

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

@8090-software-factory

Copy link
Copy Markdown

⚠️ Drift Bot (ClawMetry): 1 potential drift finding(s)

1. Blueprint: Daemon Field-Failure Reporting

File: docs/MODULE_MAP.md

The new clawmetry/field_report.py module is implemented with comprehensive docstring documentation but is missing from the auto-generated MODULE_MAP.md. The MODULE_MAP should include an entry for this module between clawmetry/extensions.py and clawmetry/flow_trace.py to maintain documentation completeness.
true

vivekchand and others added 2 commits September 10, 2026 08:06
Merging main brought in #5790, whose reconcile step closes a field-failure
issue when its signature stops being reported. Both branches touch the same
workflow; the merge was textually clean and the autoclose tests still pass,
but it exposed a collision this PR introduces.

/api/desktop/_failures groups by stage x class x OS x Python. The issue title
is the dedupe key and carried only class/OS/Python, so a daemon that stopped
ingesting and an install that never started could share a title. The second to
arrive would merely refresh the first, under a lede describing the wrong
failure and pointing the fixer at the wrong file -- the exact confusion the
new per-stage lede exists to prevent. Demonstrated: same class, OS and Python,
two stages, one title.

A stage other than bootstrap_failed now prefixes the title. bootstrap_failed
keeps the historical shape byte for byte, because renaming it orphans every
open issue, which the title contract forbids; a new stage takes a new prefix
rather than reshaping the existing key. Pinned by three tests that compute the
title with the workflow's OWN shell rather than a reimplementation, and proven
red against the colliding version. Recorded on the Field Failure Reporting and
Auto-Triage blueprint as a contract on the key.

docs/MODULE_MAP.md was stale after the merge, which is what reddened Syntax &
Lint AND produced Drift Bot's only finding (field_report.py 'missing' from the
map). One root cause, both cleared by regenerating.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xb6A5G74JiMe3zHFs1JZEP
@8090-software-factory

Copy link
Copy Markdown

⚠️ Drift Bot (ClawMetry): 1 potential drift finding(s)

1. Blueprint: Daemon Field-Failure Reporting

File: docs/MODULE_MAP.md

The blueprint specifies that clawmetry/field_report.py is a feature-specific component that should be documented. The module is implemented with comprehensive docstrings, but it is missing from the auto-generated MODULE_MAP.md file, which should include an entry between clawmetry/efficiency.py and clawmetry/endpoints.py to maintain documentation completeness.

The blueprint's status banner said 'until BOTH merge', but clawmetry-cloud
#2364 merged 2026-09-09 and is deployed: /api/desktop/_failures returns a
stage on every signature today, verified against the live endpoint. So the
ordering this feature needs is already satisfied and holding this PR only
leaves the cloud change unused.

The banner now says which half is live and which is in review, and records
the caveat found while merging: a drift finding that field_report.py is absent
from MAIN is correct and expected until this merges, but a finding that it is
missing from this PR's own MODULE_MAP.md has now been wrong twice -- the entry
sits between extensions.py and flow_trace.py, alphabetically correct, while
one finding proposed a position between efficiency.py and endpoints.py that it
could not occupy.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xb6A5G74JiMe3zHFs1JZEP
@8090-software-factory

Copy link
Copy Markdown

✅ Drift Bot (ClawMetry): no drift detected

Drift Bot analyzed the changed files against this project's blueprints and requirements and found no drift.

@vivekchand
vivekchand merged commit 77747b6 into main Sep 10, 2026
45 of 46 checks passed
@vivekchand
vivekchand deleted the feat/daemon-field-report branch September 10, 2026 07:50
vivekchand added a commit that referenced this pull request Sep 10, 2026
@vivekchand

Copy link
Copy Markdown
Owner Author

Live in PyPI 0.12.855 — verified by driving the published wheel, not the branch.

Downloaded clawmetry-0.12.855-py3-none-any.whl, unpacked it, imported clawmetry.field_report from those bytes (asserted __file__ resolves inside the unpacked wheel):

module: /tmp/r855/x/clawmetry/field_report.py   (11,290 bytes)
FAILURE_CLASSES: ['daemon_ingest_stalled', 'daemon_lock_refused']

Both call sites ship in clawmetry/sync.py: _report_if_ingest_stalled() at :406 raising daemon_ingest_stalled at :417, and daemon_lock_refused at :24174 — with no lock-acquire between it and the end of the function, i.e. still above the pid-lock gate, which is the entire point of the feature.

The payload, from the shipped code:

{"arch": "arm64", "bootstrap_python": "3.14", "desktop_version": "0.12.855",
 "failure_class": "daemon_lock_refused", "install_id": "763e5d9c-…",
 "os": "Darwin", "os_version": "25.3.0",
 "session_id": "daemon_lock_refused-20260910", "stage": "daemon_failed"}

Privacy asserted over the serialised body, not key by key, so a nested value cannot smuggle anything past a key check:

checked for present
home path no
username no
hostname no
cwd no

Suppression is total: CLAWMETRY_OFFLINE=1report_daemon_failure(...) returns False, nothing sent.

last_sync_age_secs() is tri-state: returns None when no sync has ever completed, rather than 0 — so "never ran" cannot masquerade as "just ran" and trigger a false stall report.

The sink half was already live and stage-aware (clawmetry-cloud #2364, merged 2026-09-09): /api/desktop/_failures returns a stage on every signature today. The pipeline is now complete end to end — daemon producer on PyPI, sink deployed, filer wording each stage correctly with a per-stage dedupe key.

No further cloud promotion needed: the cloud change shipped a day ago, and this is the node-side half that reaches machines through the daemon's own PyPI self-update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants