feat(adapters/openclaw): surface nemoclaw onboarding OTel trace artifacts (#5193) - #5198
Conversation
✅ Drift Bot (ClawMetry): no drift detectedDrift Bot analyzed the changed files against this project's blueprints and requirements and found no drift. |
|
![8090-software-factory[bot]](https://avatars.githubusercontent.com/in/1847834?s=60&v=4){kind=small}
| return result | ||
|
|
||
|
|
||
| def _nemoclaw_onboard_trace() -> dict: |
There was a problem hiding this comment.
The OpenClaw adapter now surfaces NemoClaw onboarding OTel trace metadata (nemoclawOnboardTraceStatus, nemoclawOnboardTraceSpanCount, nemoclawOnboardTraceErrors, nemoclawOnboardSlowSpans) through the detect() method, but this functionality is not documented in the Runtime and Session Observability requirement or its corresponding blueprint.
|
Merged latest main into this branch to resolve behind-head state and re-trigger CI. Generated by Claude Code |
|
| return result | ||
|
|
||
|
|
||
| def _nemoclaw_onboard_trace() -> dict: |
There was a problem hiding this comment.
The OpenClaw adapter now surfaces NemoClaw onboarding OTel trace metadata (nemoclawOnboardTraceStatus, nemoclawOnboardTraceSpanCount, nemoclawOnboardTraceErrors, nemoclawOnboardSlowSpans) through the detect() method, but this functionality is not documented in the Runtime and Session Observability requirement.
| return result | ||
|
|
||
|
|
||
| def _nemoclaw_onboard_trace() -> dict: |
There was a problem hiding this comment.
The OpenClaw adapter now surfaces NemoClaw onboarding OTel trace metadata through the detect() method, but this functionality is not documented in the Runtime and Session Observability blueprint.
6f38e9d to
5c64f6a
Compare
|
| return result | ||
|
|
||
|
|
||
| def _nemoclaw_onboard_trace() -> dict: |
There was a problem hiding this comment.
The OpenClaw adapter now surfaces NemoClaw onboarding OTel trace metadata (nemoclawOnboardTraceStatus, nemoclawOnboardTraceSpanCount, nemoclawOnboardTraceErrors, nemoclawOnboardSlowSpans) through the detect() method, but this functionality is not documented in the Runtime and Session Observability blueprint.
| return result | ||
|
|
||
|
|
||
| def _nemoclaw_onboard_trace() -> dict: |
There was a problem hiding this comment.
The OpenClaw adapter now surfaces NemoClaw onboarding OTel trace metadata (nemoclawOnboardTraceStatus, nemoclawOnboardTraceSpanCount, nemoclawOnboardTraceErrors, nemoclawOnboardSlowSpans) through the detect() method, but this functionality is not documented in the Runtime and Session Observability requirement.
|
blocked on author decision — skipping (auto-mergeability sweep) Drift Bot found 2 drift finding(s) on the head commit. Blueprint contradictions require author review. Generated by Claude Code |
…-ingest (PR #5198) Bring branch up to date with main. Co-Authored-By: Claude <noreply@anthropic.com>
|
✨ auto-fixed: merged main into branch to bring it up to date (was BEHIND by multiple commits) Generated by Claude Code |
|
| return result | ||
|
|
||
|
|
||
| def _nemoclaw_onboard_trace() -> dict: |
There was a problem hiding this comment.
The OpenClaw adapter now surfaces NemoClaw onboarding OTel trace metadata (nemoclawOnboardTraceStatus, nemoclawOnboardTraceSpanCount, nemoclawOnboardTraceErrors, nemoclawOnboardSlowSpans) through the detect() method, but this observability capability is not documented in the Runtime and Session Observability requirement.
| return result | ||
|
|
||
|
|
||
| def _nemoclaw_onboard_trace() -> dict: |
There was a problem hiding this comment.
The OpenClaw adapter now surfaces NemoClaw onboarding OTel trace metadata (nemoclawOnboardTraceStatus, nemoclawOnboardTraceSpanCount, nemoclawOnboardTraceErrors, nemoclawOnboardSlowSpans) through the detect() method, but this observability capability is not documented in the Runtime and Session Observability blueprint.
|
| return result | ||
|
|
||
|
|
||
| def _nemoclaw_onboard_trace() -> dict: |
There was a problem hiding this comment.
The OpenClaw adapter now surfaces NemoClaw onboarding OTel trace metadata (nemoclawOnboardTraceStatus, nemoclawOnboardTraceSpanCount, nemoclawOnboardTraceErrors, nemoclawOnboardSlowSpans) through the detect() method, but this observability capability is not documented in the Runtime and Session Observability requirement.
| return result | ||
|
|
||
|
|
||
| def _nemoclaw_onboard_trace() -> dict: |
There was a problem hiding this comment.
The OpenClaw adapter now surfaces NemoClaw onboarding OTel trace metadata (nemoclawOnboardTraceStatus, nemoclawOnboardTraceSpanCount, nemoclawOnboardTraceErrors, nemoclawOnboardSlowSpans) through the detect() method, but this functionality is not documented in the Runtime and Session Observability blueprint.
|
✨ auto-fixed: merged latest main into branch to resolve BEHIND state Generated by Claude Code |
|
| return result | ||
|
|
||
|
|
||
| def _nemoclaw_onboard_trace() -> dict: |
There was a problem hiding this comment.
The OpenClaw adapter now surfaces NemoClaw onboarding OTel trace metadata (nemoclawOnboardTraceStatus, nemoclawOnboardTraceSpanCount, nemoclawOnboardTraceErrors, nemoclawOnboardSlowSpans) through the detect() method, but this new observability capability is not documented in the Runtime and Session Observability requirement.
| return result | ||
|
|
||
|
|
||
| def _nemoclaw_onboard_trace() -> dict: |
There was a problem hiding this comment.
The OpenClaw adapter now surfaces NemoClaw onboarding OTel trace metadata (nemoclawOnboardTraceStatus, nemoclawOnboardTraceSpanCount, nemoclawOnboardTraceErrors, nemoclawOnboardSlowSpans) through the detect() method, but this functionality is not documented in the Runtime and Session Observability blueprint.
|
E2E Gate is blocked by Drift Bot failing on Generated by Claude Code |
|
blocked on author decision — skipping (auto-mergeability sweep) Drift-bot is reporting blueprint drift (2 findings) for this PR. The fix requires updating the corresponding Blueprint at factory.8090.ai — that's an author action outside what the sweep can automate. Generated by Claude Code |
|
| return result | ||
|
|
||
|
|
||
| def _nemoclaw_onboard_trace() -> dict: |
There was a problem hiding this comment.
REQ-OBS-RSO-002 (Inspect trace context) specifies presenting available trace-related activity records but does not document the NemoClaw-specific onboarding OTel trace metadata fields (nemoclawOnboardTraceStatus, nemoclawOnboardTraceSpanCount, nemoclawOnboardTraceErrors, nemoclawOnboardSlowSpans) that the code surfaces through the OpenClaw adapter's detect() method.
| return result | ||
|
|
||
|
|
||
| def _nemoclaw_onboard_trace() -> dict: |
There was a problem hiding this comment.
The blueprint's trace context inspection section does not document the _nemoclaw_onboard_trace() helper, which implements NemoClaw onboarding OTel trace artifact surfacing with path resolution (NEMOCLAW_TRACE_FILE/NEMOCLAW_TRACE_DIR/.e2e/traces/trace.json fallback) and surfaces nemoclawOnboardTraceStatus, nemoclawOnboardTraceSpanCount, nemoclawOnboardTraceErrors, and nemoclawOnboardSlowSpans.
|
|
||
| _STATUS_RANK = {"ERROR": 2, "UNSET": 1, "OK": 0} | ||
| worst_rank = -1 | ||
| worst_status = "UNKNOWN" |
There was a problem hiding this comment.
The implementation includes specific observability design details (error detection via span-status ranking with ERROR/UNSET/OK priorities, error phase name extraction from ERROR spans, slowest_spans output capped at 5 entries, support for both flat harness and resource_spans OTel trace formats) that are not documented in the blueprint's approach to trace context inspection.
PR health check — 2026-09-03Repo: vivekchand/clawmetry Status: Blocked by Drift Bot since 2026-08-25. Code and tests are complete; 2 blueprint documentation gaps are blocking the E2E Gate. Items to add in the Software Factory:
No code changes needed. After updating both entries and re-triggering drift-bot, this PR should be merge-ready. Smoke (once gate clears): Generated by Claude Code |
|
Blocked on author decision — skipping (auto-mergeability sweep): Generated by Claude Code |
vivekchand
left a comment
vivekchand
left a comment
There was a problem hiding this comment.
Test plan & review notes
Repo: vivekchand/clawmetry
What changed
clawmetry/adapters/openclaw.py: adds_nemoclaw_onboard_trace()— reads NeMoClaw harness OTel trace files (flat{spans:[...]}or standardresource_spansshape), extracts worst-case status, error phase names, and slowest-span summary; surfacesnemoclawOnboardTraceStatus/SpanCount/Errors/SlowSpansintodetect()metadata whenNEMOCLAW_TRACE=1.tests/test_obs_gap_nemoclaw_onboard_trace_5193.py(new, 190 lines): 10 test cases covering all trace shapes, disabled states, missing files, camelCase OTel keys, and the slow-span forwarding.
Smoke commands
pytest tests/test_obs_gap_nemoclaw_onboard_trace_5193.py -v # 10 tests
# Manually:
NEMOCLAW_TRACE=1 NEMOCLAW_TRACE_FILE=/tmp/t.json python3 -c "
import json; json.dump({'spans':[{'name':'phase.gateway','status':'ERROR'}]}, open('/tmp/t.json','w'))
import clawmetry.adapters.openclaw as oc; print(oc._nemoclaw_onboard_trace())
"
# Expected: {'nemoclawOnboardTraceStatus': 'ERROR', 'nemoclawOnboardTraceSpanCount': 1, 'nemoclawOnboardTraceErrors': ['phase.gateway']}Likely failure modes from the diff
NEMOCLAW_TRACEnot set → returns{}silently — verify this doesn't log a warning.- OTel
scopeSpanscamelCase key — covered bytest_resource_spans_camel_case_scope_spans. - Zone-id on file path on Windows —
errors="replace"handles encoding issues.
Blocker: 2 Drift Bot findings on the Runtime and Session Observability requirement and blueprint — NeMoClaw OTel trace surfacing not documented. Needs factory.8090.ai update, then a push to retrigger.
Issue link: Closes #5193
Generated by Claude Code
…acts (#5193) The NemoClaw harness writes OpenTelemetry-style spans for each onboarding phase when NEMOCLAW_TRACE is set, but the adapter discarded the file entirely. Adds _nemoclaw_onboard_trace() which resolves the file via NEMOCLAW_TRACE_FILE / NEMOCLAW_TRACE_DIR / .e2e/traces/trace.json, parses both the flat harness shape and the OTel resource_spans format, and surfaces worst-case status, error phase names, and summary.slowest_spans so a failed or slow onboarding step is diagnosable from the dashboard. Closes #5193 Co-Authored-By: ClawMetry Autofix Bot <bot-autofix@clawmetry.dev>
Add two tests covering the standard OTel export shape (resource_spans → scope_spans/scopeSpans → spans) to close the gap identified in babysit review — only the flat harness format was previously tested. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01751f3a-c1f5-5543-a2cf-cc88f096e807
365814c to
e6304fe
Compare
|
Auto-rebase pushed; CI now running. If still not green in 10min, may need manual attention. Generated by Claude Code |
|
| return result | ||
|
|
||
|
|
||
| def _nemoclaw_onboard_trace() -> dict: |
There was a problem hiding this comment.
REQ-OBS-RSO-002 (Inspect trace context) specifies presenting available trace-related activity records but does not document the NemoClaw-specific onboarding OTel trace metadata fields (nemoclawOnboardTraceStatus, nemoclawOnboardTraceSpanCount, nemoclawOnboardTraceErrors, nemoclawOnboardSlowSpans) that the code surfaces through the OpenClaw adapter's detect() method.
| return result | ||
|
|
||
|
|
||
| def _nemoclaw_onboard_trace() -> dict: |
There was a problem hiding this comment.
The blueprint's trace context inspection section does not document the _nemoclaw_onboard_trace() helper, which implements NemoClaw onboarding OTel trace artifact surfacing with path resolution (NEMOCLAW_TRACE_FILE/NEMOCLAW_TRACE_DIR/.e2e/traces/trace.json fallback) and surfaces nemoclawOnboardTraceStatus, nemoclawOnboardTraceSpanCount, nemoclawOnboardTraceErrors, and nemoclawOnboardSlowSpans.
| if isinstance(raw, list): | ||
| spans = raw | ||
| elif "resource_spans" in data: | ||
| for rs in data.get("resource_spans", []): |
There was a problem hiding this comment.
The implementation includes specific observability design details (error detection via span-status ranking with ERROR/UNSET/OK priorities, error phase name extraction from ERROR spans, slowest_spans output capped at 5 entries, support for both flat harness and resource_spans OTel trace formats) that are not documented in the blueprint's approach to trace context inspection.
|
|
Auto-merge-base pushed (branch was behind main at Generated by Claude Code |
| return result | ||
|
|
||
|
|
||
| def _nemoclaw_onboard_trace() -> dict: |
There was a problem hiding this comment.
REQ-OBS-RSO-002 (Inspect trace context) specifies presenting available trace-related activity records but does not document the NemoClaw-specific onboarding OTel trace metadata fields (nemoclawOnboardTraceStatus, nemoclawOnboardTraceSpanCount, nemoclawOnboardTraceErrors, nemoclawOnboardSlowSpans) that the code surfaces through the OpenClaw adapter's detect() method.
| return result | ||
|
|
||
|
|
||
| def _nemoclaw_onboard_trace() -> dict: |
There was a problem hiding this comment.
The blueprint's trace context inspection section does not document the _nemoclaw_onboard_trace() helper, which implements NemoClaw onboarding OTel trace artifact surfacing with path resolution (NEMOCLAW_TRACE_FILE/NEMOCLAW_TRACE_DIR/.e2e/traces/trace.json fallback) and surfaces nemoclawOnboardTraceStatus, nemoclawOnboardTraceSpanCount, nemoclawOnboardTraceErrors, and nemoclawOnboardSlowSpans.
| if isinstance(raw, list): | ||
| spans = raw | ||
| elif "resource_spans" in data: | ||
| for rs in data.get("resource_spans", []): |
There was a problem hiding this comment.
The implementation includes specific observability design details (error detection via span-status ranking with ERROR/UNSET/OK priorities, error phase name extraction from ERROR spans, slowest_spans output capped at 5 entries, support for both flat harness and resource_spans OTel trace formats) that are not documented in the blueprint's approach to trace context inspection.
…w onboarding trace
The standard OTel JSON export writes span status as {"code": N} (0 UNSET,
1 OK, 2 ERROR) and some exporters as STATUS_CODE_*. The ranking compared
raw strings, so an export-shaped trace reported "{'CODE': 2}" as its
status. Map both forms (and unrecognised values) onto OK/ERROR/UNSET so
nemoclawOnboardTraceStatus is always one of the three documented values.
Spec: REQ-OBS-RSO-034 / AC-OBS-RSO-002.3 and the "NemoClaw onboarding
trace artifacts" blueprint section (Runtime and Session Observability).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KpoJsfi8kkY29uvPWNQj3P
…moclaw-onboard-otel-trace-ingest
✅ Drift Bot (ClawMetry): no drift detectedDrift Bot analyzed the changed files against this project's blueprints and requirements and found no drift. |
…y partly read (#5764) Closes #5750. #5750 was filed automatically at severity **high**: "No code path in the OpenClaw/NemoClaw adapter reads NEMOCLAW_TRACE_FILE / the .e2e/traces directory or parses this trace artifact." That code has shipped. `clawmetry/adapters/openclaw.py:1705` reads `NEMOCLAW_TRACE_FILE`, falls back to `NEMOCLAW_TRACE_DIR` and then the harness default, and puts `nemoclawOnboardTraceStatus`, `nemoclawOnboardTraceSpanCount`, `nemoclawOnboardTraceErrors` and `nemoclawOnboardSlowSpans` on the detection record. `clawmetry/adapters/nemo.py` reads it too. REQ-OBS-RSO-034 specifies the whole capability, delivered by PR #5198. The auditor never saw any of it. `_adapter_source` read the first 60,000 characters; `openclaw.py` is 193,340. The reader sits at line 1690, roughly 18k characters past the cut, so **69% of the adapter was invisible** -- and both runtimes this OSS audit covers map to that same file, so every run judged a two-thirds-clipped adapter. Then the prompt told the model: "verify against the FULL adapter above (it is provided in full)". For a task that is entirely about reporting ABSENCE, that sentence converts "I did not see it" into "it is not there". The model did exactly what it was told the evidence supported. This is the second time. The docstring records the first: aider's conditional COST at line ~527 was cut, the audit flagged "no COST", and the cap was raised to 60k in response. Raising a number is not a fix for a file that grows. * the budget is now far above any adapter here, and `_adapter_source` returns whether it trimmed rather than trimming silently; * when it DOES trim, the prompt says so instead of claiming completeness; * `_adapter_index` always carries every `def` and every UPPER_CASE string from the WHOLE file, so an absence claim stays checkable even under trimming. Small, complete, and built from the full source regardless of the body. tests/test_harness_audit_reads_whole_adapter.py auto-discovers from the manifest, so an adapter that outgrows the budget tomorrow fails here instead of quietly filing fiction. Guard proven: restoring the 60k cap and the old completeness claim reds 5 of 6, including both audited runtimes and the `NEMOCLAW_TRACE_FILE`-past-60k witness. The four sibling issues #5746 to #5749 were filed by the same truncated runs and each needs the same check against the full adapter before anyone works it. Claude-Session: https://claude.ai/code/session_01Xb6A5G74JiMe3zHFs1JZEP Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
1 participant
Product record: https://factory.8090.ai/project/b415065f-ab2f-4f53-8864-0c009fd098cb/requirements/8e389016-a9c8-4352-9121-72f0e361fdf6 (REQ-OBS-RSO-034 + AC-OBS-RSO-002.3) and blueprint https://factory.8090.ai/project/b415065f-ab2f-4f53-8864-0c009fd098cb/blueprints/40e35d24-8249-4f72-b311-f311dc71aec5 (section "NemoClaw onboarding trace artifacts", ADR-105).
Summary
When
NEMOCLAW_TRACEis set, the NemoClaw harness writes OTel-style spans for each onboarding phase to disk -- but the ClawMetry adapter was discarding the file entirely, so a failed or slow onboarding phase was invisible on the dashboard. This PR adds a_nemoclaw_onboard_trace()helper that reads the file and surfaces the worst-case status, error phase names, and slowest-span summary.Changes
Follow-up (2026-09-03): normalise OTel span status objects (
{code: N}/STATUS_CODE_*) sonemoclawOnboardTraceStatusis always one ofOK/ERROR/UNSET; two more unit tests (11 in the file).clawmetry/adapters/openclaw.py: Add_nemoclaw_onboard_trace() -> dictafter_model_router_live(). Resolves the trace file viaNEMOCLAW_TRACE_FILE->NEMOCLAW_TRACE_DIR/trace.json->.e2e/traces/trace.json(harness default). Handles both the flat harness{spans:[...], summary:{...}}shape and the standard OTelresource_spansexport. Self-gates onNEMOCLAW_TRACEenv var; returns{}when disabled or file absent. Wire it intodetect()after_workshop_approval_config().tests/test_obs_gap_nemoclaw_onboard_trace_5193.py: 7 unit tests covering disabled (env absent,=0), file missing ->{}, all-OK spans, ERROR span surfaces status + name,summary.slowest_spansforwarded, andNEMOCLAW_TRACE_DIRpath resolution.Test plan
python3 -m pytest tests/test_obs_gap_nemoclaw_onboard_trace_5193.py -v-- 7/7 pass (verified locally)python3 -c 'import ast; ast.parse(open("clawmetry/adapters/openclaw.py").read())'-- syntax OK (verified locally)NEMOCLAW_TRACE=1andNEMOCLAW_TRACE_FILEto a sample trace JSON; runclawmetryand confirmnemoclawOnboardTraceStatusappears in the adapter's DetectResult meta.Bot meta
Draft PR opened autonomously based on the plan in #5193. Marked draft for human review -- mark Ready for Review once happy.
Closes #5193