This package is a small, reliable polling worker for Tropibot. On a dedicated, disposable Dappnode it claims one package job at a time, resolves the requested baseline (the latest published release when omitted), reuses an already-installed target only when it matches that baseline, upgrades it to the candidate, captures bounded evidence, runs deterministic and advisory log analysis, restores the intended target state, and delivers the result back to Tropibot.
It is intentionally destructive. Never connect it to a personal or production Dappnode. The dedicated node is the safety boundary: cleanup is best effort and is not a security boundary. The harness refuses its own package and core packages, mutates only the claimed target, never mounts the Docker socket, and does not execute shell commands.
Tropibot claim → atomically persist claim → execute one package job
→ heartbeat phase/cancellation → cleanup → persist exact completion JSON
→ retry completion until Tropibot acknowledges it
The harness has no public job-submission or GitHub API. It needs outbound HTTPS access to Tropibot. Its package UI opens an embedded operator dashboard at / with local job history, worker state, cleanup and delivery status, and cleanup-recovery controls. The dashboard can also read the current worker-lost record from Tropibot and, after the operator confirms the old process is fenced and cleanup is complete, either release the worker or release it and requeue the package. These requests are proxied by the harness backend; the shared worker token is never exposed to browser JavaScript. GET /api/jobs supplies local history and GET /api/coordinator/lost-job supplies the coordinator recovery state. GET /healthz is a process check; GET /readyz also checks Dappmanager tool availability and unresolved local recovery state.
For every accepted job, the deterministic controller:
- verifies Dappmanager MCP tools and target safety;
- checks whether the target is already installed and records its exact version for restoration when appropriate;
- previews the requested
baselineRef, or previews the latest published release when it was omitted, and reuses an installed package only when its version matches that resolved baseline; - waits for a stable, non-empty all-running container set;
- captures normalized details and bounded, redacted log tails;
- upgrades the baseline in place to the candidate;
- repeats stabilization and capture, compares both observations, and performs optional Nexus analysis;
- restores a pre-existing target to its recorded version; for allowlisted slow packages, restores and retains the resolved published baseline for reuse; otherwise removes only the target with
deleteVolumes: true; and fails cleanup when final inventory cannot be verified or contains unexpected dependencies.
Nexus is advisory. With no key, the harness uses the heuristic analyzer only. With a key, it sends one bounded redacted request with no tools; Nexus failures fall back safely, and Nexus can never override a deterministic failure.
Required worker settings are validated on startup:
TROPIBOT_URL=https://tropibot.example
PACKAGE_HARNESS_WORKER_ID=worker-01
PACKAGE_HARNESS_WORKER_TOKEN=<shared trusted secret>
PACKAGE_HARNESS_POLL_SECONDS=10
PACKAGE_HARNESS_HEARTBEAT_SECONDS=20
The shared bearer token is appropriate for this small trusted v1 deployment. The protocol deliberately leaves room for later per-worker credentials or mTLS, but neither is part of this version. The worker never logs the token.
Use .env.example for all timeout, Dappmanager, cleanup, and optional Nexus settings. MCP_TIMEOUT_MS bounds read-only Dappmanager calls at 30 seconds by default, while MCP_MUTATION_TIMEOUT_MS gives each install, update, or removal attempt 30 minutes by default (up to one hour). Transient transport and package-download failures are retried three times with bounded exponential backoff; tune this with MCP_MUTATION_ATTEMPTS and MCP_MUTATION_RETRY_DELAY_MS. NEXUS_TIMEOUT_MS gives advisory Nexus analysis five minutes by default. RETAIN_BASELINE_PACKAGES is a comma-separated allowlist for packages with expensive downloads: after the first successful baseline install, the harness records its expected version and immutable release reference, restores it after candidate testing, and reuses it on later runs instead of removing and downloading it again. TROPIBOT_TIMEOUT_MS bounds each coordinator request. The Dappmanager MCP token and Nexus key remain local and are never sent to Tropibot or persisted in a job record.
Rust 1.96.1 is pinned by rust-toolchain.toml. The process reads a local .env if it exists; existing process variables take precedence.
mkdir -p .data
PACKAGE_MANAGER_MODE=fake \
DATA_DIR=./.data \
TROPIBOT_URL=https://tropibot.example \
PACKAGE_HARNESS_WORKER_ID=worker-01 \
PACKAGE_HARNESS_WORKER_TOKEN=development-token \
cargo runThe fake package manager makes ordinary candidates pass. A candidate reference containing unstable simulates non-running containers and install-error simulates a failed upgrade; both paths still run cleanup.
The default info logs are a concise execution narrative while retaining searchable event, run_id, package, duration, attempt, and status fields. Log levels carry severity, so messages do not repeat it with decorative prefixes. Long MCP mutations announce every attempt and configured deadline; active jobs emit a heartbeat liveness line about once per minute; stabilization logs each bounded container-health sample; Nexus logs its model, input size, timeout, duration, and fallback error; and cleanup reports both its selected recovery action and final inventory verification. Credentials and raw package evidence are never logged. Internal startup, persistence, and heartbeat lifecycle messages are available at debug.
Set RUST_LOG=dappnode_package_harness=debug,tower_http=info for polling and persistence diagnostics. The normal package default remains info to avoid logging an empty Tropibot poll on every interval.
The worker uses the fixed v1 paths:
POST /v1/package-harness/jobs/claim
POST /v1/package-harness/jobs/{jobId}/heartbeat
POST /v1/package-harness/jobs/{jobId}/complete
Every request carries Authorization: Bearer <PACKAGE_HARNESS_WORKER_TOKEN>, Content-Type: application/json, and User-Agent: dappnode-package-harness/<version>. Protocol DTOs and their golden fixtures live under src/coordinator and tests/fixtures.
Before mutating, the worker atomically persists the full claim and opaque claim token under DATA_DIR. It persists every recovery-relevant phase and, before network delivery, the exact serialized completion JSON. Transient failures and coordinator compatibility rejections retry the same bytes with capped backoff and worker-specific jitter, so a coordinator upgrade heals delivery without a worker restart. Successful delivery clears legacy delivery-recovery markers. The worker never claims another job until completion is acknowledged as recorded or duplicate.
On restart, a pending completion is retried first. An interrupted job is never rerun: if it may have mutated the target, the worker inspects Dappmanager, performs bounded cleanup, then sends an interrupted worker-error completion. A lost claim is released locally after cleanup succeeds; if Tropibot still holds unresolved coordinator state, the next claim receives a conflict and polling stops without inventing new local recovery work. Manual recovery is reserved for failed cleanup, ambiguous local records, or a conflicting completion that proves Tropibot already holds different state for the claim.
If Tropibot has marked this worker lost, the dashboard shows the exact fenced job and its last phase. The operator may choose Cleanup done — release & retry or Cleanup done — release only. The backend calls Tropibot's authenticated POST /v1/package-harness/jobs/{jobId}/ready endpoint with explicit cleanup confirmation, displays Tropibot's retry disposition, and wakes the paused polling task so no package restart is required. The emergency operator-token abandonment endpoint remains a separate Tropibot operation.
After manually completing a failed target cleanup, acknowledge that exact run through the internal supervision API:
curl --fail-with-body \
--request POST \
--header 'content-type: application/json' \
--data '{"runId":"gh-pr-177-f33e0a37375d-f0181f51d12f8734"}' \
http://HARNESS_HOST:8080/operator/recovery/continueThe endpoint accepts only cleanup-related manual holds. It preserves the claim token and stored result, clears the persisted hold, and wakes the paused worker without a container restart. It does not override coordinator completion conflicts.
Heartbeats run independently of package execution. A cancellation request is observed at safe phase boundaries and inside stabilization polling. It prevents a new mutating phase but never bypasses required cleanup. If Tropibot no longer recognizes the claim, the worker finishes the current operation, reconciles cleanup, and releases the obsolete local claim when the target is safe.
For real nodes set at least:
PACKAGE_MANAGER_MODE=mcp
DAPPMANAGER_MCP_URL=https://dedicated-test-dappnode.example/mcp
DAPPMANAGER_MCP_TOKEN=<token>
Dappmanager must expose all seven tools listed in src/package_manager/mod.rs, including mutation tools. Before using a new Dappmanager version, run:
TROPIBOT_URL=https://tropibot.example \
PACKAGE_HARNESS_WORKER_ID=worker-01 \
PACKAGE_HARNESS_WORKER_TOKEN=development-token \
DAPPMANAGER_MCP_URL=... \
DAPPMANAGER_MCP_TOKEN=... \
cargo run -- --mcp-smokeThis checks the MCP transport, tool inventory, and normalized package listing without running a destructive test.
cargo fmt --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --all-features
cargo build --releaseThe Makefile provides matching development commands. To build the container and package scaffold:
docker build -t package-harness.dnp.dappnode.eth:0.1.1 .
npx @dappnode/dappnodesdk@0.3.52 build --skip_save --skip_uploadThe runtime is non-root, keeps /data in a persistent volume, has no Docker socket or host networking, drops all Linux capabilities, and uses no-new-privileges.
More detail: docs/architecture.md and docs/result-schema.md.