diff --git a/.claude/skills/pr-description/SKILL.md b/.claude/skills/pr-description/SKILL.md
index bd7a072f40e..52a19f9dd0c 100644
--- a/.claude/skills/pr-description/SKILL.md
+++ b/.claude/skills/pr-description/SKILL.md
@@ -13,7 +13,8 @@ Generate a pull request title and description for the current branch using the p
1. Determine the base branch:
- Use the argument if provided
- Otherwise, auto-detect: `git remote set-head origin --auto >/dev/null 2>&1 && git symbolic-ref refs/remotes/origin/HEAD 2>/dev/null | sed 's|refs/remotes/||'`
- - Fall back to `v4.0-dev` if the command above fails
+ - If that fails, use the repository's default branch on GitHub: `gh repo view --json defaultBranchRef -q .defaultBranchRef.name`
+ - If both fail, ask the user rather than guessing a version branch
2. Gather context by running these git commands:
- `git log --oneline $(git merge-base HEAD )..HEAD` — all commits on this branch
@@ -25,19 +26,30 @@ Generate a pull request title and description for the current branch using the p
- What specific code changes were made
- Whether there are breaking changes
- What tests were added or modified
+ - What value the change adds, and for whom
+ - What could go wrong: consensus impact, behaviour users could notice, slow or flaky tests
4. Output a suggested PR title using conventional commits format:
- - Scopes: `sdk`, `drive`, `dpp`, `dapi`, `dashmate`, `wasm-dpp`, `wasm-sdk`, `platform`
- - Types: `feat`, `fix`, `refactor`, `chore`, `docs`, `test`, `build`
+ - Types and scopes: use only the `types:` and `scopes:` lists in `.github/workflows/pr.yml`. The PR title check rejects anything else, so read them from that file rather than from memory
+ - The scope is optional: leave it out (e.g. `chore: ...`) when the change spans several packages or no listed scope fits. Never invent a scope
+ - The subject must not start with an uppercase letter
- Add `!` after the type for breaking changes (e.g. `feat!:`)
- Format: **Suggested title:** `type(scope): description`
-5. Fill in this PR template (preserve all HTML comments exactly as shown):
+5. Fill in this PR template (preserve all HTML comments exactly as shown). The `## Basic explanation` section always comes first; the sections after it follow `.github/PULL_REQUEST_TEMPLATE.md`. If that file differs from the copy below, follow the file:
```markdown
+## Basic explanation
+
+**What this does:**
+
+**Value:**
+
+**Risks:**
+
## Issue being fixed or feature implemented
@@ -69,6 +81,7 @@ Generate a pull request title and description for the current branch using the p
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have added "!" to the title and described breaking changes in the corresponding section if my code contains any
- [ ] I have made corresponding changes to the documentation if needed
+- [ ] If I added or changed GroveDB structure, I described it in the area's `structure.rs`, regenerated `grovedb-structure.json`, and checked the structure viewer link posted on this pull request
**For repository code-owners and collaborators only**
- [ ] I have assigned this pull request to a milestone
@@ -80,6 +93,7 @@ Output the entire PR description (title + body) as a single raw Markdown code bl
## Guidelines
+- Always open with `## Basic explanation`: three short paragraphs (what it does, value, risks) that someone outside this code can follow. For a security fix, keep it neutral: describe what the fix does, not how the bug could be exploited
- Keep the description **concise** — avoid walls of text. Prefer short bullet points over paragraphs
- Be specific — reference file paths, struct/function names, and types
- For "How Has This Been Tested?", check `git diff` for new `*test*`, `*spec*` files. Briefly describe what tests cover (1 line per test file), not every individual test case
diff --git a/.editorconfig b/.editorconfig
index 2e9dc07ba04..35057547424 100644
--- a/.editorconfig
+++ b/.editorconfig
@@ -11,6 +11,10 @@ end_of_line = lf
[*.rs]
indent_size = 4
+# Swift and Kotlin follow their languages' 4-space convention.
+[*.{swift,kt,kts}]
+indent_size = 4
+
# Preserve the existing indentation of the Swift SDK Python scripts.
[packages/swift-sdk/scripts/*.py]
indent_size = 4
diff --git a/.github/NPM_RUNNER.md b/.github/NPM_RUNNER.md
new file mode 100644
index 00000000000..893eb82ecfb
--- /dev/null
+++ b/.github/NPM_RUNNER.md
@@ -0,0 +1,103 @@
+# NPM release runners
+
+NPM release compilation uses a fresh single-job runner with a unique
+`platform-release---npm` label in the `platform-release-builds`
+runner group. Kotlin releases use the same lifecycle with a `-kotlin` label. Publishing
+continues on GitHub-hosted Ubuntu with OIDC; the builder receives no publishing
+credentials. The `npm-release-build` action is shared by releases and image
+validation so both compile and pack with the same setup.
+
+## Image contract
+
+`.github/runner-requirements.json` pins the complete Linux image requirements and
+recipe commit. The job runs `ci-image-contract verify` before compiling. The
+runner must have `/opt/client-codegen` matching `packages/dapi-grpc/codegen.json`.
+Its protobuf 3.18.1 compiler is intentionally separate from Rust's protoc 32.0.
+TypeScript generation comes from the workspace's pinned `ts-protoc-gen` dependency.
+
+Image-owned native dependencies are verified, never installed using sudo. Rust,
+Node and the pinned WASM tools use writable runner/user locations. Cargo targets
+remain in job-local HOME; each release starts with fresh runner, HOME and workspace state. The
+runner needs no Docker CLI/socket or KVM device.
+
+## Provisioning and promotion
+
+Use the reviewed `dashpay/dash-selfhosted-image` recipe and a tested immutable
+image digest, not a moving tag. Deploy the host-side disposable release controller
+only after the NPM validation workflow succeeds on that image. Do not add generic
+release labels to persistent CI registrations. See the
+[controller installation and cleanup runbook](https://github.com/dashpay/dash-selfhosted-image/blob/main/docs/disposable-releases.md).
+Old release tags
+still contain their original workflows and do not automatically gain this fix.
+
+Requirements-changing PRs select a candidate label bound to the complete PR head
+and image digest. The image controller must support the `npm` job kind and
+`.github/workflows/npm-runner-validation.yml`. Manifests requesting native client
+generation require successful Rust, Kotlin and NPM candidate jobs for promotion;
+skipped fork jobs do not qualify. Existing same-repository/trusted-fork guards
+remain in effect.
+
+The trusted `runner-image-candidate.yml` bootstrap, controller and Rust/Kotlin
+candidate routing must be installed on each consuming branch before candidate
+promotion can work. Platform PRs #4702 and #4912 establish those pieces; reconcile
+their requirements/selector files with this NPM extension when landing them. In
+particular, update both the bootstrap's reusable-workflow SHA and its
+`control_revision` to a reviewed image-repository revision supporting
+`client_codegen` and `npm`. Merely changing `recipe_revision` is insufficient.
+The default `v4.2-dev` and `v4.3-dev` branches must each use an explicit compatible
+manifest; this change does not alter an existing release tag or deploy a runner.
+
+## Verification
+
+`npm-runner-validation.yml` runs the real release build and packing action, DAPI
+unit tests, and a byte-for-byte check that packed Node/web clients match the
+freshly generated files. It uploads tarballs but never publishes them.
+`test-client-codegen.yml` also builds the native compilers on hosted Linux/macOS,
+checks committed generated output, tests failure recovery and validates packing.
+
+Local setup and generator test commands are in `packages/dapi-grpc/README.md`.
+
+After installing the controller, use the `release.yml` dispatch with
+`tag=npm-test:v` on a protected development branch for a non-publishing
+NPM build. For Kotlin, dispatch `release-kotlin-sdk.yml` from the protected branch
+with an existing published `tag` and `dry_run=true`: compilation/artifact upload
+run, but release attachment and Maven publication are both skipped. Check that
+the image contract matches the selected source. Neither controller unit tests
+nor an image smoke test establishes that these end-to-end jobs pass.
+
+## Separate PR and release state
+
+The `platform-release-builds` organization runner group selects only
+`dashpay/platform` and contains only controller-created one-job registrations.
+Each build requests:
+
+```yaml
+runs-on:
+ group: platform-release-builds
+ labels: [self-hosted, Linux, X64, 'platform-release-${{ github.run_id }}-${{ github.run_attempt }}-npm']
+```
+
+There is no fallback to `npm-build`, `rust-ci` or `kotlin-ci`. Without the
+controller, builds stay queued. Runtime markers reject accidental routing to an
+ordinary runner; they are not cryptographic attestation. The host controller
+independently checks repository, event, workflow, run, attempt and commit before
+creating fresh JIT capacity. Only one job can consume each registration; the host
+destroys its container/processes, HOME, registration and workspace afterward.
+
+**Ordinary PR caching is unchanged.** PR validation keeps its own persistent
+Cargo/Gradle/Yarn caches. Releases reuse the prebaked image/toolchains but never
+mount PR state or restore shared executable dependency caches. Yarn caching is
+opted out only for the release runtime; Kotlin release build/publication disable
+Gradle cache restores. A cold release compile is the intentional tradeoff; do not
+reintroduce shared caches to speed it up without reviewing their writer trust.
+
+`release.yml` calls its local reusable workflow, so the workflow travels with the
+release source. Port it and the matching image requirements to 4.3; the host
+controller needs no branch-specific allowlist. Branch protection, trusted tags,
+fork approvals and hosted publishing authorization remain necessary. Optional
+selected-workflow group restrictions are defense in depth, not the mechanism
+that erases prior-job state. Labels alone are not authorization.
+
+This assumes a trusted host and pinned image. A fresh container does not repair
+host compromise or retroactively secure old release tags/artifacts. Merge/deploy
+the controller before relying on this workflow change for a release.
diff --git a/.github/SELF_HOSTED_RUNNER.md b/.github/SELF_HOSTED_RUNNER.md
new file mode 100644
index 00000000000..6978bff1889
--- /dev/null
+++ b/.github/SELF_HOSTED_RUNNER.md
@@ -0,0 +1,182 @@
+# Self-hosted runner contract
+
+## Reproducible Linux image
+
+Source, locked dependencies, deployment examples and publishing workflow:
+**[dashpay/dash-selfhosted-image](https://github.com/dashpay/dash-selfhosted-image)**.
+Use its `linux/amd64` contract-1 image for persistent Linux `kotlin-ci` / `rust-ci`
+runners, or its native `linux/arm64` Rust-only image on Apple Silicon Linux VMs.
+The shared Rust action requires `/opt/ci/contract-version` to be `1` on
+self-hosted Linux and fails early if an old/native runner picks up the job.
+
+Platform's desired versions and checksums now live in
+[.github/runner-requirements.json](runner-requirements.json). This includes an immutable image-recipe
+commit. Persistent Linux jobs verify **both** the installed lock and recipe
+revision; a contract-1 marker by itself is not sufficient. The earlier published
+bootstrap image must be replaced by a matching candidate before these changes
+can be merged.
+
+ARM64 Rust jobs select [runner-requirements.arm64.json](runner-requirements.arm64.json)
+using the **actual** job's `RUNNER_OS=Linux` / `RUNNER_ARCH=ARM64`. They verify its
+exact recipe and ARM64 lock, not the AMD64 lock and not just tool version strings.
+The shared Rust action defaults to the native compilation target. Kotlin/Android
+remains AMD64-only; ARM64 does not export or pretend to provide Android tooling.
+
+The image locks Ubuntu 24.04 by digest, apt to a signed archive snapshot, and
+downloaded toolchains to exact URLs and SHA-256 hashes. It includes:
+
+| Toolchain | Contract |
+| --- | --- |
+| Native build | build-essential, clang/LLVM, Snappy, CMake, GMP, OpenSSL, pkg-config |
+| Rust | rustup plus the repository baseline; exact repo-selected toolchains may be installed user-locally |
+| Cargo helpers | llvm-cov **0.9.1**, nextest **0.9.144**, machete **0.9.2**, ndk **4.1.2** |
+| Protobuf / Java | protoc **32.0**, JDK **17** |
+| Android | API **35**, build-tools **35.0.0**, NDK **28.1.13356709**, pinned emulator/system image |
+| Other job tools | Git, GitHub CLI, jq, Python 3, gpg, zip/unzip |
+
+The SDK is image-owned. The persistent Kotlin workflow uses the image's
+`ci-android-emulator` wrapper to create user-writable AVDs, boot with KVM, and stop
+the emulator after testing. It does **not** run setup-android, sdkmanager, or an
+emulator action that upgrades image packages at job runtime. Lockscreen/PIN and
+unlocked-device checks remain in `.github/scripts/kotlin-instrumented-tests.sh`.
+
+## Runtime privileges
+
+- Non-root uid/gid **1001:1001**, no sudo, no Docker CLI or host Docker socket.
+- Drop **all** capabilities; enable **no-new-privileges**. Keep default Docker
+ seccomp/AppArmor policies, with no privileged mode or host namespaces.
+- Dedicated registration/work volumes only. Kotlin adds **`/dev/kvm:rw`** and its
+ numeric host group; Rust-only runners do not need that device.
+- The operator configures `/dev/kvm` as `root:kvm`, mode **0660**. Jobs only check
+ access; they never modify host udev rules, permissions or system packages.
+- Preserve runner-group selected-repository access and the existing fork guards.
+ Persistent job data is not isolation between mutually untrusted repositories.
+
+Building/publishing the image uses Docker on an ephemeral GitHub-hosted builder;
+that privilege is not passed into the resulting persistent runner.
+
+## Publish, prove, then roll out
+
+For Platform requirements changes, use the PR-first lifecycle below. The
+standalone image publishing workflow remains useful for recipe development,
+but its default lock is not a separate source of Platform requirements.
+
+1. Use a successful [image publishing run](https://github.com/dashpay/dash-selfhosted-image/actions/workflows/image.yml).
+ Publication requires non-root compiler/confinement checks, `KVM_CREATE_VM`, and
+ a real API 35 emulator boot. Retrieve `image-reference.txt` from the run.
+2. Set `RUNNER_IMAGE=dashpay/dash-selfhosted-image@sha256:` in
+ the operator's deployment. Do not use a floating image or a locally inherited
+ `github-runner-runner:latest` parent. The image repo's Compose files enforce the
+ runtime boundary above; the KVM overlay is optional.
+3. Drain the old runner before migration. Register a new name in the **existing
+ group**, retaining its selected repositories, with only the required labels.
+ Use a short-lived registration-token file, not a PAT stored in Compose.
+4. Prove a real Rust job and Kotlin job on that exact runner/digest before retiring
+ the old instance. Keep the previous registration/image for rollback. Rebuilding
+ or pushing source does not replace any live runner automatically.
+
+Deploy and prove the contract-1 image **before merging the consuming workflows**.
+Record the selected digest and real-job evidence with the deployment; do not infer
+runtime health from YAML validation or the image tag alone. Rebuild/repin when
+dependencies change, including runner updates required by GitHub's update policy.
+
+## Requirements changes: candidate before merge, promotion after
+
+1. Change .github/runner-requirements.json in the Platform PR, including exact download URLs,
+ checksums and package metadata. A change to this file is the automatic build
+ flag; no separate label is required. Change the pinned recipe commit only
+ when recipe/image code changes. Ordinary user-local Rust toolchain updates
+ still follow rust-toolchain.toml.
+2. The trusted base-branch publisher builds and smoke-tests a candidate on a
+ disposable VM. A separate VM publishes it without executing PR image/code
+ with Docker Hub credentials.
+3. The normal Rust and Kotlin workflows wait for the candidate, then request
+ temporary runners labelled for **this PR head, exact digest and job kind**.
+ A host-side controller creates one-job non-root containers, with KVM only
+ for Kotlin. Real application jobs must pass; skipped fork jobs do not count.
+4. After merge, the publisher verifies the merged/current requirements and both
+ real candidate jobs, then promotes **the same tested digest**, without a
+ rebuild. Each base branch gets a platform- channel; main advances only
+ for Platform's actual GitHub default branch.
+5. Promotion does not restart production runners. The operator drains and
+ switches ordinary runner capacity to the reviewed digest using the rollout
+ procedure above. Requirements checks fail explicitly until capacity matches.
+
+The trusted caller must land separately before a PR can use this flow.
+See the image repository's
+[bootstrap, GitHub App and candidate-controller setup](https://github.com/dashpay/dash-selfhosted-image/blob/feat/platform-pr-images/docs/platform-pr-images.md).
+No App key, Docker socket, registry credential or host workspace enters a job.
+The existing trusted-fork restrictions are unchanged; the controller's
+exact-head approvals do not override workflow-side guards.
+
+Run the routing checks with:
+
+~~~sh
+python3 -m unittest discover -s .github/scripts/tests -v
+~~~
+
+## Apple Silicon rollout and ARM64 requirements changes
+
+The Rust workspace and wallet jobs select `[self-hosted, Linux, rust-ci]`. This
+includes Linux containers on Macs; it does **not** remove Mac hardware from CI.
+Keep native macOS registrations for Swift, Xcode and simulator jobs. Never reuse
+their registration, HOME or workspaces inside a container. Rust and Swift may run
+concurrently, so reserve host resources rather than assigning both the whole Mac.
+Use Linux-owned named volumes for build/cache data, not macOS bind mounts.
+
+Initial ARM64 recipe: `772673c94f2c0b39e7e796198a6ea407c087dd72`, published by
+[image run 36419475060](https://github.com/dashpay/dash-selfhosted-image/actions/runs/36419475060).
+Its tested immutable reference is
+`dashpay/dash-selfhosted-image@sha256:2ef7934f6877b4b78bdc3d4b81c07ee260d1648c0338c86145cec02760390a24`.
+The existing AMD64 requirements and deployed images are unchanged.
+
+Before merging/routing ordinary CI, provision each Mac's ARM64 VM and validate
+the digest with the image's smoke test. Register it separately in the existing
+selected-repository group with `rust-ci-validation`, prove a real Platform
+workspace job on that exact runner, and verify unattended restart. Only validated
+instances get `rust-ci`; keep the validation label for future image qualification.
+Do not count image-build CI, local unit tests or skipped fork jobs as this proof.
+
+The existing automatic PR-candidate publisher/controller is **AMD64-only**.
+ARM64 requirements currently use explicit operator deployment, not that publisher:
+
+1. Build/publish the ARM64 recipe and pin its exact lock and recipe here.
+2. Deploy the tested digest to an idle validation runner, preserving rollback.
+3. `ARM64 runner image validation` runs the **full** Rust workspace on ARM64 when
+ this manifest changes. Exact lock/recipe mismatch fails before compilation.
+ Its selector compares the PR-head manifest with the checked-out merge tree;
+ an AMD64 candidate status cannot satisfy ARM64 validation.
+ That PR's ordinary Rust job stays on AMD64 so it cannot land on ARM64
+ production capacity still running the old image; unrelated PRs use both
+ architectures as usual.
+4. Require successful real ARM64 validation before merging the requirements and
+ rolling out other Mac-backed capacity. If AMD64 requirements also change,
+ their separate Rust/Kotlin candidate gates still apply.
+
+Keep shared Rust/helper versions aligned across both manifests. Automatic ARM64
+candidate creation/promotion is not implemented; never infer ARM64 validation or
+deployment from an AMD64 publisher result.
+
+## Hosted Linux and native macOS remain distinct
+
+The shared Rust action branches on `runner.environment`: persistent Linux verifies
+the image's native libraries and protoc, while GitHub-hosted consumers retain apt
+provisioning and the user-local protoc cache.
+Both select clang through `CC`/`CXX`, without mutating system alternatives.
+
+The Linux image does not provision macOS. Native macOS runners retain their
+existing Swift/Homebrew dependencies and registrations; generic Rust jobs now
+use the Linux image pool. Do not silently install tools or swallow failures in
+persistent jobs.
+
+Kotlin release builds use persistent `kotlin-ci` capacity and retain their separate
+release hardening: forcibly reinstall cargo-ndk 4.1.2 and verify a fresh protoc
+download by checksum. A version-only check of a binary left by a previous job is
+not a substitute for these release checks. Release attachment and Maven
+publication stay on separate GitHub-hosted jobs, keeping publishing credentials
+off the persistent build runner.
+
+Docker publication and Kotlin nightly jobs remain on ephemeral GitHub-hosted
+runners; the nightly job explicitly installs cargo-ndk 4.1.2. Any future self-hosted
+job that genuinely needs Docker must use separately isolated capacity; the
+persistent Rust/Kotlin runner must not regain the host Docker socket.
diff --git a/.github/actionlint.yaml b/.github/actionlint.yaml
index 14ff10fba49..6527a051ce5 100644
--- a/.github/actionlint.yaml
+++ b/.github/actionlint.yaml
@@ -9,3 +9,5 @@ self-hosted-runner:
labels:
- rust-ci
- kotlin-ci
+ - npm-build
+ - npm-pr
diff --git a/.github/actions/librocksdb/action.yaml b/.github/actions/librocksdb/action.yaml
index 8bb32f9fae1..3edae56999e 100644
--- a/.github/actions/librocksdb/action.yaml
+++ b/.github/actions/librocksdb/action.yaml
@@ -21,7 +21,7 @@ runs:
uses: actions/cache@v5
id: librocksdb-cache
with:
- key: librocksdb/${{ inputs.version }}/${{ runner.os }}/${{ runner.arch }}
+ key: librocksdb/pic-v1/${{ inputs.version }}/${{ runner.os }}/${{ runner.arch }}
path: /opt/rocksdb
- if: ${{ steps.librocksdb-cache.outputs.cache-hit != 'true' || inputs.force == 'true' }}
@@ -29,6 +29,10 @@ runs:
name: Build librocksdb
env:
PORTABLE: 1
+ # This static archive is also linked into Rust cdylibs. In particular,
+ # RocksDB's thread-local objects must not use non-PIC relocations.
+ EXTRA_CFLAGS: -fPIC
+ EXTRA_CXXFLAGS: -fPIC
run: |
set -ex
WORKDIR=/tmp/rocksdb-build
diff --git a/.github/actions/nodejs/action.yaml b/.github/actions/nodejs/action.yaml
index 8c8066edf8e..96a834eb632 100644
--- a/.github/actions/nodejs/action.yaml
+++ b/.github/actions/nodejs/action.yaml
@@ -2,6 +2,10 @@
name: "Setup Node.JS"
description: "Setup Node.JS binaries, dependencies and cache"
inputs:
+ cache:
+ description: "Restore/save executable dependency build caches"
+ required: false
+ default: "true"
node-version:
description: "Node.js version to use"
required: false
@@ -29,6 +33,7 @@ runs:
run: npm config set audit false
- name: Cache NPM build artifacts
+ if: inputs.cache == 'true'
uses: actions/cache@v5
with:
# Cache the unplugged packages (unpacked native builds), yarn's
diff --git a/.github/actions/npm-release-build/action.yaml b/.github/actions/npm-release-build/action.yaml
new file mode 100644
index 00000000000..6250df11fa2
--- /dev/null
+++ b/.github/actions/npm-release-build/action.yaml
@@ -0,0 +1,132 @@
+name: Build release NPM packages
+description: Compile and pack on the provisioned unprivileged Linux image
+inputs:
+ cache-name:
+ description: Cargo target directory name (job-local on disposable release runners)
+ default: release-npm-target
+runs:
+ using: composite
+ steps:
+ - name: Verify provisioned runner dependencies
+ shell: bash
+ run: |
+ set -euo pipefail
+
+ ci-image-contract verify .github/runner-requirements.json
+ python3 packages/dapi-grpc/scripts/setup-codegen.py
+ for pkg in build-essential cmake curl jq libgmp-dev libssl-dev pkg-config python3 unzip zip; do
+ dpkg-query -W -f='${Status}' "$pkg" | grep -Fx 'install ok installed'
+ done
+ test -x "$HOME/.cargo/bin/rustup"
+ echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
+
+ - name: Setup Rust
+ uses: ./.github/actions/rust
+ with:
+ target: wasm32-unknown-unknown
+ # Never restore a shared Rust build cache into a release.
+ cache: 'false'
+ system-dependencies: verify
+
+ # The Rust action above reuses a protoc left in ~/.local by earlier jobs.
+ # Override it with a fresh, checksum-verified copy for the release build;
+ # prost-build reads PROTOC first.
+ - name: Install protoc v32.0
+ env:
+ PROTOC_SHA256: 7ca037bfe5e5cabd4255ccd21dd265f79eb82d3c010117994f5dc81d2140ee88
+ shell: bash
+ run: |
+ set -euo pipefail
+ PROTOC_DIR="$RUNNER_TEMP/protoc-32.0"
+ curl -fsSL -o "$RUNNER_TEMP/protoc.zip" \
+ https://github.com/protocolbuffers/protobuf/releases/download/v32.0/protoc-32.0-linux-x86_64.zip
+ echo "$PROTOC_SHA256 $RUNNER_TEMP/protoc.zip" | sha256sum -c -
+ rm -rf "$PROTOC_DIR"
+ unzip -q "$RUNNER_TEMP/protoc.zip" -d "$PROTOC_DIR"
+ echo "PROTOC=$PROTOC_DIR/bin/protoc" >> "$GITHUB_ENV"
+ echo "$PROTOC_DIR/bin" >> "$GITHUB_PATH"
+ "$PROTOC_DIR/bin/protoc" --version
+
+ - name: Prepare release Cargo target cache
+ uses: ./.github/actions/release-cargo-target-cache
+ with:
+ name: ${{ inputs.cache-name }}
+
+ - name: Setup Node.JS
+ uses: ./.github/actions/nodejs
+ with:
+ # PR image validation may use its own caches; release jobs must not
+ # import executable state from the ordinary CI cache namespace.
+ cache: ${{ env.DASH_RELEASE_RUNNER != '1' }}
+
+ - name: Install Cargo binstall
+ uses: cargo-bins/cargo-binstall@v1.3.1
+
+ # Always reinstalled, never trusted from an earlier job: a binary left on
+ # this persistent runner can print the pinned version and still be
+ # something else.
+ - name: Install wasm-bindgen-cli
+ shell: bash
+ run: |
+ set -euo pipefail
+ cargo binstall wasm-bindgen-cli@0.2.108 --no-confirm --force
+ wasm-bindgen --version | grep -Fxq 'wasm-bindgen 0.2.108'
+
+ - name: Install wasm-pack
+ shell: bash
+ run: |
+ set -euo pipefail
+ cargo binstall wasm-pack@0.15.0 --no-confirm --force
+ wasm-pack --version | grep -Fxq 'wasm-pack 0.15.0'
+
+ # Fresh, checksum-verified copy in this job's temp directory rather than
+ # one left in ~/.local by an earlier job.
+ - name: Install Binaryen
+ env:
+ BINARYEN_SHA256: c90e0e295e8f8484ba5b47da92f26e5d1d18db6cd2fcc0c5cc265a5a73609f17
+ shell: bash
+ run: |
+ set -euo pipefail
+ ARCHIVE="$RUNNER_TEMP/binaryen-version_121-x86_64-linux.tar.gz"
+ curl -fsSL -o "$ARCHIVE" \
+ https://github.com/WebAssembly/binaryen/releases/download/version_121/binaryen-version_121-x86_64-linux.tar.gz
+ echo "$BINARYEN_SHA256 $ARCHIVE" | sha256sum -c -
+ rm -rf "$RUNNER_TEMP/binaryen-version_121"
+ tar -xzf "$ARCHIVE" -C "$RUNNER_TEMP"
+ echo "$RUNNER_TEMP/binaryen-version_121/bin" >> "$GITHUB_PATH"
+
+ # Binaryen otherwise sizes its thread pool from the host CPU count, which
+ # inside a Docker CPU quota oversubscribes and spins on futexes. Four
+ # threads measured ~3x faster with byte-identical output. Cargo's job
+ # budget is unaffected; a runner-provided BINARYEN_CORES still wins.
+ - name: Build packages
+ shell: bash
+ run: |
+ export BINARYEN_CORES="${BINARYEN_CORES:-4}"
+ echo "::notice::Binaryen threads: $BINARYEN_CORES"
+ time yarn build
+ env:
+ CARGO_BUILD_PROFILE: release
+
+ - name: Ignore only already cached artifacts
+ shell: bash
+ run: |
+ find . -name '.gitignore' -exec rm -f {} +
+ {
+ echo ".yarn"
+ echo "target"
+ echo "node_modules"
+ echo ".nyc_output"
+ echo ".idea"
+ echo ".ultra.cache.json"
+ echo "db/*"
+ echo "npm-packages"
+ } >> .gitignore
+
+ - name: Pack public workspaces
+ shell: bash
+ run: |
+ mkdir -p npm-packages
+ yarn workspaces foreach --all --no-private --parallel pack \
+ --out "$GITHUB_WORKSPACE/npm-packages/%s-%v.tgz"
+ test -n "$(find npm-packages -maxdepth 1 -type f -name '*.tgz' -print -quit)"
diff --git a/.github/actions/release-cargo-target-cache/action.yaml b/.github/actions/release-cargo-target-cache/action.yaml
new file mode 100644
index 00000000000..618117b86ac
--- /dev/null
+++ b/.github/actions/release-cargo-target-cache/action.yaml
@@ -0,0 +1,51 @@
+---
+name: "Release Cargo target cache"
+description: >-
+ Point CARGO_TARGET_DIR at a per-workflow directory outside the checkout.
+ On disposable release runners this is job-local HOME state, destroyed with
+ the container, never a cache shared with PRs. Reset it when it outgrows
+ max-gib or available space drops below min-free-gib.
+inputs:
+ name:
+ description: Cache directory name, unique per release workflow
+ required: true
+ max-gib:
+ description: Reset the cache when it grows past this many GiB
+ required: false
+ default: "60"
+ min-free-gib:
+ description: Reset the cache when the volume has less than this many GiB free
+ required: false
+ default: "40"
+runs:
+ using: composite
+ steps:
+ - name: Prepare release Cargo target cache
+ # Quoted: a self-hosted runner's temp directory can contain spaces.
+ shell: bash --noprofile --norc -e -o pipefail "{0}"
+ env:
+ CACHE_NAME: ${{ inputs.name }}
+ MAX_GIB: ${{ inputs.max-gib }}
+ MIN_FREE_GIB: ${{ inputs.min-free-gib }}
+ run: |
+ set -euo pipefail
+ case "$CACHE_NAME" in
+ ''|*/*|.*)
+ echo "::error::Invalid release target cache name '$CACHE_NAME'."
+ exit 1
+ ;;
+ esac
+ RELEASE_TARGET_DIR="$HOME/.cache/dash-platform/$CACHE_NAME"
+ MAX_CACHE_KIB=$((MAX_GIB * 1024 * 1024))
+ MIN_FREE_KIB=$((MIN_FREE_GIB * 1024 * 1024))
+ mkdir -p "$RELEASE_TARGET_DIR"
+ USED_KIB=$(du -sk "$RELEASE_TARGET_DIR" | cut -f1)
+ FREE_KIB=$(df -Pk "$RELEASE_TARGET_DIR" | awk 'NR == 2 { print $4 }')
+ if [ "$USED_KIB" -gt "$MAX_CACHE_KIB" ] || [ "$FREE_KIB" -lt "$MIN_FREE_KIB" ]; then
+ echo "::notice::Resetting the release target cache (${USED_KIB} KiB used, ${FREE_KIB} KiB free on the volume)."
+ rm -rf "${RELEASE_TARGET_DIR:?}"
+ mkdir -p "$RELEASE_TARGET_DIR"
+ fi
+ du -sh "$RELEASE_TARGET_DIR"
+ df -h "$RELEASE_TARGET_DIR"
+ echo "CARGO_TARGET_DIR=$RELEASE_TARGET_DIR" >> "$GITHUB_ENV"
diff --git a/.github/actions/rust/action.yaml b/.github/actions/rust/action.yaml
index 808f86cbe32..7dfed6fc848 100644
--- a/.github/actions/rust/action.yaml
+++ b/.github/actions/rust/action.yaml
@@ -6,12 +6,15 @@ inputs:
description: Rust toolchain to use, stable / nightly / beta, or exact version; uses rust-toolchain.toml if not specified
default: ""
target:
- description: Target Rust platform
+ description: Additional Rust target to install; defaults to the runner's native target
required: false
- default: x86_64-unknown-linux-gnu
+ default: ""
components:
description: List of additional Rust toolchain components to install
required: false
+ system-dependencies:
+ description: Install native packages on hosted runners, or verify a provisioned image
+ default: install
cache:
description: Enable Rust cache
required: false
@@ -21,6 +24,31 @@ inputs:
runs:
using: composite
steps:
+ - name: Validate native dependency mode
+ shell: bash
+ env:
+ DEPENDENCY_MODE: ${{ inputs.system-dependencies }}
+ run: |
+ case "$DEPENDENCY_MODE" in
+ install|verify) ;;
+ *) echo "::error::Unknown native dependency mode"; exit 1 ;;
+ esac
+
+ - name: Read shared runner tool requirements
+ shell: bash
+ run: python3 .github/scripts/runner-image.py env
+
+ - name: Verify persistent Linux runner image contract
+ if: runner.os == 'Linux' && runner.environment == 'self-hosted'
+ shell: bash
+ run: |
+ if [ "$(cat /opt/ci/contract-version 2>/dev/null)" != 1 ]; then
+ echo '::error::This runner needs the versioned dashpay/dash-selfhosted-image (contract 1); see .github/SELF_HOSTED_RUNNER.md.'
+ exit 1
+ fi
+ python3 .github/scripts/runner-image.py verify
+ command -v rustup
+
- name: Resolve HOME path for caching
id: resolved_home
shell: bash
@@ -47,7 +75,7 @@ runs:
components: ${{ inputs.components }}
- name: Get protoc arch
- if: runner.os == 'Linux'
+ if: runner.os == 'Linux' && runner.environment == 'github-hosted'
shell: bash
id: protoc_arch
run: |
@@ -66,40 +94,48 @@ runs:
;;
esac
- - name: Restore cached protoc (v32.0)
- if: runner.os == 'Linux'
+ - name: Restore cached repository-pinned protoc
+ if: runner.os == 'Linux' && runner.environment == 'github-hosted'
id: cache-protoc
uses: actions/cache@v5
with:
path: |
- ${{ steps.resolved_home.outputs.home }}/.local/protoc-32.0/bin
- ${{ steps.resolved_home.outputs.home }}/.local/protoc-32.0/include
- key: protoc/32.0/${{ runner.os }}/${{ steps.protoc_arch.outputs.arch }}
+ ${{ steps.resolved_home.outputs.home }}/.local/protoc-${{ env.CI_PROTOC_VERSION }}/bin
+ ${{ steps.resolved_home.outputs.home }}/.local/protoc-${{ env.CI_PROTOC_VERSION }}/include
+ key: protoc/${{ env.CI_PROTOC_VERSION }}/${{ runner.os }}/${{ steps.protoc_arch.outputs.arch }}
- - name: Install protoc (cached v32.0)
- if: runner.os == 'Linux'
+ - name: Install repository-pinned protoc
+ if: runner.os == 'Linux' && runner.environment == 'github-hosted'
id: deps-protoc
shell: bash
run: |
set -euxo pipefail
- PROTOC_DIR="${HOME}/.local/protoc-32.0"
+ PROTOC_DIR="${HOME}/.local/protoc-${{ env.CI_PROTOC_VERSION }}"
if [ ! -x "${PROTOC_DIR}/bin/protoc" ]; then
mkdir -p "${PROTOC_DIR}"
curl -fsSL -o /tmp/protoc.zip \
- "https://github.com/protocolbuffers/protobuf/releases/download/v32.0/protoc-32.0-linux-${{ steps.protoc_arch.outputs.arch }}.zip"
+ "https://github.com/protocolbuffers/protobuf/releases/download/v${CI_PROTOC_VERSION}/protoc-${{ env.CI_PROTOC_VERSION }}-linux-${{ steps.protoc_arch.outputs.arch }}.zip"
unzip -o /tmp/protoc.zip -d "${PROTOC_DIR}"
fi
echo "${PROTOC_DIR}/bin" >> "$GITHUB_PATH"
echo "PROTOC=${PROTOC_DIR}/bin/protoc" >> "$GITHUB_ENV"
- - name: Save cached protoc (v32.0)
- if: runner.os == 'Linux' && steps.cache-protoc.outputs.cache-hit != 'true'
+ - name: Save cached repository-pinned protoc
+ if: runner.os == 'Linux' && runner.environment == 'github-hosted' && steps.cache-protoc.outputs.cache-hit != 'true'
uses: actions/cache/save@v5
with:
path: |
- ${{ steps.resolved_home.outputs.home }}/.local/protoc-32.0/bin
- ${{ steps.resolved_home.outputs.home }}/.local/protoc-32.0/include
- key: protoc/32.0/${{ runner.os }}/${{ steps.protoc_arch.outputs.arch }}
+ ${{ steps.resolved_home.outputs.home }}/.local/protoc-${{ env.CI_PROTOC_VERSION }}/bin
+ ${{ steps.resolved_home.outputs.home }}/.local/protoc-${{ env.CI_PROTOC_VERSION }}/include
+ key: protoc/${{ env.CI_PROTOC_VERSION }}/${{ runner.os }}/${{ steps.protoc_arch.outputs.arch }}
+
+ - name: Verify prebaked repository-pinned protoc
+ if: runner.os == 'Linux' && runner.environment == 'self-hosted'
+ shell: bash
+ run: |
+ set -euo pipefail
+ protoc --version | grep -Fx "libprotoc $CI_PROTOC_VERSION"
+ echo "PROTOC=$(command -v protoc)" >> "$GITHUB_ENV"
- name: Set HOME variable to github context
shell: bash
@@ -113,17 +149,42 @@ runs:
${{ steps.resolved_home.outputs.home }}/.cargo/registry/index
${{ steps.resolved_home.outputs.home }}/.cargo/registry/cache
${{ steps.resolved_home.outputs.home }}/.cargo/git
- key: ${{ runner.os }}/cargo/registry/${{ hashFiles('**/Cargo.lock') }}
+ key: ${{ runner.os }}/${{ runner.arch }}/cargo/registry/${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
- ${{ runner.os }}/cargo/registry/${{ hashFiles('**/Cargo.lock') }}
- ${{ runner.os }}/cargo/registry/
+ ${{ runner.os }}/${{ runner.arch }}/cargo/registry/${{ hashFiles('**/Cargo.lock') }}
+ ${{ runner.os }}/${{ runner.arch }}/cargo/registry/
- - name: Install clang
+ # This composite is also used by hosted release, nightly and book jobs.
+ # Keep their bootstrap path; only persistent runners require a prebaked image.
+ - name: Install native dependencies on ephemeral GitHub-hosted Linux
+ if: runner.os == 'Linux' && runner.environment == 'github-hosted' && inputs.system-dependencies == 'install'
+ shell: bash
+ run: |
+ sudo apt-get update
+ sudo apt-get install -y --no-install-recommends clang llvm libsnappy-dev
+
+ # Linux self-hosted runners provide the compiler and native libraries in
+ # the pinned image. Do not install packages or mutate system alternatives
+ # from a job: the runner is intentionally non-root.
+ - name: Verify clang and native dependencies
id: deps-clang
shell: bash
if: runner.os == 'Linux'
run: |
- sudo apt update
- # snappy is required by rust rocksdb
- sudo apt install -qq --yes clang llvm libsnappy-dev
- sudo update-alternatives --set cc /usr/bin/clang
+ set -euo pipefail
+ missing=()
+ for tool in clang llvm-config; do
+ command -v "$tool" >/dev/null 2>&1 || missing+=("$tool")
+ done
+ for pkg in clang llvm libsnappy-dev; do
+ dpkg -s "$pkg" >/dev/null 2>&1 || missing+=("$pkg")
+ done
+ if [ ${#missing[@]} -gt 0 ]; then
+ echo "::error::Linux runner is missing: ${missing[*]}"
+ echo "::error::Persistent runners must provision these dependencies in the pinned image; hosted runners use the install step above."
+ exit 1
+ fi
+ # Use clang for native C/C++ build scripts without changing the
+ # system-wide cc alternative.
+ echo "CC=/usr/bin/clang" >> "$GITHUB_ENV"
+ echo "CXX=/usr/bin/clang++" >> "$GITHUB_ENV"
diff --git a/.github/runner-requirements.arm64.json b/.github/runner-requirements.arm64.json
new file mode 100644
index 00000000000..3c787c338e3
--- /dev/null
+++ b/.github/runner-requirements.arm64.json
@@ -0,0 +1,142 @@
+{
+ "schema": 1,
+ "recipe_revision": "772673c94f2c0b39e7e796198a6ea407c087dd72",
+ "requirements": {
+ "schema": 2,
+ "contract_version": "1",
+ "platform": "linux/arm64",
+ "ubuntu_image": "ubuntu:24.04@sha256:11dc1ccb427f0464a2369e645454c272bb0baece7357c892ba69d313b3a332cf",
+ "apt_snapshot": "20260920T000000Z",
+ "rust_version": "1.98.1",
+ "rust_manifest_sha256": "a7c8774a5fd8441c997d94c029776cbc5eb111e9d72ab5d256fa69866644347e",
+ "artifacts": [
+ {
+ "name": "runner",
+ "url": "https://github.com/actions/runner/releases/download/v2.337.0/actions-runner-linux-arm64-2.337.0.tar.gz",
+ "sha256": "9b1dc70626422526e3c94767cf024896beb15da5342a3f4819bf2feac13e0393",
+ "format": "tar",
+ "destination": "/opt/actions-runner"
+ },
+ {
+ "name": "cargo-llvm-cov",
+ "url": "https://github.com/taiki-e/cargo-llvm-cov/releases/download/v0.9.1/cargo-llvm-cov-aarch64-unknown-linux-gnu.tar.gz",
+ "sha256": "abf5f13c1520f8756d2192bfaaeadb0208f5b7eaa8cadc15bf847befa42b7360",
+ "format": "tar",
+ "destination": "/opt/ci/bin/cargo-llvm-cov",
+ "binary": "cargo-llvm-cov"
+ },
+ {
+ "name": "cargo-nextest",
+ "url": "https://github.com/nextest-rs/nextest/releases/download/cargo-nextest-0.9.144/cargo-nextest-0.9.144-aarch64-unknown-linux-gnu.tar.gz",
+ "sha256": "7fecfd431b810c05c589d800524286b8f80ce2fe9fb1fef05caf16d095cea407",
+ "format": "tar",
+ "destination": "/opt/ci/bin/cargo-nextest",
+ "binary": "cargo-nextest"
+ },
+ {
+ "name": "cargo-machete",
+ "url": "https://github.com/bnjbvr/cargo-machete/releases/download/v0.9.2/cargo-machete-v0.9.2-aarch64-unknown-linux-gnu.tar.gz",
+ "sha256": "6f96c3e6026a5bdd241b6ae600c6fb86c9197c6e189a894f91371baa01fd10f5",
+ "format": "tar",
+ "destination": "/opt/ci/bin/cargo-machete",
+ "binary": "cargo-machete"
+ },
+ {
+ "name": "protoc",
+ "url": "https://github.com/protocolbuffers/protobuf/releases/download/v32.0/protoc-32.0-linux-aarch_64.zip",
+ "sha256": "56af3fc2e43a0230802e6fadb621d890ba506c5c17a1ae1070f685fe79ba12d0",
+ "format": "zip",
+ "destination": "/opt/protoc"
+ },
+ {
+ "name": "rustup-init",
+ "url": "https://static.rust-lang.org/rustup/archive/1.28.2/aarch64-unknown-linux-gnu/rustup-init",
+ "sha256": "e3853c5a252fca15252d07cb23a1bdd9377a8c6f3efa01531109281ae47f841c",
+ "format": "file",
+ "destination": "/opt/ci/rustup-init",
+ "build_only": true
+ }
+ ],
+ "bootstrap_ca": {
+ "url": "https://snapshot.ubuntu.com/ubuntu/20260920T000000Z/pool/main/c/ca-certificates/ca-certificates_20240203_all.deb",
+ "sha256": "641de77d8f142cfd62a1a6f964ba67b20754d3337c480efb529d086075a06c9a",
+ "version": "20240203"
+ },
+ "apt_packages": [
+ "ca-certificates",
+ "curl",
+ "git",
+ "gh",
+ "jq",
+ "python3",
+ "unzip",
+ "zip",
+ "xz-utils",
+ "bzip2",
+ "gnupg",
+ "build-essential",
+ "clang",
+ "llvm",
+ "libsnappy-dev",
+ "cmake",
+ "libgmp-dev",
+ "libssl-dev",
+ "pkg-config",
+ "openjdk-17-jdk-headless",
+ "libicu74",
+ "libkrb5-3",
+ "zlib1g",
+ "libgcc-s1",
+ "libstdc++6",
+ "libcurl4t64",
+ "liblttng-ust1t64",
+ "libunwind8",
+ "libpulse0",
+ "libx11-xcb1",
+ "libnss3",
+ "libxcomposite1",
+ "libxcursor1",
+ "libxi6",
+ "libxrandr2",
+ "libxtst6",
+ "libasound2t64",
+ "libgl1",
+ "libegl1",
+ "libdbus-1-3",
+ "libxdamage1",
+ "libxfixes3"
+ ],
+ "java_major": 17,
+ "versions": {
+ "runner": "2.337.0",
+ "llvm_cov": "0.9.1",
+ "nextest": "0.9.144",
+ "machete": "0.9.2",
+ "protoc": "32.0",
+ "rustup": "1.28.2"
+ },
+ "client_codegen": {
+ "schema": 1,
+ "versions": {
+ "protobuf": "3.18.1",
+ "grpc": "1.46.3",
+ "grpc_java": "1.42.1"
+ },
+ "sources": [
+ {
+ "url": "https://github.com/protocolbuffers/protobuf/releases/download/v3.18.1/protobuf-cpp-3.18.1.tar.gz",
+ "sha256": "6ee35eda3f79e49608d2ace8d866313fdec539d8bb14c6c54e8d2a16fa4e6780"
+ },
+ {
+ "url": "https://github.com/grpc/grpc/archive/refs/tags/v1.46.3.tar.gz",
+ "sha256": "d6cbf22cb5007af71b61c6be316a79397469c58c82a942552a62e708bce60964"
+ },
+ {
+ "url": "https://github.com/grpc/grpc-java/archive/refs/tags/v1.42.1.tar.gz",
+ "sha256": "33775a1ad05974bbba6ff97801cd9b326485b21fab91b08a4e1bc53e500e6326"
+ }
+ ]
+ },
+ "profile": "rust"
+ }
+}
diff --git a/.github/runner-requirements.json b/.github/runner-requirements.json
new file mode 100644
index 00000000000..2987919fc83
--- /dev/null
+++ b/.github/runner-requirements.json
@@ -0,0 +1,228 @@
+{
+ "schema": 1,
+ "recipe_revision": "e49e8bc9977f5f961a76ba1d1f7673c72173679f",
+ "requirements": {
+ "schema": 2,
+ "contract_version": "1",
+ "platform": "linux/amd64",
+ "ubuntu_image": "ubuntu:24.04@sha256:496754492fb28b4d3049432f2ca787449331e23fb14f0dd3fffea86bf5a93eb4",
+ "apt_snapshot": "20260920T000000Z",
+ "rust_version": "1.98.1",
+ "rust_manifest_sha256": "a7c8774a5fd8441c997d94c029776cbc5eb111e9d72ab5d256fa69866644347e",
+ "artifacts": [
+ {
+ "name": "runner",
+ "url": "https://github.com/actions/runner/releases/download/v2.337.0/actions-runner-linux-x64-2.337.0.tar.gz",
+ "sha256": "70920811a4f8ad4328818682bca5c6469c1c942fab52448868071d0063816613",
+ "format": "tar",
+ "destination": "/opt/actions-runner"
+ },
+ {
+ "name": "cargo-llvm-cov",
+ "url": "https://github.com/taiki-e/cargo-llvm-cov/releases/download/v0.9.1/cargo-llvm-cov-x86_64-unknown-linux-gnu.tar.gz",
+ "sha256": "b3f68e625481fed9b16444174f3fa5ebcdbde4a1878803a35eabe2dcefcdc41a",
+ "format": "tar",
+ "destination": "/opt/ci/bin/cargo-llvm-cov",
+ "binary": "cargo-llvm-cov"
+ },
+ {
+ "name": "cargo-nextest",
+ "url": "https://github.com/nextest-rs/nextest/releases/download/cargo-nextest-0.9.144/cargo-nextest-0.9.144-x86_64-unknown-linux-gnu.tar.gz",
+ "sha256": "8a4f726272b0a1c499bd87ca3978bfbb1a8c20bb08ccf075b9996e2081bd1e1e",
+ "format": "tar",
+ "destination": "/opt/ci/bin/cargo-nextest",
+ "binary": "cargo-nextest"
+ },
+ {
+ "name": "cargo-machete",
+ "url": "https://github.com/bnjbvr/cargo-machete/releases/download/v0.9.2/cargo-machete-v0.9.2-x86_64-unknown-linux-musl.tar.gz",
+ "sha256": "48200087f54c55aabcd4db4af1e25742b49846c02a1b1bfa134711945b35b2e9",
+ "format": "tar",
+ "destination": "/opt/ci/bin/cargo-machete",
+ "binary": "cargo-machete"
+ },
+ {
+ "name": "cargo-ndk",
+ "url": "https://github.com/bbqsrc/cargo-ndk/releases/download/v4.1.2/cargo-ndk-x86_64-unknown-linux-gnu-v4.1.2.tgz",
+ "sha256": "9451622c4567e8abb2c8005001855e32901c0b716ccdd3a173a4375fd03426e1",
+ "format": "tar",
+ "destination": "/opt/ci/bin/cargo-ndk",
+ "binary": "cargo-ndk"
+ },
+ {
+ "name": "protoc",
+ "url": "https://github.com/protocolbuffers/protobuf/releases/download/v32.0/protoc-32.0-linux-x86_64.zip",
+ "sha256": "7ca037bfe5e5cabd4255ccd21dd265f79eb82d3c010117994f5dc81d2140ee88",
+ "format": "zip",
+ "destination": "/opt/protoc"
+ },
+ {
+ "name": "rustup-init",
+ "url": "https://static.rust-lang.org/rustup/archive/1.28.2/x86_64-unknown-linux-gnu/rustup-init",
+ "sha256": "20a06e644b0d9bd2fbdbfd52d42540bdde820ea7df86e92e533c073da0cdd43c",
+ "format": "file",
+ "destination": "/opt/ci/rustup-init",
+ "build_only": true
+ },
+ {
+ "name": "platforms;android-35",
+ "url": "https://dl.google.com/android/repository/platform-35_r02.zip",
+ "upstream_sha1": "0bb560a90a7a2cbd0dd8348224d518b638fe7949",
+ "format": "zip",
+ "destination": "/opt/android-sdk/platforms/android-35",
+ "archive_root": "android-35",
+ "package_xml": "\n \n 35\n 13\n true\n \n \n \n 2\n \n Android SDK Platform 35\n \n ",
+ "sha256": "0988cacad01b38a18a47bac14a0695f246bc76c1b06c0eeb8eb0dc825ab0c8e0"
+ },
+ {
+ "name": "ndk;28.1.13356709",
+ "url": "https://dl.google.com/android/repository/android-ndk-r28b-linux.zip",
+ "upstream_sha1": "f574d3165405bd59ffc5edaadac02689075a729f",
+ "format": "zip",
+ "destination": "/opt/android-sdk/ndk/28.1.13356709",
+ "archive_root": "android-ndk-r28b",
+ "package_xml": "\n \n \n 28\n 1\n 13356709\n \n NDK (Side by side) 28.1.13356709\n \n ",
+ "sha256": "e9f2759862cecfd48c20bbb7d8cfedbb020f4d91b5f78d9a2fc106f7db3c27ed"
+ },
+ {
+ "name": "build-tools;35.0.0",
+ "url": "https://dl.google.com/android/repository/build-tools_r35_linux.zip",
+ "upstream_sha1": "2cfaa0bbb2336e9ec18ed3ecea84fa2e2af607bc",
+ "format": "zip",
+ "destination": "/opt/android-sdk/build-tools/35.0.0",
+ "archive_root": "android-15",
+ "package_xml": "\n \n \n 35\n 0\n 0\n \n Android SDK Build-Tools 35\n \n ",
+ "sha256": "bd3a4966912eb8b30ed0d00b0cda6b6543b949d5ffe00bea54c04c81e1561d88"
+ },
+ {
+ "name": "cmdline-tools;19.0",
+ "url": "https://dl.google.com/android/repository/commandlinetools-linux-13114758_latest.zip",
+ "upstream_sha1": "5fdcc763663eefb86a5b8879697aa6088b041e70",
+ "format": "zip",
+ "destination": "/opt/android-sdk/cmdline-tools/19.0",
+ "archive_root": "cmdline-tools",
+ "package_xml": "\n \n \n 19\n 0\n \n Android SDK Command-line Tools\n \n ",
+ "sha256": "7ec965280a073311c339e571cd5de778b9975026cfcbe79f2b1cdcb1e15317ee"
+ },
+ {
+ "name": "platform-tools",
+ "url": "https://dl.google.com/android/repository/platform-tools_r37.0.1-linux.zip",
+ "upstream_sha1": "477254aa5f903c15cf51001717bdf347fb6b53e0",
+ "format": "zip",
+ "destination": "/opt/android-sdk/platform-tools",
+ "archive_root": "platform-tools",
+ "package_xml": "\n \n \n 37\n 0\n 1\n \n Android SDK Platform-Tools\n \n ",
+ "sha256": "d230f13842f60f782a8645f9c813f8f845bf36089ea7289f28c48f17979313f1"
+ },
+ {
+ "name": "emulator",
+ "url": "https://dl.google.com/android/repository/emulator-linux_x64-15917651.zip",
+ "upstream_sha1": "1b1f78891abf8ec268264356e1365c25519e8379",
+ "format": "zip",
+ "destination": "/opt/android-sdk/emulator",
+ "archive_root": "emulator",
+ "package_xml": "\n \n \n 37\n 1\n 11\n \n Android Emulator\n \n ",
+ "sha256": "95771e0ae431897b2a4bd2d97fa095f29a8b0624a7b216baf529f9306161c266"
+ },
+ {
+ "name": "system-images;android-35;default;x86_64",
+ "url": "https://dl.google.com/android/repository/sys-img/android/x86_64-35_r02.zip",
+ "upstream_sha1": "2d857d170c0d1b827149565da34b3383e5306f7f",
+ "format": "zip",
+ "destination": "/opt/android-sdk/system-images/android-35/default/x86_64",
+ "archive_root": "x86_64",
+ "package_xml": "\n \n 35\n 13\n true\n \n default\n Default Android System Image\n \n x86_64\n \n \n 2\n \n Intel x86_64 Atom System Image\n \n \n \n 29\n 1\n 11\n \n \n \n \n ",
+ "sha256": "6dd7de33e63ef105cf2fabea6badda1dbe7665c96d8908e5f6e1407e63ff4556"
+ }
+ ],
+ "bootstrap_ca": {
+ "url": "https://snapshot.ubuntu.com/ubuntu/20260920T000000Z/pool/main/c/ca-certificates/ca-certificates_20240203_all.deb",
+ "sha256": "641de77d8f142cfd62a1a6f964ba67b20754d3337c480efb529d086075a06c9a",
+ "version": "20240203"
+ },
+ "apt_packages": [
+ "ca-certificates",
+ "curl",
+ "git",
+ "gh",
+ "jq",
+ "python3",
+ "unzip",
+ "zip",
+ "xz-utils",
+ "bzip2",
+ "gnupg",
+ "build-essential",
+ "clang",
+ "llvm",
+ "libsnappy-dev",
+ "cmake",
+ "libgmp-dev",
+ "libssl-dev",
+ "pkg-config",
+ "openjdk-17-jdk-headless",
+ "libicu74",
+ "libkrb5-3",
+ "zlib1g",
+ "libgcc-s1",
+ "libstdc++6",
+ "libcurl4t64",
+ "liblttng-ust1t64",
+ "libunwind8",
+ "libpulse0",
+ "libx11-xcb1",
+ "libnss3",
+ "libxcomposite1",
+ "libxcursor1",
+ "libxi6",
+ "libxrandr2",
+ "libxtst6",
+ "libasound2t64",
+ "libgl1",
+ "libegl1",
+ "libdbus-1-3",
+ "libxdamage1",
+ "libxfixes3"
+ ],
+ "java_major": 17,
+ "versions": {
+ "runner": "2.337.0",
+ "llvm_cov": "0.9.1",
+ "nextest": "0.9.144",
+ "machete": "0.9.2",
+ "cargo_ndk": "4.1.2",
+ "protoc": "32.0",
+ "rustup": "1.28.2"
+ },
+ "android": {
+ "api": 35,
+ "build_tools": "35.0.0",
+ "ndk": "28.1.13356709",
+ "cmdline_tools": "19.0",
+ "system_image": "system-images;android-35;default;x86_64",
+ "abi": "x86_64"
+ },
+ "client_codegen": {
+ "schema": 1,
+ "versions": {
+ "protobuf": "3.18.1",
+ "grpc": "1.46.3",
+ "grpc_java": "1.42.1"
+ },
+ "sources": [
+ {
+ "url": "https://github.com/protocolbuffers/protobuf/releases/download/v3.18.1/protobuf-cpp-3.18.1.tar.gz",
+ "sha256": "6ee35eda3f79e49608d2ace8d866313fdec539d8bb14c6c54e8d2a16fa4e6780"
+ },
+ {
+ "url": "https://github.com/grpc/grpc/archive/refs/tags/v1.46.3.tar.gz",
+ "sha256": "d6cbf22cb5007af71b61c6be316a79397469c58c82a942552a62e708bce60964"
+ },
+ {
+ "url": "https://github.com/grpc/grpc-java/archive/refs/tags/v1.42.1.tar.gz",
+ "sha256": "33775a1ad05974bbba6ff97801cd9b326485b21fab91b08a4e1bc53e500e6326"
+ }
+ ]
+ }
+ }
+}
diff --git a/.github/scripts/kotlin-instrumented-tests.sh b/.github/scripts/kotlin-instrumented-tests.sh
new file mode 100644
index 00000000000..d0b239a64bc
--- /dev/null
+++ b/.github/scripts/kotlin-instrumented-tests.sh
@@ -0,0 +1,34 @@
+#!/usr/bin/env bash
+# Called from packages/kotlin-sdk by the image's ci-android-emulator wrapper.
+set -euo pipefail
+
+# Keystore's unlocked-device-required keys fail if the screen re-locks mid-test.
+adb shell settings put system screen_off_timeout 2147483647
+adb shell svc power stayon true
+
+# Auth-required identity keys need an enrolled secure lockscreen on the test AVD.
+adb shell locksettings set-pin 1234
+
+# Cold boot can race credential acceptance and keyguard dismissal. Preserve the
+# upstream retry and trust-state gate, now in one shell (not line-by-line sh -c).
+unlocked=false
+for attempt in 1 2 3; do
+ adb shell input keyevent KEYCODE_WAKEUP
+ adb shell wm dismiss-keyguard
+ adb shell input text 1234
+ adb shell input keyevent KEYCODE_ENTER
+ sleep 2
+ adb shell wm dismiss-keyguard
+ sleep 1
+ if adb shell dumpsys trust | grep -q 'deviceLocked=0'; then
+ unlocked=true
+ break
+ fi
+done
+if [ "$unlocked" != true ]; then
+ echo '::error::Emulator is still locked; Keystore-backed tests would fail spuriously.'
+ adb shell dumpsys trust
+ exit 1
+fi
+
+./gradlew :sdk:connectedDebugAndroidTest --stacktrace "$@"
diff --git a/.github/scripts/runner-image.py b/.github/scripts/runner-image.py
new file mode 100644
index 00000000000..6f55c45b158
--- /dev/null
+++ b/.github/scripts/runner-image.py
@@ -0,0 +1,218 @@
+#!/usr/bin/env python3
+"""Select an exact PR image and export the shared runner tool requirements."""
+import argparse
+import hashlib
+import json
+import os
+from pathlib import Path
+import re
+import subprocess
+import sys
+import time
+import urllib.parse
+import urllib.request
+
+MANIFEST = ".github/runner-requirements.json"
+ARM64_MANIFEST = ".github/runner-requirements.arm64.json"
+REPO = "dashpay/platform"
+
+
+def require(condition, message):
+ if not condition:
+ raise ValueError(message)
+
+
+def read_manifest(path):
+ data = Path(path).read_bytes()
+ require(len(data) <= 256 * 1024, "Requirements manifest is too large")
+ manifest = json.loads(data)
+ require(set(manifest) == {"schema", "recipe_revision", "requirements"} and manifest["schema"] == 1,
+ "Unsupported requirements manifest")
+ require(re.fullmatch(r"[0-9a-f]{40}", manifest["recipe_revision"]), "Pin the image recipe to a full SHA")
+ require(manifest["requirements"]["platform"] in ("linux/amd64", "linux/arm64"),
+ "Unsupported image platform")
+ if manifest["requirements"]["platform"] == "linux/arm64":
+ require(manifest["requirements"].get("profile") == "rust", "ARM64 requires the Rust-only profile")
+ return manifest
+
+
+def runtime_manifest():
+ # Hosted selector jobs must not select a manifest for the eventual runner.
+ # Only env/verify use the actual job runner's OS and architecture.
+ return ARM64_MANIFEST if (os.environ.get("RUNNER_OS") == "Linux"
+ and os.environ.get("RUNNER_ARCH") == "ARM64") else MANIFEST
+
+
+def fingerprint(value):
+ return hashlib.sha256(json.dumps(value, sort_keys=True, separators=(",", ":")).encode()).hexdigest()
+
+
+def api(path):
+ token = os.environ.get("GH_TOKEN", "")
+ headers = {"Accept": "application/vnd.github+json", "User-Agent": "platform-runner-image"}
+ if token:
+ headers["Authorization"] = "Bearer " + token
+ request = urllib.request.Request("https://api.github.com/repos/" + REPO + "/" + path, headers=headers)
+ with urllib.request.urlopen(request, timeout=30) as response:
+ return json.load(response)
+
+
+def changed_requirements(pr, manifest_path=MANIFEST):
+ require(pr.get("changed_files", 0) <= 3000, "PR exceeds GitHub's file-list limit; requirements need explicit review")
+ for page in range(1, 31):
+ files = api(f"pulls/{pr['number']}/files?per_page=100&page={page}")
+ if any(f["filename"] == manifest_path or f.get("previous_filename") == manifest_path for f in files):
+ return True
+ if len(files) < 100:
+ return False
+ return False
+
+
+def latest_status(head, context):
+ # The combined /status endpoint omits creator. Full statuses are newest
+ # first: select before validating, never fall back to an older success.
+ page = 1
+ while True:
+ statuses = api(f"commits/{head}/statuses?per_page=100&page={page}")
+ # GitHub contexts are case-insensitive; a case variant must shadow
+ # older canonical statuses even though it cannot be trusted below.
+ candidate = next((s for s in statuses if s["context"].casefold() == context.casefold()), None)
+ if candidate is not None:
+ return candidate
+ if len(statuses) < 100:
+ return None
+ page += 1
+
+
+def export_environment(manifest, output):
+ lock = manifest["requirements"]
+ versions = lock["versions"]
+ values = {
+ "CI_CARGO_LLVM_COV_VERSION": versions["llvm_cov"],
+ "CI_CARGO_NEXTEST_VERSION": versions["nextest"],
+ "CI_CARGO_MACHETE_VERSION": versions["machete"],
+ "CI_PROTOC_VERSION": versions["protoc"], "CI_JAVA_MAJOR": str(lock["java_major"]),
+ }
+ if lock.get("profile", "full") == "full":
+ android = lock["android"]
+ values.update({
+ "CI_CARGO_NDK_VERSION": versions["cargo_ndk"],
+ "CI_ANDROID_API": str(android["api"]), "CI_ANDROID_NDK": android["ndk"],
+ "CI_ANDROID_BUILD_TOOLS": android["build_tools"],
+ })
+ require(all(isinstance(value, str) and re.fullmatch(r"[0-9]+(?:[.][0-9]+){0,3}(?:[-+][A-Za-z0-9.-]+)?", value)
+ for value in values.values()), "Versions must be version-pinned, newline-free values")
+ with open(output, "a") as handle:
+ for key, value in values.items():
+ handle.write(f"{key}={value}\n")
+
+
+def select(manifest, kind, output, wait_seconds, arch=None, validation=False):
+ require(arch in (None, "", "X64", "ARM64"), "Unsupported runner architecture")
+ require(not arch or kind == "rust", "Architecture selection is only supported for Rust")
+ require(not validation or (kind == "rust" and arch == "ARM64"),
+ "The validation-only pool is for explicitly selected ARM64 Rust jobs")
+ # Linux describes the runner process, not the physical host: ARM64 Linux
+ # containers on Macs remain in this pool; native macOS stays for Swift.
+ fallback = ["self-hosted"] + (["Linux"] if kind == "rust" else [])
+ if arch:
+ fallback.append(arch)
+ fallback.append("rust-ci-validation" if validation else
+ {"rust": "rust-ci", "kotlin": "kotlin-ci", "npm": "npm-pr"}[kind])
+ event = json.loads(Path(os.environ["GITHUB_EVENT_PATH"]).read_text())
+ requested = event.get("pull_request")
+ labels, changed = fallback, False
+ if requested:
+ pr = api(f"pulls/{requested['number']}")
+ head = requested["head"]["sha"]
+ require(pr["state"] == "open" and pr["head"]["sha"] == head, "This PR run has been superseded")
+ changed = changed_requirements(pr, ARM64_MANIFEST if arch == "ARM64" else MANIFEST)
+ if not arch and kind == "rust" and changed_requirements(pr, ARM64_MANIFEST):
+ # Only validation capacity has the new ARM64 image before rollout.
+ # Keep this PR's ordinary job on unchanged AMD64 capacity while its
+ # separate ARM64 job proves the new manifest on the validation pool.
+ labels = fallback = ["self-hosted", "Linux", "X64", "rust-ci"]
+ if arch == "ARM64":
+ # ARM64 is explicitly provisioned from a published immutable image.
+ # The AMD64/KVM candidate publisher is not ARM64 validation. The
+ # separate ARM64 job verifies its exact lock/recipe on real capacity.
+ if changed:
+ import base64
+ remote = api("contents/" + ARM64_MANIFEST + "?" + urllib.parse.urlencode({"ref": head}))
+ expected = json.loads(base64.b64decode(remote["content"]))
+ require(fingerprint(expected) == fingerprint(read_manifest(ARM64_MANIFEST)),
+ "Merge-tree ARM64 requirements differ from PR head; rebase before validation")
+ with open(output, "a") as handle:
+ handle.write("labels=" + json.dumps(fallback, separators=(",", ":")) + "\n")
+ handle.write("image_changed=" + str(changed).lower() + "\n")
+ print("Using explicitly provisioned ARM64 image capacity; exact runtime verification is required")
+ return
+ if changed:
+ # Use the exact PR requirement, not an accidental merge-tree mix
+ # after both branches edited this file. Rebase such a PR first.
+ import base64
+ remote = api("contents/" + MANIFEST + "?" + urllib.parse.urlencode({"ref": head}))
+ expected = json.loads(base64.b64decode(remote["content"]))
+ require(fingerprint(expected) == fingerprint(manifest),
+ "Merge-tree requirements differ from PR head; rebase before building a candidate")
+ deadline = time.monotonic() + wait_seconds
+ context = f"Runner image candidate / PR {pr['number']}"
+ while True:
+ candidate = latest_status(head, context)
+ if candidate and candidate["state"] == "success":
+ require(candidate["context"] == context, "Candidate status must use the exact publisher context")
+ creator = candidate.get("creator")
+ require(isinstance(creator, dict) and creator.get("login") == "github-actions[bot]",
+ "Candidate status must come from the trusted publisher")
+ require(re.fullmatch(r"sha256:[0-9a-f]{64}", candidate.get("description", "")),
+ "Publisher did not record an immutable digest")
+ match = re.fullmatch(r"https://github[.]com/dashpay/platform/actions/runs/([0-9]+)",
+ candidate.get("target_url", ""))
+ require(match, "Candidate status is not linked to its publishing workflow")
+ run = api(f"actions/runs/{match.group(1)}")
+ require(run["path"] == ".github/workflows/runner-image-candidate.yml"
+ and run["event"] == "pull_request_target", "Unexpected candidate publisher")
+ if run["conclusion"] == "success":
+ labels = ["self-hosted", "Linux", "X64",
+ f"platform-image-pr-{pr['number']}-{head}-{candidate['description'][7:]}-{kind}"]
+ break
+ require(time.monotonic() < deadline,
+ "Candidate image was not published in time. Check Runner image candidate CI and bootstrap setup.")
+ current = api(f"pulls/{pr['number']}")
+ require(current["state"] == "open" and current["head"]["sha"] == head, "PR changed while waiting")
+ time.sleep(20)
+ with open(output, "a") as handle:
+ handle.write("labels=" + json.dumps(labels, separators=(",", ":")) + "\n")
+ handle.write("image_changed=" + str(changed).lower() + "\n")
+ print("Candidate runner required" if changed else "Using the ordinary provisioned runner pool")
+
+
+def main():
+ parser = argparse.ArgumentParser(description=__doc__)
+ parser.add_argument("command", choices=["env", "verify", "select"])
+ parser.add_argument("--manifest")
+ parser.add_argument("--env", default=os.environ.get("GITHUB_ENV"))
+ parser.add_argument("--output", default=os.environ.get("GITHUB_OUTPUT"))
+ parser.add_argument("--kind", choices=["rust", "kotlin", "npm"])
+ parser.add_argument("--arch", choices=["", "X64", "ARM64"])
+ parser.add_argument("--validation", action="store_true")
+ parser.add_argument("--wait-seconds", type=int, default=7200)
+ args = parser.parse_args()
+ manifest_path = args.manifest or (MANIFEST if args.command == "select" else runtime_manifest())
+ manifest = read_manifest(manifest_path)
+ if args.command == "select":
+ require(args.kind and args.output, "Runner selection needs kind and output")
+ select(manifest, args.kind, args.output, args.wait_seconds, args.arch, args.validation)
+ return
+ if args.command == "verify":
+ subprocess.run(["ci-image-contract", "verify", manifest_path], check=True)
+ require(args.env, "Environment output file is required")
+ export_environment(manifest, args.env)
+
+
+if __name__ == "__main__":
+ try:
+ main()
+ except (ValueError, KeyError, TypeError, OSError) as error:
+ print(f"::error::{error}", file=sys.stderr)
+ sys.exit(1)
diff --git a/.github/scripts/tests/fixtures/candidate-status-pr5151.json b/.github/scripts/tests/fixtures/candidate-status-pr5151.json
new file mode 100644
index 00000000000..7cb66d5a787
--- /dev/null
+++ b/.github/scripts/tests/fixtures/candidate-status-pr5151.json
@@ -0,0 +1,40 @@
+{
+ "combined": {
+ "state": "pending",
+ "sha": "a02b1460736e18b6345bb4722c622e55e787371d",
+ "total_count": 4,
+ "statuses": [
+ {
+ "id": 55116170875,
+ "context": "Runner image candidate / PR 5151",
+ "state": "success",
+ "description": "sha256:e5ebd957d28d15976320023b76cffa8b982e1d131c572d92eb9c91814dbf807b",
+ "target_url": "https://github.com/dashpay/platform/actions/runs/36474975257",
+ "created_at": "2026-09-28T20:13:10Z",
+ "updated_at": "2026-09-28T20:13:10Z"
+ }
+ ]
+ },
+ "statuses": [
+ {
+ "id": 55116170875,
+ "context": "Runner image candidate / PR 5151",
+ "state": "success",
+ "description": "sha256:e5ebd957d28d15976320023b76cffa8b982e1d131c572d92eb9c91814dbf807b",
+ "target_url": "https://github.com/dashpay/platform/actions/runs/36474975257",
+ "created_at": "2026-09-28T20:13:10Z",
+ "updated_at": "2026-09-28T20:13:10Z",
+ "creator": {
+ "login": "github-actions[bot]",
+ "id": 41898282
+ }
+ }
+ ],
+ "publisher_run": {
+ "id": 36474975257,
+ "path": ".github/workflows/runner-image-candidate.yml",
+ "event": "pull_request_target",
+ "status": "completed",
+ "conclusion": "success"
+ }
+}
diff --git a/.github/scripts/tests/test_npm_release_boundary.py b/.github/scripts/tests/test_npm_release_boundary.py
new file mode 100644
index 00000000000..f457408204b
--- /dev/null
+++ b/.github/scripts/tests/test_npm_release_boundary.py
@@ -0,0 +1,100 @@
+"""Exercise caller/runtime guards and preserve the PR/release cache boundary."""
+import os
+from pathlib import Path
+import subprocess
+import textwrap
+import unittest
+
+ROOT = Path(__file__).resolve().parents[3]
+
+
+def step_script(filename, name):
+ workflow = (ROOT / '.github/workflows' / filename).read_text()
+ start = workflow.index(' - name: ' + name)
+ tail = workflow[start:].split(' run: |\n', 1)[1]
+ lines = []
+ for line in tail.splitlines():
+ if line.strip() and not line.startswith(' '):
+ break
+ lines.append(line)
+ return textwrap.dedent('\n'.join(lines))
+
+
+class ReleaseBoundaryTests(unittest.TestCase):
+ def run_guard(self, event, ref, repository='dashpay/platform', protected=False):
+ script = step_script('release-npm-build.yml', 'Reject untrusted release callers')
+ return subprocess.run(['bash', '-c', script], capture_output=True, text=True,
+ env=dict(os.environ, GITHUB_REPOSITORY=repository,
+ GITHUB_EVENT_NAME=event, GITHUB_REF=ref,
+ GITHUB_REF_PROTECTED=str(protected).lower())).returncode
+
+ def test_should_allow_release_tags_and_protected_branch_dry_runs(self):
+ for event, ref in [('release', 'refs/tags/v4.2.0-beta.5'),
+ ('workflow_dispatch', 'refs/tags/v4.2.0-beta.5'),
+ ('workflow_dispatch', 'refs/heads/v4.2-dev'),
+ ('workflow_dispatch', 'refs/heads/v4.3-dev')]:
+ with self.subTest(event=event, ref=ref):
+ self.assertEqual(self.run_guard(event, ref, protected=True), 0)
+
+ def test_should_reject_prs_forks_and_unprotected_dispatches(self):
+ for event, ref in [('pull_request', 'refs/pull/5068/merge'),
+ ('pull_request_target', 'refs/heads/v4.2-dev'),
+ ('workflow_dispatch', 'refs/heads/attacker'),
+ ('push', 'refs/heads/v4.2-dev'),
+ ('release', 'refs/heads/v4.2-dev')]:
+ with self.subTest(event=event, ref=ref):
+ self.assertNotEqual(self.run_guard(event, ref), 0)
+ self.assertNotEqual(self.run_guard('release', 'refs/tags/v4.2.0', 'unknown/platform'), 0)
+ self.assertNotEqual(self.run_guard('workflow_dispatch', 'refs/heads/v4.2-dev'), 0)
+
+ def test_should_reject_persistent_wrong_attempt_and_wrong_kind_runners(self):
+ for kind, filename in [('npm', 'release-npm-build.yml'), ('kotlin', 'release-kotlin-sdk.yml')]:
+ script = step_script(filename, 'Verify disposable release runner')
+ env = dict(os.environ, GITHUB_RUN_ID='123', GITHUB_RUN_ATTEMPT='2',
+ DASH_RELEASE_RUNNER='1', DASH_RELEASE_RUN_ID='123',
+ DASH_RELEASE_RUN_ATTEMPT='2', DASH_RELEASE_KIND=kind)
+ self.assertEqual(subprocess.run(['bash', '-c', script], env=env).returncode, 0)
+ for key, value in [('DASH_RELEASE_RUNNER', ''), ('DASH_RELEASE_RUN_ID', '124'),
+ ('DASH_RELEASE_RUN_ATTEMPT', '1'), ('DASH_RELEASE_KIND', 'rust')]:
+ with self.subTest(kind=kind, key=key):
+ self.assertNotEqual(subprocess.run(['bash', '-c', script],
+ env=dict(env, **{key: value})).returncode, 0)
+
+ def test_should_route_both_release_builds_away_from_persistent_ci(self):
+ for kind, filename in [('npm', 'release-npm-build.yml'), ('kotlin', 'release-kotlin-sdk.yml')]:
+ workflow = (ROOT / '.github/workflows' / filename).read_text()
+ build = workflow.split(' attach-release:', 1)[0]
+ self.assertIn('group: platform-release-builds', build)
+ self.assertIn('platform-release-${{ github.run_id }}-${{ github.run_attempt }}-' + kind, build)
+ self.assertNotIn('runs-on: [self-hosted, kotlin-ci]', build)
+ self.assertNotIn('Linux, X64, npm-build', build)
+ self.assertLess(build.index('Verify disposable release runner'), build.index('uses: actions/checkout'))
+ caller = (ROOT / '.github/workflows/release.yml').read_text()
+ self.assertIn('uses: ./.github/workflows/release-npm-build.yml', caller)
+ self.assertNotIn('release-npm-build.yml@v4.2-dev', caller)
+
+ def test_should_keep_pr_caching_enabled_but_exclude_it_from_releases(self):
+ node = (ROOT / '.github/actions/nodejs/action.yaml').read_text()
+ cache_input = node.split(' cache:\n', 1)[1].split(' node-version:', 1)[0]
+ self.assertIn('default: "true"', cache_input)
+ self.assertIn("if: inputs.cache == 'true'", node)
+ self.assertIn('uses: actions/cache@v5', node)
+ npm = (ROOT / '.github/actions/npm-release-build/action.yaml').read_text()
+ self.assertIn("cache: ${{ env.DASH_RELEASE_RUNNER != '1' }}", npm)
+ kotlin = (ROOT / '.github/workflows/release-kotlin-sdk.yml').read_text()
+ for gradle in kotlin.split('uses: gradle/actions/setup-gradle@v4')[1:]:
+ self.assertIn('cache-disabled: true', gradle.split('\n - ', 1)[0])
+ # The ordinary image-validation workflow retains its own cache name
+ # and does not opt into the release runtime marker.
+ validation = (ROOT / '.github/workflows/npm-runner-validation.yml').read_text()
+ self.assertIn('cache-name: npm-validation-target', validation)
+ self.assertNotIn('DASH_RELEASE_RUNNER:', validation)
+
+ def test_should_keep_kotlin_dry_runs_out_of_both_publishing_jobs(self):
+ kotlin = (ROOT / '.github/workflows/release-kotlin-sdk.yml').read_text()
+ attach = kotlin.split(' attach-release:', 1)[1].split(' steps:', 1)[0]
+ maven = kotlin.split(' maven-central-deploy:', 1)[1].split(' steps:', 1)[0]
+ self.assertIn('if: ${{ !inputs.dry_run }}', attach)
+ self.assertIn('if: ${{ !inputs.dry_run &&', maven)
+ self.assertIn("github.ref == format('refs/tags/{0}', inputs.tag)", maven)
+ self.assertIn('environment: maven-central', maven)
diff --git a/.github/scripts/tests/test_runner_image.py b/.github/scripts/tests/test_runner_image.py
new file mode 100644
index 00000000000..c811e406d35
--- /dev/null
+++ b/.github/scripts/tests/test_runner_image.py
@@ -0,0 +1,331 @@
+"""Exercise runner routing, stale candidates and safe environment exports."""
+import base64
+import copy
+import importlib.util
+import json
+import os
+import sys
+from pathlib import Path
+import tempfile
+import unittest
+from urllib.error import URLError
+from unittest.mock import patch
+
+ROOT = Path(__file__).resolve().parents[3]
+spec = importlib.util.spec_from_file_location("runner_image", ROOT / ".github/scripts/runner-image.py")
+runner = importlib.util.module_from_spec(spec)
+spec.loader.exec_module(runner)
+# Minimal projections of public REST responses captured 2026-09-28 for PR 5151.
+# /status has Simple Commit Status objects (no creator); /statuses has creator.
+FIXTURE = json.loads((Path(__file__).parent / "fixtures/candidate-status-pr5151.json").read_text())
+HEAD = FIXTURE["combined"]["sha"]
+DIGEST = FIXTURE["statuses"][0]["description"]
+STATUS_PATH = f"commits/{HEAD}/statuses?per_page=100&page=1"
+RUN_PATH = f"actions/runs/{FIXTURE['publisher_run']['id']}"
+
+
+class SelectorTests(unittest.TestCase):
+ def setUp(self):
+ self.manifest = runner.read_manifest(ROOT / runner.MANIFEST)
+ self.temp = tempfile.TemporaryDirectory()
+ self.addCleanup(self.temp.cleanup)
+ self.event = Path(self.temp.name) / "event.json"
+ self.output = Path(self.temp.name) / "output"
+ self.pr = {"number": 5151, "state": "open", "changed_files": 1,
+ "head": {"sha": HEAD}}
+ self.responses = {
+ "pulls/5151": self.pr,
+ "pulls/5151/files?per_page=100&page=1": [{"filename": runner.MANIFEST}],
+ f"contents/{runner.MANIFEST}?ref={HEAD}": {
+ "content": base64.b64encode(json.dumps(self.manifest).encode()).decode()},
+ f"commits/{HEAD}/status": copy.deepcopy(FIXTURE["combined"]),
+ STATUS_PATH: copy.deepcopy(FIXTURE["statuses"]),
+ RUN_PATH: copy.deepcopy(FIXTURE["publisher_run"]),
+ }
+
+ def api_response(self, path):
+ response = self.responses[path]
+ if isinstance(response, Exception):
+ raise response
+ return response
+
+ def status_calls(self):
+ return [call.args[0] for call in self.api.call_args_list
+ if call.args[0].startswith("commits/")]
+
+ def select(self, event=None, kind="rust", arch=None, validation=False, wait_seconds=0):
+ self.event.write_text(json.dumps(event if event is not None else {"pull_request": self.pr}))
+ with patch.dict(os.environ, {"GITHUB_EVENT_PATH": str(self.event)}), \
+ patch.object(runner, "api", side_effect=self.api_response) as api:
+ self.api = api
+ runner.select(self.manifest, kind, self.output, wait_seconds, arch, validation)
+ return dict(line.split("=", 1) for line in self.output.read_text().splitlines())
+
+ def test_non_pr_and_unchanged_pr_use_existing_pool(self):
+ self.assertEqual(json.loads(self.select({})["labels"]), ["self-hosted", "Linux", "rust-ci"])
+ self.responses["pulls/5151/files?per_page=100&page=1"] = [{"filename": "Cargo.lock"}]
+ self.assertEqual(self.select()["image_changed"], "false")
+
+ def test_exact_candidate_includes_head_digest_and_kind(self):
+ self.assertNotIn("creator", FIXTURE["combined"]["statuses"][0])
+ for kind in ("kotlin", "rust", "npm"):
+ with self.subTest(kind=kind):
+ output = self.select(kind=kind)
+ self.assertEqual(json.loads(output["labels"]), ["self-hosted", "Linux", "X64",
+ f"platform-image-pr-5151-{HEAD}-{DIGEST[7:]}-{kind}"])
+ self.assertEqual(output["image_changed"], "true")
+ self.assertEqual(self.status_calls(), [STATUS_PATH])
+
+ def test_should_keep_npm_ordinary_pool_labels(self):
+ self.assertEqual(json.loads(self.select({}, kind="npm")["labels"]), ["self-hosted", "npm-pr"])
+
+ def test_new_head_or_closed_pr_rejects_stale_run(self):
+ event = {"pull_request": copy.deepcopy(self.pr)}
+ self.pr["head"]["sha"] = "e" * 40
+ with self.assertRaisesRegex(ValueError, "superseded"):
+ self.select(event)
+ self.pr["head"]["sha"] = HEAD
+ self.pr["state"] = "closed"
+ with self.assertRaisesRegex(ValueError, "superseded"):
+ self.select(event)
+
+ def test_merge_tree_cannot_mix_requirements_from_both_branches(self):
+ self.manifest["recipe_revision"] = "e" * 40
+ with self.assertRaisesRegex(ValueError, "rebase"):
+ self.select()
+
+ def test_missing_or_incomplete_publisher_cannot_select_image(self):
+ for conclusion in (None, "failure", "cancelled"):
+ with self.subTest(conclusion=conclusion):
+ self.responses[RUN_PATH]["conclusion"] = conclusion
+ with self.assertRaisesRegex(ValueError, "not published"):
+ self.select()
+ self.assertFalse(self.output.exists())
+ self.responses[STATUS_PATH] = []
+ with self.assertRaisesRegex(ValueError, "not published"):
+ self.select()
+
+ def test_other_workflow_cannot_supply_candidate_status(self):
+ for field, value in (("path", ".github/workflows/tests.yml"), ("event", "pull_request")):
+ with self.subTest(field=field):
+ self.responses[RUN_PATH] = dict(FIXTURE["publisher_run"], **{field: value})
+ with self.assertRaisesRegex(ValueError, "Unexpected candidate"):
+ self.select()
+ self.assertFalse(self.output.exists())
+
+ def test_should_reject_missing_null_malformed_or_wrong_creator_without_fallback(self):
+ # The real combined response is also a regression case: no creator.
+ missing = FIXTURE["combined"]["statuses"][0]
+ good = FIXTURE["statuses"][0]
+ candidates = [missing] + [dict(good, creator=creator) for creator in (
+ None, {}, "github-actions[bot]", [], 42,
+ {"login": None}, {"login": "untrusted-user"},
+ )]
+ for candidate in candidates:
+ with self.subTest(creator=candidate.get("creator", "absent")):
+ self.responses[STATUS_PATH] = [candidate, good]
+ with self.assertRaisesRegex(ValueError, "trusted publisher"):
+ self.select()
+ self.assertEqual(self.status_calls(), [STATUS_PATH])
+ self.assertNotIn(RUN_PATH, [call.args[0] for call in self.api.call_args_list])
+ self.assertFalse(self.output.exists())
+
+ def test_should_reject_invalid_digest_or_publisher_url_without_fallback(self):
+ good = FIXTURE["statuses"][0]
+ for field, value, error in (
+ ("description", "sha256:abc", "immutable digest"),
+ ("description", "latest", "immutable digest"),
+ ("target_url", "https://github.com/other/platform/actions/runs/7", "publishing workflow"),
+ ("target_url", good["target_url"] + "/jobs/1", "publishing workflow"),
+ ):
+ with self.subTest(field=field, value=value):
+ self.responses[STATUS_PATH] = [dict(good, **{field: value}), good]
+ with self.assertRaisesRegex(ValueError, error):
+ self.select()
+ self.assertFalse(self.output.exists())
+
+ def test_should_block_older_success_when_newest_is_pending_failure_or_error(self):
+ good = FIXTURE["statuses"][0]
+ for state in ("pending", "failure", "error"):
+ with self.subTest(state=state):
+ self.responses[STATUS_PATH] = [dict(good, state=state), good]
+ with self.assertRaisesRegex(ValueError, "not published"):
+ self.select()
+ self.assertEqual(self.status_calls(), [STATUS_PATH])
+ self.assertNotIn(RUN_PATH, [call.args[0] for call in self.api.call_args_list])
+ self.assertFalse(self.output.exists())
+
+ def test_should_use_first_success_without_unnecessary_pagination(self):
+ good = FIXTURE["statuses"][0]
+ older = dict(good, description="sha256:" + "e" * 64)
+ self.responses[STATUS_PATH] = [good] + [older] * 99
+ labels = json.loads(self.select()["labels"])
+ self.assertEqual(labels[-1], f"platform-image-pr-5151-{HEAD}-{DIGEST[7:]}-rust")
+ self.assertEqual(self.status_calls(), [STATUS_PATH])
+
+ def test_should_shadow_canonical_success_with_newer_case_variant(self):
+ good = FIXTURE["statuses"][0]
+ for state in ("success", "pending", "failure", "error"):
+ with self.subTest(state=state):
+ newer = dict(good, context=good["context"].lower(), state=state)
+ self.responses[STATUS_PATH] = [newer] + [good] * 99
+ error = "exact publisher context" if state == "success" else "not published"
+ with self.assertRaisesRegex(ValueError, error):
+ self.select()
+ self.assertEqual(self.status_calls(), [STATUS_PATH])
+ self.assertNotIn(RUN_PATH, [call.args[0] for call in self.api.call_args_list])
+ self.assertFalse(self.output.exists())
+
+ def test_should_select_first_match_on_later_page_before_validation(self):
+ good = FIXTURE["statuses"][0]
+ self.responses[STATUS_PATH] = [dict(good, context="unrelated")] * 100
+ page2 = STATUS_PATH.replace("&page=1", "&page=2")
+ for state in ("pending", "success"):
+ with self.subTest(state=state):
+ self.responses[page2] = [dict(good, state=state)] + [good] * 99
+ if state == "pending":
+ with self.assertRaisesRegex(ValueError, "not published"):
+ self.select()
+ self.assertFalse(self.output.exists())
+ else:
+ self.assertEqual(self.select()["image_changed"], "true")
+ self.assertEqual(self.status_calls(), [STATUS_PATH, page2])
+
+ def test_should_require_exact_context_and_stop_at_page_exhaustion(self):
+ good = FIXTURE["statuses"][0]
+ unrelated = [dict(good, context=context) for context in (
+ "Runner image candidate / PR 51510", "Runner image candidate / PR 5151 suffix",
+ "Runner image candidate / PR 5151 ", "Runner image candidate / PR 515",
+ )]
+ page2 = STATUS_PATH.replace("&page=1", "&page=2")
+ for last_page in ([], unrelated):
+ with self.subTest(last_page_size=len(last_page)):
+ self.responses[STATUS_PATH] = unrelated * 25
+ self.responses[page2] = last_page
+ with self.assertRaisesRegex(ValueError, "not published"):
+ self.select()
+ self.assertEqual(self.status_calls(), [STATUS_PATH, page2])
+ self.assertFalse(self.output.exists())
+
+ def test_should_fail_closed_on_status_api_failure(self):
+ page2 = STATUS_PATH.replace("&page=1", "&page=2")
+ for failed_path in (STATUS_PATH, page2):
+ with self.subTest(failed_path=failed_path):
+ self.responses[STATUS_PATH] = [dict(FIXTURE["statuses"][0], context="other")] * 100
+ self.responses[failed_path] = URLError("status API unavailable")
+ with self.assertRaisesRegex(URLError, "status API unavailable"):
+ self.select()
+ self.assertFalse(self.output.exists())
+
+ def test_should_retry_pending_status_and_publisher_run(self):
+ good = FIXTURE["statuses"][0]
+ for pending in ("status", "run"):
+ with self.subTest(pending=pending):
+ self.responses[STATUS_PATH] = [dict(good, state="pending"), good] if pending == "status" else [good]
+ self.responses[RUN_PATH]["conclusion"] = None if pending == "run" else "success"
+
+ def publish(_seconds):
+ self.responses[STATUS_PATH] = [good]
+ self.responses[RUN_PATH]["conclusion"] = "success"
+
+ with patch.object(runner.time, "monotonic", return_value=0), \
+ patch.object(runner.time, "sleep", side_effect=publish) as sleep:
+ self.assertEqual(self.select(wait_seconds=60)["image_changed"], "true")
+ sleep.assert_called_once_with(20)
+ self.assertEqual(self.status_calls(), [STATUS_PATH, STATUS_PATH])
+ self.assertEqual([call.args[0] for call in self.api.call_args_list].count("pulls/5151"), 2)
+
+ def test_environment_export_rejects_multiline_values_before_writing(self):
+ self.manifest["requirements"]["versions"]["protoc"] = "32.0\nINJECTED=yes"
+ with self.assertRaisesRegex(ValueError, "newline-free"):
+ runner.export_environment(self.manifest, self.output)
+ self.assertFalse(self.output.exists())
+
+ def test_arm64_validation_never_consumes_amd64_candidate_status(self):
+ self.responses[STATUS_PATH] = []
+ output = self.select(arch="ARM64")
+ self.assertEqual(json.loads(output["labels"]), ["self-hosted", "Linux", "ARM64", "rust-ci"])
+
+ def test_arm64_manifest_change_does_not_use_stale_ordinary_arm64_runners(self):
+ self.responses["pulls/5151/files?per_page=100&page=1"] = [{"filename": runner.ARM64_MANIFEST}]
+ output = self.select()
+ self.assertEqual(json.loads(output["labels"]), ["self-hosted", "Linux", "X64", "rust-ci"])
+ self.assertEqual(output["image_changed"], "false")
+ # Mac-backed ARM64 capacity remains available to ordinary, unrelated PRs.
+ self.responses["pulls/5151/files?per_page=100&page=1"] = [{"filename": "Cargo.lock"}]
+ self.assertEqual(json.loads(self.select()["labels"]), ["self-hosted", "Linux", "rust-ci"])
+
+ def test_both_manifest_changes_still_require_exact_amd64_candidate(self):
+ self.responses["pulls/5151/files?per_page=100&page=1"] = [
+ {"filename": runner.MANIFEST}, {"filename": runner.ARM64_MANIFEST}]
+ output = self.select()
+ self.assertEqual(json.loads(output["labels"]), ["self-hosted", "Linux", "X64",
+ f"platform-image-pr-5151-{HEAD}-{DIGEST[7:]}-rust"])
+ self.assertEqual(output["image_changed"], "true")
+
+ def test_arm64_validation_rejects_merge_tree_drift(self):
+ arm = runner.read_manifest(ROOT / runner.ARM64_MANIFEST)
+ self.responses["pulls/5151/files?per_page=100&page=1"] = [{"filename": runner.ARM64_MANIFEST}]
+ remote = copy.deepcopy(arm)
+ remote["recipe_revision"] = "e" * 40
+ self.responses[f"contents/{runner.ARM64_MANIFEST}?ref={HEAD}"] = {
+ "content": base64.b64encode(json.dumps(remote).encode()).decode()}
+ with patch.object(runner, "read_manifest", return_value=arm):
+ with self.assertRaisesRegex(ValueError, "rebase"):
+ self.select(arch="ARM64")
+ self.responses[f"contents/{runner.ARM64_MANIFEST}?ref={HEAD}"] = {
+ "content": base64.b64encode(json.dumps(arm).encode()).decode()}
+ with patch.object(runner, "read_manifest", return_value=arm):
+ self.assertEqual(self.select(arch="ARM64")["image_changed"], "true")
+
+ def test_arm64_cannot_be_requested_for_android(self):
+ with self.assertRaisesRegex(ValueError, "only supported for Rust"):
+ self.select(kind="kotlin", arch="ARM64")
+
+ def test_arm64_validation_pool_is_not_ordinary_capacity(self):
+ labels = json.loads(self.select({}, arch="ARM64", validation=True)["labels"])
+ self.assertEqual(labels, ["self-hosted", "Linux", "ARM64", "rust-ci-validation"])
+ self.assertNotIn("rust-ci", labels)
+ with self.assertRaisesRegex(ValueError, "validation-only pool"):
+ self.select({}, validation=True)
+
+ def test_runtime_manifest_keeps_native_macos_and_amd64_separate(self):
+ for os_name, arch, expected in [
+ ("Linux", "ARM64", runner.ARM64_MANIFEST),
+ ("Linux", "X64", runner.MANIFEST),
+ ("macOS", "ARM64", runner.MANIFEST),
+ ]:
+ with self.subTest(os=os_name, arch=arch), \
+ patch.dict(os.environ, {"RUNNER_OS": os_name, "RUNNER_ARCH": arch}):
+ self.assertEqual(runner.runtime_manifest(), expected)
+
+ def test_arm64_verify_uses_exact_contract_without_android_exports(self):
+ with patch.dict(os.environ, {"RUNNER_OS": "Linux", "RUNNER_ARCH": "ARM64",
+ "GITHUB_ENV": str(self.output)}), \
+ patch.object(sys, "argv", ["runner-image.py", "verify"]), \
+ patch.object(runner.subprocess, "run") as verify:
+ runner.main()
+ verify.assert_called_once_with(["ci-image-contract", "verify", runner.ARM64_MANIFEST], check=True)
+ values = dict(line.split("=", 1) for line in self.output.read_text().splitlines())
+ self.assertEqual(values["CI_CARGO_NEXTEST_VERSION"], "0.9.144")
+ self.assertFalse(any("ANDROID" in name or "NDK" in name for name in values))
+
+ def test_shared_rust_toolchain_does_not_drift_between_architectures(self):
+ amd = self.manifest["requirements"]
+ arm = runner.read_manifest(ROOT / runner.ARM64_MANIFEST)["requirements"]
+ self.assertEqual(arm["versions"], {k: v for k, v in amd["versions"].items() if k != "cargo_ndk"})
+ for key in ("rust_version", "rust_manifest_sha256", "apt_snapshot", "java_major", "client_codegen"):
+ self.assertEqual(amd[key], arm[key], key)
+
+ def test_rejected_image_contract_stops_before_environment_export(self):
+ with patch.dict(os.environ, {"RUNNER_OS": "Linux", "RUNNER_ARCH": "ARM64",
+ "GITHUB_ENV": str(self.output)}), \
+ patch.object(sys, "argv", ["runner-image.py", "verify"]), \
+ patch.object(runner.subprocess, "run", side_effect=runner.subprocess.CalledProcessError(1, "ci-image-contract")):
+ with self.assertRaises(runner.subprocess.CalledProcessError):
+ runner.main()
+ self.assertFalse(self.output.exists())
+
+
+if __name__ == "__main__":
+ unittest.main()
diff --git a/.github/workflows/kotlin-sdk-build.yml b/.github/workflows/kotlin-sdk-build.yml
index 8919fd7b7dd..f1fc725e270 100644
--- a/.github/workflows/kotlin-sdk-build.yml
+++ b/.github/workflows/kotlin-sdk-build.yml
@@ -18,11 +18,14 @@ on:
- 'packages/*-contract/**'
- 'packages/simple-signer/**'
- 'docs/sdk/sdk-parity-manifest.json'
- - 'docs/sdk/KOTLIN_SWIFT_SHARED_PARITY_SPEC.md'
- 'packages/kotlin-sdk/PARITY_SUMMARY.md'
- 'scripts/check_sdk_parity_manifest.py'
- 'scripts/tests/**'
- '.github/workflows/kotlin-sdk-build.yml'
+ - '.github/scripts/kotlin-instrumented-tests.sh'
+ - '.github/actions/rust/**'
+ - '.github/runner-requirements.json'
+ - '.github/scripts/runner-image.py'
permissions:
contents: read
@@ -32,9 +35,36 @@ concurrency:
cancel-in-progress: true
jobs:
+ select-runner:
+ name: Select compatible runner image
+ if: >-
+ github.event_name != 'pull_request'
+ || github.event.pull_request.head.repo.full_name == github.repository
+ || github.event.pull_request.head.repo.owner.login == 'thepastaclaw'
+ runs-on: ubuntu-24.04
+ # Allow the separate 90-minute build and publication/queue window to finish.
+ timeout-minutes: 135
+ permissions:
+ contents: read
+ pull-requests: read
+ statuses: read
+ actions: read
+ outputs:
+ labels: ${{ steps.select.outputs.labels }}
+ steps:
+ - uses: actions/checkout@v4
+ with:
+ persist-credentials: false
+ - name: Select provisioned pool or wait for the exact PR candidate
+ id: select
+ env:
+ GH_TOKEN: ${{ github.token }}
+ run: python3 .github/scripts/runner-image.py select --kind kotlin
+
kotlin-sdk-build:
name: Kotlin SDK build + tests (x86_64 emulator)
- runs-on: [self-hosted, kotlin-ci]
+ needs: select-runner
+ runs-on: ${{ fromJSON(needs.select-runner.outputs.labels) }}
# Fork PRs must not execute on a persistent runner. Keep this guard in sync
# with tests-rs-workspace.yml.
if: >-
@@ -50,83 +80,90 @@ jobs:
# Preserve target/ and other untracked build outputs between runs.
clean: false
- - name: Ensure runner dependencies
+ # The persistent runner image is the versioned CI environment. Keep
+ # dependency installation out of the job: it required sudo and made a
+ # CI job capable of mutating its container. Rebuild the runner image when
+ # one of these requirements changes instead.
+ - name: Verify exact runner requirements and load versions
+ run: python3 .github/scripts/runner-image.py verify
+
+ - name: Verify runner image dependencies
run: |
set -euo pipefail
- MISSING=()
- for pkg in build-essential cmake curl jq libgmp-dev libpulse0 libssl-dev libx11-xcb1 pkg-config python3 unzip zip; do
- dpkg -s "$pkg" >/dev/null 2>&1 || MISSING+=("$pkg")
+ missing=()
+ for tool in cmake curl jq python3 rustup unzip zip; do
+ command -v "$tool" >/dev/null 2>&1 || missing+=("$tool")
done
- if [ ${#MISSING[@]} -gt 0 ]; then
- echo "Installing: ${MISSING[*]}"
- sudo apt-get update -qq
- sudo apt-get install -qq --yes "${MISSING[@]}"
- fi
-
- if [ ! -x "$HOME/.cargo/bin/rustup" ]; then
- curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs \
- | sh -s -- -y --no-modify-path --default-toolchain none
+ for pkg in build-essential libgmp-dev libpulse0 libssl-dev libx11-xcb1 pkg-config; do
+ dpkg -s "$pkg" >/dev/null 2>&1 || missing+=("$pkg")
+ done
+ if [ ${#missing[@]} -gt 0 ]; then
+ echo "::error::Self-hosted runner image is missing: ${missing[*]}"
+ echo "::error::Provision these in the pinned runner image; CI jobs must not install host packages."
+ exit 1
fi
- echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
- name: Validate executable SDK parity manifest
run: |
python3 scripts/check_sdk_parity_manifest.py
python3 -m unittest discover -s scripts/tests -p 'test_*.py'
- - name: Verify JDK 17
+ - name: Verify repository-pinned JDK
run: |
JAVA_HOME_RESOLVED=$(dirname "$(dirname "$(readlink -f "$(command -v java)")")")
JAVA_VERSION_OUTPUT=$("$JAVA_HOME_RESOLVED/bin/java" -version 2>&1)
printf '%s\n' "$JAVA_VERSION_OUTPUT"
- printf '%s\n' "$JAVA_VERSION_OUTPUT" | grep -Eq 'version "17([.]|\")'
+ printf '%s\n' "$JAVA_VERSION_OUTPUT" | grep -E "version \"${CI_JAVA_MAJOR}([.]|\")"
echo "JAVA_HOME=$JAVA_HOME_RESOLVED" >> "$GITHUB_ENV"
echo "$JAVA_HOME_RESOLVED/bin" >> "$GITHUB_PATH"
- - name: Set up Android SDK
- uses: android-actions/setup-android@v3
- with:
- packages: >-
- platforms;android-35
- build-tools;35.0.0
- ndk;28.1.13356709
- platform-tools
-
- - name: Set up Rust toolchain
- uses: dtolnay/rust-toolchain@stable
+ - name: Verify prebaked Android SDK
+ run: |
+ set -euo pipefail
+ test "${ANDROID_HOME:-}" = /opt/android-sdk
+ test -x "$ANDROID_HOME/ndk/$CI_ANDROID_NDK/ndk-build"
+ test -x "$ANDROID_HOME/build-tools/$CI_ANDROID_BUILD_TOOLS/aapt2"
+ test -f "$ANDROID_HOME/platforms/android-$CI_ANDROID_API/android.jar"
+ test -f "$ANDROID_HOME/system-images/android-$CI_ANDROID_API/default/x86_64/system.img"
+ command -v ci-android-emulator
+ command -v adb
+ # Do not run setup-android/sdkmanager: the SDK is pinned and root-owned.
+
+ - name: Set up repository-pinned Rust toolchain
+ uses: ./.github/actions/rust
with:
- targets: x86_64-linux-android
+ target: x86_64-linux-android
+ cache: false
- # Pinned: this runner is persistent, so an unpinned `cargo install`
- # leaves whatever version happened to be current on the day it first ran,
- # and every later job silently builds with it. Assert after installing so
- # a drifted host fails here instead of somewhere in the NDK build.
- - name: Ensure cargo-ndk v4.1.2 is installed
+ # Pinned: this runner is persistent, so cargo-ndk is provisioned in the
+ # image and checked here before the NDK build can start.
+ - name: Verify repository-pinned cargo-ndk
run: |
set -euo pipefail
- if ! cargo ndk --version 2>/dev/null | grep -qx 'cargo-ndk 4.1.2'; then
- cargo install cargo-ndk --version 4.1.2 --locked --force
- fi
cargo ndk --version
- cargo ndk --version | grep -qx 'cargo-ndk 4.1.2'
+ cargo ndk --version | grep -Fx "cargo-ndk $CI_CARGO_NDK_VERSION"
- - name: Ensure protoc v32.0 is installed (repo-standard; apt's 3.21 breaks tenderdash-proto)
+ - name: Verify repository-pinned protoc
run: |
set -euo pipefail
- if ! protoc --version 2>/dev/null | grep -qx 'libprotoc 32.0'; then
- curl -fsSL -o /tmp/protoc.zip https://github.com/protocolbuffers/protobuf/releases/download/v32.0/protoc-32.0-linux-x86_64.zip
- sudo unzip -o /tmp/protoc.zip -d /usr/local 'bin/protoc' 'include/*'
- fi
protoc --version
- # A stale protoc earlier on PATH would shadow the one just unpacked
- # into /usr/local; catch that here rather than in a codegen failure.
- protoc --version | grep -qx 'libprotoc 32.0'
+ # protoc is part of the runner image. Installing it into /usr/local
+ # from a job required sudo and made the CI container mutable.
+ protoc --version | grep -Fx "libprotoc $CI_PROTOC_VERSION"
+
+ # KVM is the only host device this job needs. The container receives it
+ # through the kvm group; no Docker socket or host package-management
+ # access is required.
+ - name: Verify KVM access
+ run: |
+ test -c /dev/kvm
+ test -r /dev/kvm
+ test -w /dev/kvm
+ ls -l /dev/kvm
- name: Build native library (x86_64, dev profile)
working-directory: packages/kotlin-sdk
- env:
- ANDROID_NDK_HOME: ${{ env.ANDROID_SDK_ROOT }}/ndk/28.1.13356709
run: ./build_android.sh --abi x86_64 --profile dev --verify
- name: Setup Gradle
@@ -144,79 +181,9 @@ jobs:
working-directory: packages/kotlin-sdk
run: ./gradlew :sdk:compileDebugAndroidTestKotlin --stacktrace
- - name: Verify KVM access
- run: |
- test -c /dev/kvm
- test -r /dev/kvm
- test -w /dev/kvm
- ls -l /dev/kvm
-
- - name: Align Android emulator configuration paths
- run: |
- # android-emulator-runner puts AVD data in $HOME/.android/avd.
- # An inherited user/emulator-home override can make avdmanager
- # write the .ini elsewhere, leaving the emulator unable to find it.
- mkdir -p "$HOME/.android/avd"
- {
- printf 'ANDROID_USER_HOME=%s/.android\n' "$HOME"
- printf 'ANDROID_EMULATOR_HOME=%s/.android\n' "$HOME"
- printf 'ANDROID_SDK_HOME=%s\n' "$HOME"
- } >> "$GITHUB_ENV"
-
- - name: Run instrumented FFI smoke test (API 35 emulator)
- uses: reactivecircus/android-emulator-runner@v2
- with:
- api-level: 35
- arch: x86_64
- profile: pixel_6
- # Avoid sharing the default "test" AVD with other jobs on this host.
- avd-name: platform-${{ github.run_id }}-${{ github.run_attempt }}
- disable-animations: true
- emulator-options: -no-snapshot -no-window -no-audio -no-boot-anim -camera-back none -dns-server 8.8.8.8,1.1.1.1
- working-directory: packages/kotlin-sdk
- script: |
- # NOTE: android-emulator-runner executes this script line by line —
- # each line is its own `sh -c` with no shared state — so every
- # command must be self-contained on a single line (multi-line loops
- # or cross-line variables silently fail).
-
- # Keep the screen on and never time out. THIS is the fix: the flake
- # was the emulator screen turning off mid-run and RE-LOCKING the
- # device. The mnemonic MASTER_ALIAS AES key is
- # setUnlockedDeviceRequired(true), so once the device re-locks even
- # an ENCRYPT throws `InvalidKeyException: Keystore operation failed`
- # (see WalletStorage.storeMnemonic). Unlocking alone is not enough —
- # a slower suite (more instrumented tests) crosses the screen-off
- # deadline before its wallet tests run, which is why this branch
- # failed where lighter ones passed. Prevent the re-lock outright.
- adb shell settings put system screen_off_timeout 2147483647
- adb shell svc power stayon true
-
- # Enroll a device-wide secure lock screen (PIN) — Android Keystore
- # refuses to generate an auth-required key
- # (setUserAuthenticationRequired(true), used for the identity-key
- # KEYS_ALIAS RSA pair) without one enrolled, even though nothing
- # here ever prompts for it (private-key ENCRYPT is never auth-gated;
- # only DECRYPT is).
- adb shell locksettings set-pin 1234
-
- # Unlock the keyguard. With the screen kept on above, the device now
- # stays unlocked for the whole run instead of re-locking.
- # Credential acceptance and keyguard dismissal can race during a
- # cold emulator boot. Retry the complete sequence atomically, then
- # fail loudly before tests if the device never reaches unlocked.
- for attempt in 1 2 3; do adb shell input keyevent KEYCODE_WAKEUP; adb shell wm dismiss-keyguard; adb shell input text 1234; adb shell input keyevent KEYCODE_ENTER; sleep 2; adb shell wm dismiss-keyguard; sleep 1; adb shell dumpsys trust | grep -q 'deviceLocked=0' && exit 0; done; echo "::error::Emulator is still locked (deviceLocked=1); Keystore-backed tests would fail spuriously."; adb shell dumpsys trust; exit 1
-
- ./gradlew :sdk:connectedDebugAndroidTest --stacktrace
-
- - name: Remove this run's emulator device
- if: always()
- env:
- CI_AVD_NAME: platform-${{ github.run_id }}-${{ github.run_attempt }}
- run: |
- if [ -f "$HOME/.android/avd/$CI_AVD_NAME.ini" ]; then
- avdmanager delete avd --name "$CI_AVD_NAME"
- fi
+ - name: Run instrumented FFI smoke test (prebaked emulator)
+ working-directory: packages/kotlin-sdk
+ run: ci-android-emulator bash ../../.github/scripts/kotlin-instrumented-tests.sh
- name: Upload test reports on failure
if: failure()
diff --git a/.github/workflows/kotlin-sdk-nightly.yml b/.github/workflows/kotlin-sdk-nightly.yml
index c53510ddc11..9be76995c46 100644
--- a/.github/workflows/kotlin-sdk-nightly.yml
+++ b/.github/workflows/kotlin-sdk-nightly.yml
@@ -15,8 +15,8 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v4
- with:
- ref: v4.1-dev
+ # Scheduled runs test the default branch; dispatches test their selected
+ # ref. Do not silently compile an old release branch with newer CI.
- name: Free disk space
run: |
@@ -54,8 +54,8 @@ jobs:
restore-keys: |
kotlin-sdk-cargo-
- - name: Install cargo-ndk
- run: cargo install cargo-ndk --locked
+ - name: Install pinned cargo-ndk v4.1.2 (ephemeral runner)
+ run: cargo install cargo-ndk --version 4.1.2 --locked
- name: Install protoc v32.0 (repo-standard; apt's 3.21 breaks tenderdash-proto)
run: |
@@ -90,7 +90,9 @@ jobs:
working-directory: packages/kotlin-sdk
# -Ptestnet=true lifts the TestnetGuard so the live-network
# queries (identity fetch, DPNS resolve, contract fetch) run.
- script: ./gradlew :sdk:connectedDebugAndroidTest -Ptestnet=true --stacktrace
+ # Use the same secure-lockscreen setup as daytime instrumented tests.
+ # Keystore authentication-required keys cannot be created otherwise.
+ script: bash ../../.github/scripts/kotlin-instrumented-tests.sh -Ptestnet=true
- name: Upload reports on failure
if: failure()
diff --git a/.github/workflows/npm-runner-validation.yml b/.github/workflows/npm-runner-validation.yml
new file mode 100644
index 00000000000..b21412df94a
--- /dev/null
+++ b/.github/workflows/npm-runner-validation.yml
@@ -0,0 +1,75 @@
+name: Validate NPM runner image
+on:
+ pull_request:
+ paths:
+ - '.github/runner-requirements.json'
+ - '.github/scripts/runner-image.py'
+ - '.github/scripts/tests/**'
+ - '.github/actions/npm-release-build/**'
+ - '.github/actions/rust/**'
+ - '.github/actions/nodejs/**'
+ - '.github/workflows/npm-runner-validation.yml'
+ - '.github/workflows/release.yml'
+ - '.github/workflows/release-npm-build.yml'
+ - '.github/workflows/release-kotlin-sdk.yml'
+ - 'packages/dapi-grpc/**'
+ workflow_dispatch:
+permissions:
+ contents: read
+jobs:
+ runner-contract-tests:
+ runs-on: ubuntu-24.04
+ timeout-minutes: 5
+ steps:
+ - uses: actions/checkout@v4
+ with:
+ persist-credentials: false
+ - name: Test runner selection and release caller restrictions
+ run: python3 -m unittest discover -s .github/scripts/tests -v
+ select-runner:
+ if: >-
+ github.event_name != 'pull_request'
+ || github.event.pull_request.head.repo.full_name == github.repository
+ || github.event.pull_request.head.repo.owner.login == 'thepastaclaw'
+ runs-on: ubuntu-24.04
+ timeout-minutes: 135
+ permissions:
+ contents: read
+ pull-requests: read
+ statuses: read
+ actions: read
+ outputs:
+ labels: ${{ steps.select.outputs.labels }}
+ steps:
+ - uses: actions/checkout@v4
+ with:
+ persist-credentials: false
+ - id: select
+ env:
+ GH_TOKEN: ${{ github.token }}
+ run: python3 .github/scripts/runner-image.py select --kind npm
+ build:
+ name: NPM release build validation
+ needs: select-runner
+ runs-on: ${{ fromJSON(needs.select-runner.outputs.labels) }}
+ timeout-minutes: 120
+ steps:
+ - name: Empty the previous job workspace
+ run: find "$GITHUB_WORKSPACE" -mindepth 1 -maxdepth 1 -exec rm -rf {} +
+ - uses: actions/checkout@v4
+ with:
+ persist-credentials: false
+ - name: Run the release build and pack without publishing
+ uses: ./.github/actions/npm-release-build
+ with:
+ cache-name: npm-validation-target
+ - name: Test the generated clients
+ run: yarn workspace @dashevo/dapi-grpc test:unit
+ - name: Verify the DAPI archive contains generated clients
+ run: python3 packages/dapi-grpc/scripts/check-packed-clients.py npm-packages
+ - uses: actions/upload-artifact@v4
+ with:
+ name: npm-validation-${{ github.sha }}
+ path: npm-packages/*.tgz
+ if-no-files-found: error
+ retention-days: 3
diff --git a/.github/workflows/release-kotlin-sdk.yml b/.github/workflows/release-kotlin-sdk.yml
index 8286d0b371e..7d7308f9fbf 100644
--- a/.github/workflows/release-kotlin-sdk.yml
+++ b/.github/workflows/release-kotlin-sdk.yml
@@ -19,12 +19,22 @@ name: Release Kotlin SDK
on:
workflow_call:
inputs:
+ dry_run:
+ description: 'Build and upload CI artifacts only; never attach or publish'
+ type: boolean
+ required: false
+ default: false
tag:
description: 'Platform release tag (vX.Y.Z[-pre.N])'
required: true
type: string
workflow_dispatch:
inputs:
+ dry_run:
+ description: 'Build and upload CI artifacts only; never attach or publish'
+ type: boolean
+ required: false
+ default: false
tag:
description: >-
Existing platform release tag to (re-)release the Kotlin SDK for.
@@ -37,10 +47,19 @@ on:
jobs:
build-and-release:
name: Build release AAR (arm64-v8a + x86_64)
- runs-on: ubuntu-24.04
+ # Fresh runner, registration, HOME and workspace for exactly one job.
+ # Publication remains hosted; no publishing credentials enter this pool.
+ if: >-
+ github.repository == 'dashpay/platform'
+ && (github.event_name == 'release'
+ || (github.event_name == 'workflow_dispatch'
+ && (github.ref_protected || startsWith(github.ref, 'refs/tags/'))))
+ runs-on:
+ group: platform-release-builds
+ labels: [self-hosted, Linux, X64, 'platform-release-${{ github.run_id }}-${{ github.run_attempt }}-kotlin']
timeout-minutes: 180
permissions:
- contents: write # attach the AAR to the platform release
+ contents: read # release attachment runs on an ephemeral hosted job
# Serialize same-tag runs (e.g. an emergency dispatch racing the
# release-triggered run) so the asset-exists guard cannot be bypassed by
# two concurrent builds. Never cancel a release build in progress.
@@ -58,6 +77,26 @@ jobs:
sha: ${{ steps.resolve-sha.outputs.sha }}
steps:
+ - name: Verify disposable release runner
+ shell: bash
+ run: |
+ set -euo pipefail
+ test "${DASH_RELEASE_RUNNER:-}" = 1
+ test "${DASH_RELEASE_RUN_ID:-}" = "$GITHUB_RUN_ID"
+ test "${DASH_RELEASE_RUN_ATTEMPT:-}" = "$GITHUB_RUN_ATTEMPT"
+ test "${DASH_RELEASE_KIND:-}" = kotlin
+
+ # Verify prebaked native tools. A release job has no sudo or Docker.
+ - name: Verify runner dependencies
+ run: |
+ set -euo pipefail
+
+ for pkg in build-essential cmake curl gh jq libgmp-dev libpulse0 libssl-dev libx11-xcb1 pkg-config python3 unzip zip; do
+ dpkg-query -W -f='${Status}' "$pkg" | grep -Fx 'install ok installed'
+ done
+ test -x "$HOME/.cargo/bin/rustup"
+ echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
+
# A workflow_dispatch `tag` input is free-form and actions/checkout would
# happily resolve it to a BRANCH (or any ref). Normalize and validate it
# here — reject anything that is not an existing platform release tag
@@ -117,6 +156,7 @@ jobs:
} >> "$GITHUB_OUTPUT"
echo "Releasing Kotlin SDK ${VERSION} for platform release ${TAG}"
+ # No previous job's Git configuration, hooks or build state exists here.
- name: Checkout repository
uses: actions/checkout@v4
with:
@@ -124,154 +164,175 @@ jobs:
# raw dispatch input — so the released AAR is built from the tag's
# commit and a manual run can never build from a branch.
ref: ${{ steps.release-ref.outputs.checkout_ref }}
+ persist-credentials: false
- name: Resolve built commit SHA
id: resolve-sha
run: echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
- - name: Free disk space
- run: |
- sudo rm -rf /usr/share/dotnet /usr/local/lib/android/sdk/ndk /opt/ghc
- df -h /
+ - name: Verify release image contract
+ run: ci-image-contract verify .github/runner-requirements.json
- - name: Set up JDK 17
- uses: actions/setup-java@v4
- with:
- distribution: temurin
- java-version: '17'
+ - name: Verify JDK 17
+ run: |
+ JAVA_HOME_RESOLVED=$(dirname "$(dirname "$(readlink -f "$(command -v java)")")")
+ JAVA_VERSION_OUTPUT=$("$JAVA_HOME_RESOLVED/bin/java" -version 2>&1)
+ printf '%s\n' "$JAVA_VERSION_OUTPUT"
+ printf '%s\n' "$JAVA_VERSION_OUTPUT" | grep -Eq 'version "17([.]|\")'
+ echo "JAVA_HOME=$JAVA_HOME_RESOLVED" >> "$GITHUB_ENV"
+ echo "$JAVA_HOME_RESOLVED/bin" >> "$GITHUB_PATH"
- - name: Set up Android SDK
- uses: android-actions/setup-android@v3
- with:
- packages: >-
- platforms;android-35
- build-tools;35.0.0
- ndk;28.1.13356709
+ - name: Verify prebaked Android SDK
+ run: |
+ test -f "$ANDROID_SDK_ROOT/platforms/android-35/android.jar"
+ test -x "$ANDROID_SDK_ROOT/build-tools/35.0.0/aapt2"
+ test -x "$ANDROID_SDK_ROOT/ndk/28.1.13356709/toolchains/llvm/prebuilt/linux-x86_64/bin/clang"
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@stable
with:
targets: aarch64-linux-android,x86_64-linux-android
- - name: Restore cargo cache
- uses: actions/cache@v4
+ # Job-local only: the host destroys HOME and this cache with the runner.
+ - name: Prepare release Cargo target cache
+ uses: ./.github/actions/release-cargo-target-cache
with:
- path: |
- ~/.cargo/registry
- ~/.cargo/git
- target
- key: kotlin-sdk-release-cargo-${{ hashFiles('**/Cargo.lock') }}
- restore-keys: |
- kotlin-sdk-release-cargo-
- kotlin-sdk-cargo-
-
- - name: Install cargo-ndk
- run: cargo install cargo-ndk --locked
-
- - name: Install protoc v32.0 (repo-standard; apt's 3.21 breaks tenderdash-proto)
+ name: release-kotlin-sdk-target
+
+ # Always reinstalled, never trusted from an earlier job: a binary left in
+ # ~/.cargo/bin can print the pinned version and still be something else.
+ # Cargo looks up `cargo ndk` in $CARGO_HOME/bin before PATH, so the
+ # install has to overwrite that copy rather than land elsewhere.
+ - name: Install cargo-ndk v4.1.2
run: |
- curl -fsSL -o /tmp/protoc.zip https://github.com/protocolbuffers/protobuf/releases/download/v32.0/protoc-32.0-linux-x86_64.zip
- sudo unzip -o /tmp/protoc.zip -d /usr/local 'bin/protoc' 'include/*'
- protoc --version
+ set -euo pipefail
+ cargo install cargo-ndk --version 4.1.2 --locked --force
+ cargo ndk --version | grep -qx 'cargo-ndk 4.1.2'
+
+ # Fresh, checksum-verified copy in this job's temp directory rather than
+ # whatever an earlier job left in /usr/local (repo-standard v32.0; apt's
+ # 3.21 breaks tenderdash-proto). prost-build reads PROTOC first.
+ - name: Install protoc v32.0
+ env:
+ PROTOC_SHA256: 7ca037bfe5e5cabd4255ccd21dd265f79eb82d3c010117994f5dc81d2140ee88
+ run: |
+ set -euo pipefail
+ PROTOC_DIR="$RUNNER_TEMP/protoc-32.0"
+ curl -fsSL -o "$RUNNER_TEMP/protoc.zip" \
+ https://github.com/protocolbuffers/protobuf/releases/download/v32.0/protoc-32.0-linux-x86_64.zip
+ echo "$PROTOC_SHA256 $RUNNER_TEMP/protoc.zip" | sha256sum -c -
+ rm -rf "$PROTOC_DIR"
+ unzip -q "$RUNNER_TEMP/protoc.zip" -d "$PROTOC_DIR"
+ echo "PROTOC=$PROTOC_DIR/bin/protoc" >> "$GITHUB_ENV"
+ echo "$PROTOC_DIR/bin" >> "$GITHUB_PATH"
+ "$PROTOC_DIR/bin/protoc" --version
- name: Build native library (both ABIs, release profile)
working-directory: packages/kotlin-sdk
- env:
- ANDROID_NDK_HOME: ${{ env.ANDROID_SDK_ROOT }}/ndk/28.1.13356709
- run: ./build_android.sh --abi all --profile release --verify
+ # Image-provided variables exist in the shell, not the Actions env context.
+ run: |
+ export ANDROID_NDK_HOME="${ANDROID_SDK_ROOT:?}/ndk/28.1.13356709"
+ ./build_android.sh --abi all --profile release --verify
- name: Setup Gradle
uses: gradle/actions/setup-gradle@v4
+ with:
+ cache-disabled: true
- name: Build release AAR
working-directory: packages/kotlin-sdk
run: ./gradlew :sdk:assembleRelease --stacktrace
- # Defenses against a force-moved tag splitting the GitHub-release AAR
- # from the (immutable) Maven Central artifact of the same version:
- # 1. Hard-fail if the tag no longer resolves to the exact commit this
- # run built.
- # 2. The AAR is attached together with a .commit.txt recording
- # the commit it was built from. When both already exist, this run
- # may proceed to the Maven deploy ONLY if that recorded commit
- # equals the commit this run built — otherwise the existing GitHub
- # AAR and the Maven artifact this run would publish came from
- # different commits, and the run hard-fails instead of letting the
- # two channels drift. An attached AAR is never overwritten; a
- # deliberate re-release must delete both assets first.
- - name: Verify tag still resolves to the built commit
- id: tag-guard
+ - name: Prepare versioned assets (AAR + build provenance)
env:
- TAG: ${{ steps.release-ref.outputs.tag_name }}
- AAR_ASSET: dash-sdk-android-${{ steps.release-ref.outputs.version }}.aar
- COMMIT_ASSET: dash-sdk-android-${{ steps.release-ref.outputs.version }}.commit.txt
+ VERSION: ${{ steps.release-ref.outputs.version }}
BUILT_SHA: ${{ steps.resolve-sha.outputs.sha }}
+ run: |
+ cd packages/kotlin-sdk/sdk/build/outputs/aar
+ cp sdk-release.aar "dash-sdk-android-${VERSION}.aar"
+ printf '%s\n' "$BUILT_SHA" > "dash-sdk-android-${VERSION}.commit.txt"
+
+ # Release attachment runs on a fresh hosted runner so release-write
+ # credentials never reach the disposable build runner.
+ - name: Upload release assets
+ uses: actions/upload-artifact@v4
+ with:
+ name: kotlin-sdk-release-assets
+ path: |
+ packages/kotlin-sdk/sdk/build/outputs/aar/dash-sdk-android-${{ steps.release-ref.outputs.version }}.aar
+ packages/kotlin-sdk/sdk/build/outputs/aar/dash-sdk-android-${{ steps.release-ref.outputs.version }}.commit.txt
+ if-no-files-found: error
+ retention-days: 7
+
+ # Hand the freshly built native libraries to the deploy job so it
+ # re-stages the SAME .so (same commit, same tag) without repeating the
+ # multi-hour cargo/NDK build.
+ - name: Upload native libraries for the deploy job
+ uses: actions/upload-artifact@v4
+ with:
+ name: kotlin-sdk-jnilibs
+ path: packages/kotlin-sdk/sdk/src/main/jniLibs
+ if-no-files-found: error
+ retention-days: 7
+
+ attach-release:
+ name: Attach AAR to platform release
+ needs: build-and-release
+ if: ${{ !inputs.dry_run }}
+ runs-on: ubuntu-24.04
+ timeout-minutes: 15
+ permissions:
+ contents: write
+ concurrency:
+ group: release-kotlin-sdk-attach-${{ needs.build-and-release.outputs.tag_name }}
+ cancel-in-progress: false
+ steps:
+ - name: Download release assets
+ uses: actions/download-artifact@v4
+ with:
+ name: kotlin-sdk-release-assets
+ path: release-assets
+
+ - name: Verify tag and existing assets
+ id: verify
+ env:
+ TAG: ${{ needs.build-and-release.outputs.tag_name }}
+ AAR_ASSET: dash-sdk-android-${{ needs.build-and-release.outputs.version }}.aar
+ COMMIT_ASSET: dash-sdk-android-${{ needs.build-and-release.outputs.version }}.commit.txt
+ BUILT_SHA: ${{ needs.build-and-release.outputs.sha }}
REPO: ${{ github.repository }}
GH_TOKEN: ${{ github.token }}
run: |
- # Resolve the tag's current commit, dereferencing an annotated tag
- # to the commit it points at.
+ set -euo pipefail
OBJ_TYPE=$(gh api "repos/${REPO}/git/ref/tags/${TAG}" --jq '.object.type')
OBJ_SHA=$(gh api "repos/${REPO}/git/ref/tags/${TAG}" --jq '.object.sha')
if [ "$OBJ_TYPE" = "tag" ]; then
OBJ_SHA=$(gh api "repos/${REPO}/git/tags/${OBJ_SHA}" --jq '.object.sha')
fi
- if [ "$OBJ_SHA" != "$BUILT_SHA" ]; then
- echo "::error::Tag ${TAG} now points at ${OBJ_SHA} but this run built ${BUILT_SHA} — the tag was force-moved. Refusing to attach/deploy."
- exit 1
- fi
+ [ "$OBJ_SHA" = "$BUILT_SHA" ] || { echo "::error::Tag ${TAG} moved from built commit ${BUILT_SHA} to ${OBJ_SHA}."; exit 1; }
ASSETS=$(gh api "repos/${REPO}/releases/tags/${TAG}" --jq '.assets[].name')
HAVE_AAR=$(echo "$ASSETS" | grep -Fxc "$AAR_ASSET" || true)
HAVE_COMMIT=$(echo "$ASSETS" | grep -Fxc "$COMMIT_ASSET" || true)
- if [ "$HAVE_AAR" = "1" ] && [ "$HAVE_COMMIT" = "1" ]; then
+ if [ "$HAVE_AAR" = 1 ] && [ "$HAVE_COMMIT" = 1 ]; then
RECORDED_SHA=$(gh release download "$TAG" --repo "$REPO" --pattern "$COMMIT_ASSET" --output - | tr -d '[:space:]')
- if [ "$RECORDED_SHA" != "$BUILT_SHA" ]; then
- echo "::error::Release ${TAG} carries ${AAR_ASSET} built from ${RECORDED_SHA}, but this run built ${BUILT_SHA} — deploying would publish a different commit to Maven Central than the attached AAR. Delete both assets deliberately to re-release."
- exit 1
- fi
+ [ "$RECORDED_SHA" = "$BUILT_SHA" ] || { echo "::error::Existing AAR provenance does not match this build."; exit 1; }
echo "asset_exists=true" >> "$GITHUB_OUTPUT"
- echo "::notice::Release ${TAG} already has ${AAR_ASSET} built from this same commit — leaving it untouched; the Maven deploy may proceed."
- elif [ "$HAVE_AAR" = "1" ] || [ "$HAVE_COMMIT" = "1" ]; then
- echo "::error::Release ${TAG} has only one of ${AAR_ASSET} / ${COMMIT_ASSET} — an earlier attach was interrupted, so the existing asset's provenance cannot be verified. Delete the surviving asset and re-run."
+ elif [ "$HAVE_AAR" = 1 ] || [ "$HAVE_COMMIT" = 1 ]; then
+ echo "::error::Release has only one provenance asset; delete it before retrying."
exit 1
else
echo "asset_exists=false" >> "$GITHUB_OUTPUT"
fi
- - name: Prepare versioned assets (AAR + build provenance)
- env:
- VERSION: ${{ steps.release-ref.outputs.version }}
- BUILT_SHA: ${{ steps.resolve-sha.outputs.sha }}
- run: |
- cd packages/kotlin-sdk/sdk/build/outputs/aar
- cp sdk-release.aar "dash-sdk-android-${VERSION}.aar"
- printf '%s\n' "$BUILT_SHA" > "dash-sdk-android-${VERSION}.commit.txt"
-
- # Attach to the PLATFORM release: only tag_name + files. Passing name/
- # body/prerelease/generate_release_notes here would overwrite the
- # platform release's own notes, title or prerelease flag.
- # overwrite_files: false makes a same-name upload race fail loudly
- # instead of silently replacing an asset the guard above vouched for.
- name: Attach AAR to the platform release
- if: steps.tag-guard.outputs.asset_exists != 'true'
+ if: steps.verify.outputs.asset_exists != 'true'
uses: softprops/action-gh-release@3bb12739c298aeb8a4eeaf626c5b8d85266b0e65 # v2
with:
- tag_name: ${{ steps.release-ref.outputs.tag_name }}
+ tag_name: ${{ needs.build-and-release.outputs.tag_name }}
overwrite_files: false
files: |
- packages/kotlin-sdk/sdk/build/outputs/aar/dash-sdk-android-${{ steps.release-ref.outputs.version }}.aar
- packages/kotlin-sdk/sdk/build/outputs/aar/dash-sdk-android-${{ steps.release-ref.outputs.version }}.commit.txt
-
- # Hand the freshly built native libraries to the deploy job so it
- # re-stages the SAME .so (same commit, same tag) without repeating the
- # multi-hour cargo/NDK build.
- - name: Upload native libraries for the deploy job
- uses: actions/upload-artifact@v4
- with:
- name: kotlin-sdk-jnilibs
- path: packages/kotlin-sdk/sdk/src/main/jniLibs
- if-no-files-found: error
- retention-days: 7
+ release-assets/dash-sdk-android-${{ needs.build-and-release.outputs.version }}.aar
+ release-assets/dash-sdk-android-${{ needs.build-and-release.outputs.version }}.commit.txt
# ---------------------------------------------------------------------------
# Maven Central deploy. Declares `environment: maven-central`, so the five
@@ -284,7 +345,7 @@ jobs:
# ---------------------------------------------------------------------------
maven-central-deploy:
name: Deploy to Maven Central (version from tag)
- needs: build-and-release
+ needs: [build-and-release, attach-release]
# Runs only when the RUN's ref is exactly the target tag — not merely
# some tag: binding the ref to inputs.tag stops a dispatch started at tag
# A from publishing input tag B under the environment authorization A's
@@ -294,7 +355,7 @@ jobs:
# selected ref) skips this job cleanly instead of failing at the
# environment policy; Maven for such tags goes through the manual
# runbook in packages/kotlin-sdk/PUBLISHING.md.
- if: ${{ github.ref == format('refs/tags/{0}', inputs.tag) }}
+ if: ${{ !inputs.dry_run && needs.build-and-release.result == 'success' && needs.attach-release.result == 'success' && github.ref == format('refs/tags/{0}', inputs.tag) }}
runs-on: ubuntu-24.04
timeout-minutes: 60
permissions:
@@ -401,6 +462,8 @@ jobs:
- name: Setup Gradle
if: steps.secrets-gate.outputs.proceed == 'true' && steps.maven-check.outputs.already_published != 'true'
uses: gradle/actions/setup-gradle@v4
+ with:
+ cache-disabled: true
# Stages the signed release AAR + sources/javadoc jars into
# sdk/build/staging-deploy. Task dependencies enforce the publish
diff --git a/.github/workflows/release-npm-build.yml b/.github/workflows/release-npm-build.yml
new file mode 100644
index 00000000000..174b160ec46
--- /dev/null
+++ b/.github/workflows/release-npm-build.yml
@@ -0,0 +1,81 @@
+name: Build release NPM artifacts
+
+# The host controller allocates a fresh one-job runner for this run/attempt.
+# No persistent CI runner carries this label or shares its writable state.
+on:
+ workflow_call:
+
+permissions:
+ contents: read
+
+jobs:
+ build:
+ name: Build NPM packages
+ if: >-
+ github.repository == 'dashpay/platform'
+ && (github.event_name == 'release'
+ || (github.event_name == 'workflow_dispatch'
+ && (github.ref_protected || startsWith(github.ref, 'refs/tags/'))))
+ runs-on:
+ group: platform-release-builds
+ labels: [self-hosted, Linux, X64, 'platform-release-${{ github.run_id }}-${{ github.run_attempt }}-npm']
+ timeout-minutes: 120
+ steps:
+ - name: Verify disposable release runner
+ shell: bash
+ run: |
+ set -euo pipefail
+ test "${DASH_RELEASE_RUNNER:-}" = 1
+ test "${DASH_RELEASE_RUN_ID:-}" = "$GITHUB_RUN_ID"
+ test "${DASH_RELEASE_RUN_ATTEMPT:-}" = "$GITHUB_RUN_ATTEMPT"
+ test "${DASH_RELEASE_KIND:-}" = npm
+
+ - name: Reject untrusted release callers
+ shell: bash
+ run: |
+ set -euo pipefail
+ test "$GITHUB_REPOSITORY" = dashpay/platform
+ case "$GITHUB_EVENT_NAME:$GITHUB_REF" in
+ release:refs/tags/*|workflow_dispatch:refs/tags/*) ;;
+ workflow_dispatch:refs/heads/*) test "${GITHUB_REF_PROTECTED:-}" = true ;;
+ *) echo '::error::NPM release runners accept only release tags or protected-branch dispatches'; exit 1 ;;
+ esac
+
+ - uses: softwareforgood/check-artifact-v4-existence@v0
+ id: check-artifact
+ with:
+ name: js-build-${{ github.sha }}
+
+ # The entire runner, including HOME and Git state, is new for this job.
+ - name: Check out repo
+ uses: actions/checkout@v4
+ with:
+ persist-credentials: false
+ if: ${{ steps.check-artifact.outputs.exists != 'true' }}
+
+ - name: Build and pack NPM packages
+ uses: ./.github/actions/npm-release-build
+ if: ${{ steps.check-artifact.outputs.exists != 'true' }}
+
+ - name: Get modified files
+ id: diff
+ run: |
+ {
+ echo "files<> "$GITHUB_OUTPUT"
+ if: ${{ steps.check-artifact.outputs.exists != 'true' }}
+
+ - name: Upload the archive of built files
+ uses: actions/upload-artifact@v4
+ with:
+ name: js-build-${{ github.sha }}
+ path: |
+ ${{ steps.diff.outputs.files }}
+ npm-packages/*.tgz
+ # Keep the handoff alive long enough to re-run only a failed publish.
+ retention-days: 7
+ if-no-files-found: error
+ include-hidden-files: true
+ if: ${{ steps.check-artifact.outputs.exists != 'true' }}
diff --git a/.github/workflows/release-swift-sdk.yml b/.github/workflows/release-swift-sdk.yml
index 4d6669c244e..dcc62c29872 100644
--- a/.github/workflows/release-swift-sdk.yml
+++ b/.github/workflows/release-swift-sdk.yml
@@ -30,19 +30,44 @@ on:
jobs:
build-and-release:
name: Build and release DashSDKFFI
- runs-on: macos-15
- # Two cold release-profile builds (device + simulator) of ~25 min each.
+ outputs:
+ tag_name: ${{ steps.release-ref.outputs.tag_name }}
+ version: ${{ steps.release-ref.outputs.version }}
+ sha: ${{ steps.resolve-sha.outputs.sha }}
+ checksum: ${{ steps.zip.outputs.checksum }}
+ # Same persistent runner as swift-sdk-build.yml. Release builds keep their
+ # own Cargo target cache on it (see "Prepare release Cargo target cache"),
+ # so later releases only rebuild what changed since the previous one. No
+ # fork PR guard is needed here: the workflow only triggers on release/
+ # workflow_dispatch (via release.yml), never on pull_request.
+ runs-on: [self-hosted, macOS, ARM64]
+ # Three cold fat-LTO slices took over 45 minutes on a hosted macos-15
+ # runner. A cold run here (first release, or after the cache is reset)
+ # needs the same headroom; warm runs finish well inside it.
timeout-minutes: 90
permissions:
- contents: write # attach the xcframework to the platform release
+ contents: read # release attachment runs on an ephemeral hosted job
# Serialize same-tag runs (e.g. an emergency dispatch racing the
# release-triggered run) so the asset-exists guard cannot be bypassed by
# two concurrent builds. Never cancel a release build in progress.
concurrency:
group: release-swift-sdk-${{ inputs.tag }}
cancel-in-progress: false
+ defaults:
+ run:
+ # The runner's work/temp directory can be on a volume with spaces.
+ shell: bash --noprofile --norc -e -o pipefail "{0}"
steps:
+ # The tag validation below needs gh before checkout; hosted images ship
+ # it, the persistent runner may not.
+ - name: Ensure gh is installed
+ run: |
+ if ! command -v gh >/dev/null 2>&1; then
+ brew install gh
+ fi
+ gh --version
+
# Same guard as release-kotlin-sdk.yml: normalize/validate the tag,
# refuse anything that is not an existing platform release tag with a
# published GitHub release, and hand checkout an explicit refs/tags/
@@ -90,50 +115,107 @@ jobs:
} >> "$GITHUB_OUTPUT"
echo "Releasing DashSDKFFI ${TAG#v} for platform release ${TAG}"
+ # PR jobs run in this same workspace on this persistent runner. Git
+ # state they leave behind (.git/hooks, .git/config, .git/info/attributes)
+ # would run inside actions/checkout's own `git checkout`, before any
+ # later cleanup could remove it. Start from an empty directory and a
+ # fresh clone; the release build cache lives outside the workspace.
+ - name: Empty the workspace left by earlier jobs
+ run: |
+ find "$GITHUB_WORKSPACE" -mindepth 1 -maxdepth 1 -exec rm -rf {} + \
+ || sudo -n find "$GITHUB_WORKSPACE" -mindepth 1 -maxdepth 1 -exec rm -rf {} +
+
- name: Checkout repository
uses: actions/checkout@v4
with:
ref: ${{ steps.release-ref.outputs.checkout_ref }}
+ persist-credentials: false
- name: Resolve built commit SHA
id: resolve-sha
run: echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
- - name: Select Xcode 16
- uses: maxim-lobanov/setup-xcode@v1
- with:
- xcode-version: '16.*'
-
+ # The runner's selected Xcode is the one every other Swift job on this
+ # machine builds with (setup-xcode only knows hosted image layouts).
- name: Show Xcode and Swift versions
run: |
xcodebuild -version
swift --version
- - name: Set up Rust toolchain (stable)
- uses: dtolnay/rust-toolchain@stable
+ # Composite actions choose their own shell and do not inherit the quoted
+ # script path above, so rustup is set up inline as in
+ # swift-sdk-build.yml. The build uses the rust-toolchain.toml channel.
+ - name: Set up Rust toolchain
+ env:
+ RUSTUP_PERMIT_COPY_RENAME: "1"
+ run: |
+ export CARGO_HOME="${CARGO_HOME:-$HOME/.cargo}"
+ export PATH="$CARGO_HOME/bin:$PATH"
+ if ! command -v rustup >/dev/null 2>&1; then
+ curl --proto '=https' --tlsv1.2 --retry 10 --retry-connrefused \
+ --location --silent --show-error --fail https://sh.rustup.rs \
+ | sh -s -- --default-toolchain none --no-modify-path -y
+ fi
+ {
+ echo "CARGO_HOME=$CARGO_HOME"
+ echo "CARGO_INCREMENTAL=${CARGO_INCREMENTAL-0}"
+ echo "CARGO_TERM_COLOR=${CARGO_TERM_COLOR-always}"
+ } >> "$GITHUB_ENV"
+ echo "$CARGO_HOME/bin" >> "$GITHUB_PATH"
- - name: Cache cargo registry
- uses: actions/cache@v5
+ # Restore-only, matching swift-sdk-build.yml: the persistent runner
+ # keeps ~/.cargo between runs, so saving it back would just re-upload
+ # gigabytes on every lockfile change.
+ - name: Restore cargo registry cache
+ uses: actions/cache/restore@v5
with:
path: |
~/.cargo/registry
~/.cargo/git
key: cargo-registry-${{ hashFiles('**/Cargo.lock') }}
+ restore-keys: |
+ cargo-registry-
- name: Add iOS Rust targets
+ env:
+ RUSTUP_PERMIT_COPY_RENAME: "1"
run: |
rustup target add aarch64-apple-ios aarch64-apple-ios-sim
+ rustc --version --verbose
- - name: Install protoc (Protocol Buffers compiler)
- uses: arduino/setup-protoc@v3
+ # Fresh, checksum-verified copy in this job's temp directory. The runner's
+ # tool cache persists between jobs, so a protoc left there by an earlier
+ # job is not trusted for a release build.
+ - name: Install protoc v32.0 (Protocol Buffers compiler)
+ env:
+ PROTOC_SHA256: 09a2c729cc821215cc0d4c564b761760961fe338c52f24b302fd7e18e7b675d1
+ run: |
+ set -euo pipefail
+ PROTOC_DIR="$RUNNER_TEMP/protoc-32.0"
+ curl -fsSL -o "$RUNNER_TEMP/protoc.zip" \
+ https://github.com/protocolbuffers/protobuf/releases/download/v32.0/protoc-32.0-osx-aarch_64.zip
+ echo "$PROTOC_SHA256 $RUNNER_TEMP/protoc.zip" | shasum -a 256 -c -
+ rm -rf "$PROTOC_DIR"
+ unzip -q "$RUNNER_TEMP/protoc.zip" -d "$PROTOC_DIR"
+ echo "PROTOC=$PROTOC_DIR/bin/protoc" >> "$GITHUB_ENV"
+ echo "$PROTOC_DIR/bin" >> "$GITHUB_PATH"
+ "$PROTOC_DIR/bin/protoc" --version
+
+ # PR builds on this runner (swift-sdk-build.yml) set PRUNE_CARGO_TARGETS,
+ # which deletes every Apple target directory under the workspace
+ # target/ before and after each slice, and this job empties the
+ # workspace anyway. Releases keep their own release-ios cache outside
+ # it instead.
+ - name: Prepare release Cargo target cache
+ uses: ./.github/actions/release-cargo-target-cache
with:
- version: '32.x'
- # Without a token the action resolves the protoc release
- # unauthenticated, and the shared macOS-runner egress IP burns
- # through the 60 req/h anonymous limit.
- repo-token: ${{ secrets.GITHUB_TOKEN }}
+ name: release-swift-sdk-target
- name: Build DashSDKFFI.xcframework and install into Swift package
+ env:
+ # Keep all three slices' intermediates in the release cache above;
+ # pruning them would make every release a cold build.
+ PRUNE_CARGO_TARGETS: "0"
run: |
bash packages/swift-sdk/build_ios.sh --target all --profile release
@@ -149,88 +231,86 @@ jobs:
printf '%s\n' "$BUILT_SHA" > "DashSDKFFI-${VERSION}.commit.txt"
echo "checksum=$(cat "DashSDKFFI-${VERSION}.checksum.txt")" >> "$GITHUB_OUTPUT"
- # Same defenses as the Kotlin job: hard-fail if the tag was force-moved
- # away from the commit this run built, and never overwrite an
- # already-attached versioned zip — SwiftPM consumers pin the zip by
- # checksum, so replacing attached bytes would break them. The zip
- # attaches together with checksum + commit provenance; existing assets
- # are reused only when their recorded commit equals the commit this run
- # built, so a force-moved tag cannot leave a green run whose assets
- # came from a different commit. A deliberate re-release must delete all
- # three assets first.
- - name: Verify tag still resolves to the built commit
- id: tag-guard
+ # Release attachment runs on a fresh hosted runner so release-write
+ # credentials never reach the persistent build machine.
+ - name: Upload release assets
+ uses: actions/upload-artifact@v4
+ with:
+ name: swift-sdk-release-assets
+ path: |
+ packages/swift-sdk/DashSDKFFI-${{ steps.release-ref.outputs.version }}.xcframework.zip
+ packages/swift-sdk/DashSDKFFI-${{ steps.release-ref.outputs.version }}.checksum.txt
+ packages/swift-sdk/DashSDKFFI-${{ steps.release-ref.outputs.version }}.commit.txt
+ if-no-files-found: error
+ retention-days: 7
+
+ attach-release:
+ name: Attach XCFramework to platform release
+ needs: build-and-release
+ runs-on: ubuntu-24.04
+ timeout-minutes: 15
+ permissions:
+ contents: write
+ concurrency:
+ group: release-swift-sdk-attach-${{ needs.build-and-release.outputs.tag_name }}
+ cancel-in-progress: false
+ steps:
+ - name: Download release assets
+ uses: actions/download-artifact@v4
+ with:
+ name: swift-sdk-release-assets
+ path: release-assets
+
+ - name: Verify tag and existing assets
+ id: verify
env:
- TAG: ${{ steps.release-ref.outputs.tag_name }}
- ZIP_ASSET: DashSDKFFI-${{ steps.release-ref.outputs.version }}.xcframework.zip
- SUM_ASSET: DashSDKFFI-${{ steps.release-ref.outputs.version }}.checksum.txt
- COMMIT_ASSET: DashSDKFFI-${{ steps.release-ref.outputs.version }}.commit.txt
- BUILT_SHA: ${{ steps.resolve-sha.outputs.sha }}
+ TAG: ${{ needs.build-and-release.outputs.tag_name }}
+ ZIP_ASSET: DashSDKFFI-${{ needs.build-and-release.outputs.version }}.xcframework.zip
+ SUM_ASSET: DashSDKFFI-${{ needs.build-and-release.outputs.version }}.checksum.txt
+ COMMIT_ASSET: DashSDKFFI-${{ needs.build-and-release.outputs.version }}.commit.txt
+ BUILT_SHA: ${{ needs.build-and-release.outputs.sha }}
REPO: ${{ github.repository }}
GH_TOKEN: ${{ github.token }}
run: |
- # Resolve the tag's current commit, dereferencing an annotated tag
- # to the commit it points at.
+ set -euo pipefail
OBJ_TYPE=$(gh api "repos/${REPO}/git/ref/tags/${TAG}" --jq '.object.type')
OBJ_SHA=$(gh api "repos/${REPO}/git/ref/tags/${TAG}" --jq '.object.sha')
if [ "$OBJ_TYPE" = "tag" ]; then
OBJ_SHA=$(gh api "repos/${REPO}/git/tags/${OBJ_SHA}" --jq '.object.sha')
fi
- if [ "$OBJ_SHA" != "$BUILT_SHA" ]; then
- echo "::error::Tag ${TAG} now points at ${OBJ_SHA} but this run built ${BUILT_SHA} — the tag was force-moved. Refusing to attach."
- exit 1
- fi
- # The three assets are attached as one unit: reuse them only when
- # ALL are present AND the recorded provenance commit matches this
- # run's built commit. A partial set means an earlier run died
- # mid-attach (provenance unverifiable); a mismatched commit means
- # the tag was force-moved after the original attach. Both hard-fail
- # and require deliberate cleanup instead of a silently-green run.
+ [ "$OBJ_SHA" = "$BUILT_SHA" ] || { echo "::error::Tag ${TAG} moved from built commit ${BUILT_SHA} to ${OBJ_SHA}."; exit 1; }
ASSETS=$(gh api "repos/${REPO}/releases/tags/${TAG}" --jq '.assets[].name')
HAVE_ZIP=$(echo "$ASSETS" | grep -Fxc "$ZIP_ASSET" || true)
HAVE_SUM=$(echo "$ASSETS" | grep -Fxc "$SUM_ASSET" || true)
HAVE_COMMIT=$(echo "$ASSETS" | grep -Fxc "$COMMIT_ASSET" || true)
- if [ "$HAVE_ZIP" = "1" ] && [ "$HAVE_SUM" = "1" ] && [ "$HAVE_COMMIT" = "1" ]; then
+ if [ "$HAVE_ZIP" = 1 ] && [ "$HAVE_SUM" = 1 ] && [ "$HAVE_COMMIT" = 1 ]; then
RECORDED_SHA=$(gh release download "$TAG" --repo "$REPO" --pattern "$COMMIT_ASSET" --output - | tr -d '[:space:]')
- if [ "$RECORDED_SHA" != "$BUILT_SHA" ]; then
- echo "::error::Release ${TAG} carries ${ZIP_ASSET} built from ${RECORDED_SHA}, but this run built ${BUILT_SHA} — the existing assets came from a different commit. Delete all three assets deliberately to re-release."
- exit 1
- fi
+ [ "$RECORDED_SHA" = "$BUILT_SHA" ] || { echo "::error::Existing XCFramework provenance does not match this build."; exit 1; }
echo "asset_exists=true" >> "$GITHUB_OUTPUT"
- echo "::notice::Release ${TAG} already has ${ZIP_ASSET} (+ checksum/commit) built from this same commit — leaving them untouched."
- elif [ "$HAVE_ZIP" = "1" ] || [ "$HAVE_SUM" = "1" ] || [ "$HAVE_COMMIT" = "1" ]; then
- echo "::error::Release ${TAG} has only some of ${ZIP_ASSET} / ${SUM_ASSET} / ${COMMIT_ASSET} — an earlier attach was interrupted, so the existing assets' provenance cannot be verified. Delete the surviving assets and re-run."
+ elif [ "$HAVE_ZIP" = 1 ] || [ "$HAVE_SUM" = 1 ] || [ "$HAVE_COMMIT" = 1 ]; then
+ echo "::error::Release has only some XCFramework provenance assets; delete them before retrying."
exit 1
else
echo "asset_exists=false" >> "$GITHUB_OUTPUT"
fi
- # Attach to the PLATFORM release: only tag_name + files. Passing name/
- # body/prerelease/generate_release_notes here would overwrite the
- # platform release's own notes, title or prerelease flag.
- # overwrite_files: false makes a same-name upload race fail loudly
- # instead of silently replacing an asset the guard above vouched for.
- name: Attach XCFramework to the platform release
- if: steps.tag-guard.outputs.asset_exists != 'true'
+ if: steps.verify.outputs.asset_exists != 'true'
uses: softprops/action-gh-release@3bb12739c298aeb8a4eeaf626c5b8d85266b0e65 # v2
with:
- tag_name: ${{ steps.release-ref.outputs.tag_name }}
+ tag_name: ${{ needs.build-and-release.outputs.tag_name }}
overwrite_files: false
files: |
- packages/swift-sdk/DashSDKFFI-${{ steps.release-ref.outputs.version }}.xcframework.zip
- packages/swift-sdk/DashSDKFFI-${{ steps.release-ref.outputs.version }}.checksum.txt
- packages/swift-sdk/DashSDKFFI-${{ steps.release-ref.outputs.version }}.commit.txt
+ release-assets/DashSDKFFI-${{ needs.build-and-release.outputs.version }}.xcframework.zip
+ release-assets/DashSDKFFI-${{ needs.build-and-release.outputs.version }}.checksum.txt
+ release-assets/DashSDKFFI-${{ needs.build-and-release.outputs.version }}.commit.txt
- # The durable home for the consumer checksum is the checksum.txt release
- # asset; this summary is a convenience copy. Skipped when the attach was
- # skipped: this run's freshly computed checksum would describe THIS
- # build, not the (not byte-reproducible) zip actually attached earlier.
- name: Write SwiftPM usage to the job summary
- if: steps.tag-guard.outputs.asset_exists != 'true'
+ if: steps.verify.outputs.asset_exists != 'true'
env:
- TAG: ${{ steps.release-ref.outputs.tag_name }}
- VERSION: ${{ steps.release-ref.outputs.version }}
- CHECKSUM: ${{ steps.zip.outputs.checksum }}
+ TAG: ${{ needs.build-and-release.outputs.tag_name }}
+ VERSION: ${{ needs.build-and-release.outputs.version }}
+ CHECKSUM: ${{ needs.build-and-release.outputs.checksum }}
REPO: ${{ github.repository }}
run: |
cat >> "$GITHUB_STEP_SUMMARY" <> "$GITHUB_OUTPUT"
+
+ # Only npm itself is needed here. Do not install dependencies or restore
+ # executable caches shared with the persistent builder in this OIDC job.
+ # No registry-url: with it, setup-node@v4 writes an _authToken line to
+ # .npmrc and exports a placeholder NODE_AUTH_TOKEN, so `npm publish` can
+ # attempt token auth instead of the trusted-publishing OIDC exchange.
+ # npm's default registry is already registry.npmjs.org.
+ - name: Setup Node.JS
+ uses: actions/setup-node@v4
with:
- target: wasm32-unknown-unknown
- if: ${{ steps.check-artifact.outputs.exists != 'true' }}
+ node-version: '24.14.1'
- - name: Setup sccache
- uses: ./.github/actions/sccache
+ - name: Download built package artifacts
+ uses: actions/download-artifact@v4
with:
- bucket: ${{ vars.CACHE_S3_BUCKET }}
- region: ${{ vars.AWS_REGION }}
- endpoint: ${{ vars.CACHE_S3_ENDPOINT }}
- access_key_id: ${{ secrets.CACHE_KEY_ID }}
- secret_access_key: ${{ secrets.CACHE_SECRET_KEY }}
-
- if: ${{ steps.check-artifact.outputs.exists != 'true' }}
-
- # Composite action defaults to Node 24+ which ships npm 11.5.1+;
- # required for trusted-publishers OIDC at the publish step below.
- # See https://docs.npmjs.com/trusted-publishers.
- - name: Setup Node.JS
- uses: ./.github/actions/nodejs
-
- - name: Install Cargo binstall
- uses: cargo-bins/cargo-binstall@v1.3.1
- if: ${{ steps.check-artifact.outputs.exists != 'true' }}
-
- - name: Install wasm-bindgen-cli
- run: cargo binstall wasm-bindgen-cli@0.2.108
- if: ${{ steps.check-artifact.outputs.exists != 'true' }}
-
- - name: Install wasm-pack
- run: cargo binstall wasm-pack
- if: ${{ steps.check-artifact.outputs.exists != 'true' }}
-
- - name: Install Binaryen
- run: |
- wget https://github.com/WebAssembly/binaryen/releases/download/version_121/binaryen-version_121-x86_64-linux.tar.gz -P /tmp
- tar -xzf /tmp/binaryen-version_121-x86_64-linux.tar.gz -C /tmp
- sudo cp -r /tmp/binaryen-version_121/* /usr/local/
- if: ${{ steps.check-artifact.outputs.exists != 'true' }}
+ name: js-build-${{ github.sha }}
+ path: release-artifacts
- - name: Build packages
- run: yarn build
+ # Verify every tarball against the trusted checkout metadata before the
+ # credentialed publish. Tarball contents are treated as data; no package
+ # script from the persistent builder is executed on this runner.
+ - name: Validate packed package identities
env:
- CARGO_BUILD_PROFILE: release
- if: ${{ steps.check-artifact.outputs.exists != 'true' }}
+ ARTIFACT_DIR: release-artifacts/npm-packages
+ run: |
+ set -euo pipefail
+ NPM_CLI_PATH="$(command -v npm)" node <<'NODE'
+ const fs = require('node:fs');
+ const path = require('node:path');
+ // Match npm publish's own manifest parser, including archive path aliases.
+ const npmBin = path.dirname(fs.realpathSync(process.env.NPM_CLI_PATH));
+ const pacote = require(path.join(npmBin, '../node_modules/pacote'));
+ (async () => {
+ const trusted = new Map();
+ for (const workspace of JSON.parse(fs.readFileSync('package.json')).workspaces) {
+ const manifest = JSON.parse(fs.readFileSync(path.join(workspace, 'package.json')));
+ if (!manifest.private) trusted.set(manifest.name, manifest.version);
+ }
+ const tarballs = fs.readdirSync(process.env.ARTIFACT_DIR).filter(file => file.endsWith('.tgz'));
+ if (!tarballs.length) throw new Error('No NPM tarballs were uploaded.');
+ const seen = new Set();
+ for (const file of tarballs) {
+ const tarball = path.resolve(process.env.ARTIFACT_DIR, file);
+ if (!fs.lstatSync(tarball).isFile()) throw new Error(`Not a regular tarball: ${file}`);
+ const manifest = await pacote.manifest(tarball, {
+ fullmetadata: true,
+ fullReadJson: true,
+ ignoreScripts: true,
+ cache: path.join(process.env.RUNNER_TEMP, 'npm-metadata-cache'),
+ });
+ if (!trusted.has(manifest.name) || trusted.get(manifest.name) !== manifest.version) {
+ throw new Error(`Untrusted package identity in ${file}`);
+ }
+ if (seen.has(manifest.name)) throw new Error(`Duplicate tarball for ${manifest.name}`);
+ seen.add(manifest.name);
+ // No workspace requires publishConfig. It can override npm's proxy,
+ // TLS and registry options even with --ignore-scripts, exposing OIDC.
+ if (manifest.publishConfig !== undefined && JSON.stringify(manifest.publishConfig) !== '{}') {
+ throw new Error(`Builder-supplied publishConfig is not allowed in ${file}`);
+ }
+ }
+ // A partial artifact would otherwise publish an incomplete release.
+ const missing = [...trusted.keys()].filter(name => !seen.has(name));
+ if (missing.length) throw new Error(`Missing NPM tarballs: ${missing.join(', ')}`);
+ })().catch(error => { console.error(error.message); process.exitCode = 1; });
+ NODE
- name: Set suffix
- uses: actions/github-script@v6
+ uses: actions/github-script@v8
id: suffix
with:
result-encoding: string
script: |
- const fullTag = "${{ inputs.tag }}" || context.payload.release.tag_name;
+ const fullTag = "${{ steps.test-tag.outputs.tag }}" || context.payload.release.tag_name;
if (fullTag.includes('-')) {
const [, fullSuffix] = fullTag.split('-');
const [suffix] = fullSuffix.split('.');
@@ -116,12 +148,12 @@ jobs:
}
- name: Set NPM release tag
- uses: actions/github-script@v6
+ uses: actions/github-script@v8
id: tag
with:
result-encoding: string
script: |
- const tag = "${{ inputs.tag }}" || context.payload.release.tag_name;
+ const tag = "${{ steps.test-tag.outputs.tag }}" || context.payload.release.tag_name;
const [, major, minor] = tag.match(/^v([0-9]+)\.([0-9]+)/);
return (tag.includes('-') ? `${major}.${minor}-${{steps.suffix.outputs.result}}` : 'latest');
@@ -130,41 +162,65 @@ jobs:
echo "NPM suffix: ${{ steps.suffix.outputs.result }}"
echo "NPM release tag: ${{ steps.tag.outputs.result }}"
+ # --provenance=false keeps what `yarn npm publish` did before. After a
+ # trusted-publishing token exchange from a public repository, npm turns
+ # provenance on unless it is set explicitly, and the registry then
+ # rejects every package whose repository.url does not name
+ # github.com/dashpay/platform (all but wasm-drive-verify today).
- name: Publish NPM packages
- run: yarn workspaces foreach --all --no-private --parallel npm publish --tolerate-republish --access public --tag ${{ steps.tag.outputs.result }}
-
- - name: Ignore only already cached artifacts
+ if: github.event_name == 'release'
+ env:
+ NPM_TAG: ${{ steps.tag.outputs.result }}
run: |
- find . -name '.gitignore' -exec rm -f {} +
- echo ".yarn" >> .gitignore
- echo "target" >> .gitignore
- echo "node_modules" >> .gitignore
- echo ".nyc_output" >> .gitignore
- echo ".idea" >> .gitignore
- echo ".ultra.cache.json" >> .gitignore
- echo "db/*" >> .gitignore
- if: ${{ steps.check-artifact.outputs.exists != 'true' }}
-
- - name: Get modified files
- id: diff
+ set -euo pipefail
+ REGISTRY="$(npm config get registry)"
+ REGISTRY="${REGISTRY%/}"
+ for tarball in release-artifacts/npm-packages/*.tgz; do
+ metadata=$(tar -xOf "$tarball" package/package.json)
+ name=$(jq -r '.name' <<<"$metadata")
+ version=$(jq -r '.version' <<<"$metadata")
+ # npm addresses scoped packages as @scope%2fname: keep the leading @,
+ # percent-encode only the scope separator.
+ encoded_name=${name//\//%2f}
+ status=$(curl --silent --show-error --location --output "$RUNNER_TEMP/npm-metadata.json" \
+ --write-out '%{http_code}' --connect-timeout 10 --max-time 30 \
+ "$REGISTRY/$encoded_name") || {
+ echo "::error::Could not query $REGISTRY for $name@$version."
+ exit 1
+ }
+ case "$status" in
+ 404)
+ ;;
+ 200)
+ jq -e --arg version "$version" '.versions[$version] != null' "$RUNNER_TEMP/npm-metadata.json" >/dev/null || {
+ jq -e . "$RUNNER_TEMP/npm-metadata.json" >/dev/null || { echo "::error::Invalid registry response for $name."; exit 1; }
+ npm publish "$tarball" --ignore-scripts --provenance=false --access public --tag "$NPM_TAG"
+ continue
+ }
+ echo "::notice::$name@$version already exists on $REGISTRY; skipping."
+ continue
+ ;;
+ *)
+ echo "::error::Registry query for $name@$version returned HTTP $status; refusing to publish blindly."
+ exit 1
+ ;;
+ esac
+ npm publish "$tarball" --ignore-scripts --provenance=false --access public --tag "$NPM_TAG"
+ done
+
+ - name: Dry-run NPM packages
+ if: github.event_name == 'workflow_dispatch'
+ env:
+ NPM_TAG: ${{ steps.tag.outputs.result }}
run: |
- echo "files<> $GITHUB_OUTPUT
- git ls-files --others --exclude-standard >> $GITHUB_OUTPUT
- echo "EOF" >> $GITHUB_OUTPUT
- if: ${{ steps.check-artifact.outputs.exists != 'true' }}
-
- - name: Upload the archive of built files
- uses: actions/upload-artifact@v4
- with:
- name: js-build-${{ github.sha }}
- path: ${{ steps.diff.outputs.files }}
- retention-days: 1
- if-no-files-found: error
- include-hidden-files: true
- if: ${{ steps.check-artifact.outputs.exists != 'true' }}
+ set -euo pipefail
+ for tarball in release-artifacts/npm-packages/*.tgz; do
+ npm publish "$tarball" --ignore-scripts --provenance=false --dry-run --access public --tag "$NPM_TAG"
+ done
release-drive-image:
name: Release Drive image
+ if: ${{ !startsWith(inputs.tag, 'npm-test:') }}
secrets: inherit
uses: ./.github/workflows/release-docker-image.yml
with:
@@ -176,6 +232,7 @@ jobs:
release-drive-image-debug:
name: Release Drive debug image
+ if: ${{ !startsWith(inputs.tag, 'npm-test:') }}
secrets: inherit
uses: ./.github/workflows/release-docker-image.yml
with:
@@ -189,7 +246,7 @@ jobs:
release-rs-dapi-image:
name: Release RS-DAPI image
- if: ${{ !inputs.only_drive }}
+ if: ${{ !inputs.only_drive && !startsWith(inputs.tag, 'npm-test:') }}
secrets: inherit
uses: ./.github/workflows/release-docker-image.yml
with:
@@ -201,7 +258,7 @@ jobs:
release-test-suite-image:
name: Release Test Suite image
- if: ${{ !inputs.only_drive }}
+ if: ${{ !inputs.only_drive && !startsWith(inputs.tag, 'npm-test:') }}
secrets: inherit
uses: ./.github/workflows/release-docker-image.yml
with:
@@ -214,7 +271,7 @@ jobs:
release-dashmate-helper-image:
name: Release Dashmate Helper image
secrets: inherit
- if: ${{ !inputs.only_drive }}
+ if: ${{ !inputs.only_drive && !startsWith(inputs.tag, 'npm-test:') }}
uses: ./.github/workflows/release-docker-image.yml
with:
name: Dashmate Helper
@@ -246,14 +303,13 @@ jobs:
with:
tag: ${{ github.event.release.tag_name }}
- release-dashmate-packages:
- name: Release Dashmate packages
+ build-dashmate-packages:
+ name: Build Dashmate packages
runs-on: ${{ matrix.os }}
- if: ${{ !inputs.only_drive }}
+ if: ${{ !inputs.only_drive && !startsWith(inputs.tag, 'npm-test:') }}
needs: release-npm
permissions:
- id-token: write # s3 cache
- contents: write # update release artifacts
+ contents: read
strategy:
fail-fast: false
matrix:
@@ -271,12 +327,15 @@ jobs:
uses: actions/checkout@v4
with:
fetch-depth: 0
+ persist-credentials: false
- name: Download JS build artifacts
uses: actions/download-artifact@v4
with:
name: js-build-${{ github.sha }}
- path: packages
+ # The mixed JS/NPM artifact is rooted at the repository, since its
+ # paths contain both packages/ and npm-packages/.
+ path: .
- name: Install macOS build deps
if: runner.os == 'macOS'
@@ -287,29 +346,6 @@ jobs:
if: runner.os == 'macOS'
uses: docker-practice/actions-setup-docker@master
- - name: Install the Apple certificate
- if: runner.os == 'macOS'
- env:
- BUILD_CERTIFICATE_BASE64: ${{ secrets.MACOS_BUILD_CERTIFICATE_BASE64 }}
- P12_PASSWORD: ${{ secrets.MACOS_P12_PASSWORD }}
- KEYCHAIN_PASSWORD: ${{ secrets.MACOS_KEYCHAIN_PASSWORD }}
- run: |
- # create variables
- CERTIFICATE_PATH=$RUNNER_TEMP/build_certificate.p12
- KEYCHAIN_PATH=$RUNNER_TEMP/app-signing.keychain-db
-
- # import certificate and provisioning profile from secrets
- echo -n "$BUILD_CERTIFICATE_BASE64" | base64 --decode -o $CERTIFICATE_PATH
-
- # create temporary keychain
- security create-keychain -p "$KEYCHAIN_PASSWORD" $KEYCHAIN_PATH
- security set-keychain-settings -lut 21600 $KEYCHAIN_PATH
- security unlock-keychain -p "$KEYCHAIN_PASSWORD" $KEYCHAIN_PATH
-
- # import certificate to keychain
- security import $CERTIFICATE_PATH -P "$P12_PASSWORD" -A -t cert -f pkcs12 -k $KEYCHAIN_PATH
- security list-keychain -d user -s $KEYCHAIN_PATH
-
- name: Install Linux build deps
if: runner.os == 'Linux'
run: sudo apt-get install -y nsis
@@ -324,23 +360,106 @@ jobs:
- name: Create package
env:
- OSX_KEYCHAIN: ${{ runner.temp }}/app-signing.keychain-db
- run: "${GITHUB_WORKSPACE}/scripts/pack_dashmate.sh ${{ matrix.package_type }}"
+ # Packaging executes dependency and lifecycle scripts, including
+ # output from the persistent builder. Signing happens in a new job.
+ DASHMATE_UNSIGNED: 'true'
+ run: |
+ "${GITHUB_WORKSPACE}/scripts/pack_dashmate.sh" "${{ matrix.package_type }}"
- - name: Upload artifacts to action summary
+ - name: Upload Dashmate packages
uses: actions/upload-artifact@v4
- if: github.event_name != 'release'
with:
- name: dashmate
+ name: dashmate-${{ matrix.package_type }}-${{ github.sha }}
path: packages/dashmate/dist/**
+ if-no-files-found: error
+ retention-days: 7
+
+ release-dashmate-packages:
+ name: Release Dashmate packages
+ needs: build-dashmate-packages
+ # `needs` on a matrix job waits for every leg and reports failure if any
+ # one leg failed, which by default would skip all four release legs. Each
+ # leg below needs only its own package type's artifact, so run whenever
+ # the build matrix ran at all: a leg whose build failed then fails at its
+ # own download step, and the other package types still ship.
+ if: >-
+ !cancelled()
+ && github.event_name == 'release'
+ && needs.build-dashmate-packages.result != 'skipped'
+ runs-on: ${{ matrix.os }}
+ permissions:
+ contents: write
+ strategy:
+ fail-fast: false
+ matrix:
+ include:
+ - package_type: tarballs
+ os: ubuntu-24.04
+ - package_type: win
+ os: ubuntu-24.04
+ - package_type: deb
+ os: ubuntu-24.04
+ - package_type: macos
+ os: macos-14
+ steps:
+ # Treat finished installers as data. This job never installs or runs
+ # package code and never restores caches from the build jobs.
+ - name: Download Dashmate packages
+ uses: actions/download-artifact@v4
+ with:
+ name: dashmate-${{ matrix.package_type }}-${{ github.sha }}
+ path: release-artifacts
+
+ - name: Install the Apple certificate
+ if: runner.os == 'macOS'
+ env:
+ BUILD_CERTIFICATE_BASE64: ${{ secrets.MACOS_BUILD_CERTIFICATE_BASE64 }}
+ P12_PASSWORD: ${{ secrets.MACOS_P12_PASSWORD }}
+ KEYCHAIN_PASSWORD: ${{ secrets.MACOS_KEYCHAIN_PASSWORD }}
+ run: |
+ set -euo pipefail
+ CERTIFICATE_PATH="$RUNNER_TEMP/build_certificate.p12"
+ KEYCHAIN_PATH="$RUNNER_TEMP/app-signing.keychain-db"
+ printf '%s' "$BUILD_CERTIFICATE_BASE64" | base64 --decode -o "$CERTIFICATE_PATH"
+ security create-keychain -p "$KEYCHAIN_PASSWORD" "$KEYCHAIN_PATH"
+ security set-keychain-settings -lut 21600 "$KEYCHAIN_PATH"
+ security unlock-keychain -p "$KEYCHAIN_PASSWORD" "$KEYCHAIN_PATH"
+ security import "$CERTIFICATE_PATH" -P "$P12_PASSWORD" -A -t cert -f pkcs12 -k "$KEYCHAIN_PATH"
+ security list-keychain -d user -s "$KEYCHAIN_PATH"
+
+ - name: Sign macOS installers
+ if: runner.os == 'macOS'
+ run: |
+ set -euo pipefail
+ found=false
+ while IFS= read -r -d '' pkg; do
+ found=true
+ productsign --sign 'Developer ID Installer: The Dash Foundation, Inc.' \
+ --keychain "$RUNNER_TEMP/app-signing.keychain-db" \
+ "$pkg" "$RUNNER_TEMP/signed.pkg"
+ mv "$RUNNER_TEMP/signed.pkg" "$pkg"
+ done < <(find release-artifacts -type f -name '*.pkg' -print0)
+ "$found" || { echo '::error::No macOS installers were uploaded.'; exit 1; }
- name: Notarize MacOS Release Build
if: runner.os == 'macOS'
+ env:
+ APPLE_ID: ${{ secrets.MACOS_APPLE_ID }}
+ TEAM_ID: ${{ secrets.MACOS_TEAM_ID }}
+ NOTARIZING_PASSWORD: ${{ secrets.MACOS_NOTARIZING_PASSWORD }}
+ run: |
+ while IFS= read -r -d '' pkg; do
+ xcrun notarytool submit "$pkg" --apple-id "$APPLE_ID" --team-id "$TEAM_ID" --password "$NOTARIZING_PASSWORD" --wait
+ done < <(find release-artifacts -type f -name '*.pkg' -print0)
+
+ - name: Delete the Apple keychain
+ if: always() && runner.os == 'macOS'
run: |
- find packages/dashmate/dist/ -name '*.pkg' -exec sh -c 'xcrun notarytool submit "{}" --apple-id "${{ secrets.MACOS_APPLE_ID }}" --team-id "${{ secrets.MACOS_TEAM_ID }}" --password "${{ secrets.MACOS_NOTARIZING_PASSWORD }}" --wait;' \;
+ security delete-keychain "$RUNNER_TEMP/app-signing.keychain-db"
+ rm -f "$RUNNER_TEMP/build_certificate.p12"
- name: Upload artifacts to release
uses: softprops/action-gh-release@v0.1.15
if: github.event_name == 'release'
with:
- files: packages/dashmate/dist/**
+ files: release-artifacts/**
diff --git a/.github/workflows/runner-image-candidate.yml b/.github/workflows/runner-image-candidate.yml
new file mode 100644
index 00000000000..5e4708466fc
--- /dev/null
+++ b/.github/workflows/runner-image-candidate.yml
@@ -0,0 +1,33 @@
+name: Runner image candidate
+
+on:
+ pull_request_target:
+ types: [opened, synchronize, reopened, ready_for_review, closed]
+ branches: [master, 'v*-dev', 'ci/*']
+ paths: ['.github/runner-requirements.json']
+
+# This is trusted base-branch orchestration. Never check out PR code or select
+# the control revision from PR data. Build/publish execute on separate hosted VMs.
+permissions:
+ contents: read
+ pull-requests: read
+ actions: read
+ statuses: write
+
+concurrency:
+ group: ${{ github.event.action == 'closed' && format('runner-image-promote-{0}', github.event.pull_request.base.ref) || format('runner-image-pr-{0}', github.event.pull_request.number) }}
+ cancel-in-progress: ${{ github.event.action != 'closed' }}
+
+jobs:
+ image:
+ if: >-
+ (github.event.action != 'closed' && !github.event.pull_request.draft)
+ || (github.event.action == 'closed' && github.event.pull_request.merged)
+ uses: dashpay/dash-selfhosted-image/.github/workflows/platform-candidate.yml@7d901150bd3d0789d50c46f365b058f2f5f1f52d
+ with:
+ pull_request: ${{ github.event.pull_request.number }}
+ control_revision: 7d901150bd3d0789d50c46f365b058f2f5f1f52d
+ mode: ${{ github.event.action == 'closed' && 'promote' || 'candidate' }}
+ secrets:
+ DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
+ DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
diff --git a/.github/workflows/test-client-codegen.yml b/.github/workflows/test-client-codegen.yml
new file mode 100644
index 00000000000..1517760dacd
--- /dev/null
+++ b/.github/workflows/test-client-codegen.yml
@@ -0,0 +1,42 @@
+name: Native client generation
+on:
+ pull_request:
+ paths:
+ - 'packages/dapi-grpc/**'
+ - '.github/workflows/test-client-codegen.yml'
+ - 'yarn.lock'
+ workflow_dispatch:
+permissions:
+ contents: read
+jobs:
+ generate:
+ strategy:
+ matrix:
+ os: [ubuntu-24.04, macos-15]
+ runs-on: ${{ matrix.os }}
+ timeout-minutes: 20
+ steps:
+ - uses: actions/checkout@v4
+ with:
+ persist-credentials: false
+ - uses: actions/setup-python@v5
+ with:
+ python-version: '3.12'
+ - uses: actions/cache@v5
+ with:
+ path: ~/.cache/dash/client-codegen
+ key: client-codegen/${{ runner.os }}/${{ runner.arch }}/${{ hashFiles('packages/dapi-grpc/codegen.json') }}
+ - name: Build the locked native compilers
+ run: python3 packages/dapi-grpc/scripts/setup-codegen.py --install
+ - uses: ./.github/actions/nodejs
+ - name: Regenerate and test clients
+ run: |
+ yarn workspace @dashevo/dapi-grpc build
+ git diff --exit-code -- packages/dapi-grpc/clients
+ yarn workspace @dashevo/dapi-grpc test:unit
+ yarn workspace @dashevo/dapi-grpc exec python3 -m unittest discover -s tests/codegen -v
+ - name: Verify published client contents
+ run: |
+ mkdir -p npm-packages
+ yarn workspace @dashevo/dapi-grpc pack --out "$GITHUB_WORKSPACE/npm-packages/dapi-grpc.tgz"
+ python3 packages/dapi-grpc/scripts/check-packed-clients.py npm-packages
diff --git a/.github/workflows/tests-build-js.yml b/.github/workflows/tests-build-js.yml
index 75d1324ee40..05e7e642362 100644
--- a/.github/workflows/tests-build-js.yml
+++ b/.github/workflows/tests-build-js.yml
@@ -5,7 +5,8 @@ jobs:
build-js:
name: Build JS
runs-on: ubuntu-24.04
- timeout-minutes: 15
+ # Allow for a cold native compiler build and the final artifact upload.
+ timeout-minutes: 25
steps:
- uses: softwareforgood/check-artifact-v4-existence@v0
id: check-artifact
@@ -18,42 +19,16 @@ jobs:
fetch-depth: 0
if: ${{ steps.check-artifact.outputs.exists != 'true' }}
- - name: Check DockerHub credentials
- id: check-dockerhub
- if: ${{ steps.check-artifact.outputs.exists != 'true' }}
- run: |
- if [ -n "$DOCKERHUB_USERNAME" ]; then
- echo "available=true" >> "$GITHUB_OUTPUT"
- else
- echo "available=false" >> "$GITHUB_OUTPUT"
- fi
- env:
- DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
-
- - name: Login to DockerHub
- uses: docker/login-action@v3
- with:
- username: ${{ secrets.DOCKERHUB_USERNAME }}
- password: ${{ secrets.DOCKERHUB_TOKEN }}
- if: ${{ steps.check-artifact.outputs.exists != 'true' && steps.check-dockerhub.outputs.available == 'true' }}
-
- - name: Cache protoc Docker image
- id: cache-protoc
+ - name: Cache native client generators
uses: actions/cache@v5
with:
- path: /tmp/protoc-image.tar
- key: docker-rvolosatovs-protoc-4.0.0
+ path: ~/.cache/dash/client-codegen
+ key: client-codegen/${{ runner.os }}/${{ runner.arch }}/${{ hashFiles('packages/dapi-grpc/codegen.json') }}
if: ${{ steps.check-artifact.outputs.exists != 'true' }}
- - name: Load protoc image from cache
- run: docker load -i /tmp/protoc-image.tar
- if: ${{ steps.check-artifact.outputs.exists != 'true' && steps.cache-protoc.outputs.cache-hit == 'true' }}
-
- - name: Pull and cache protoc Docker image
- run: |
- docker pull rvolosatovs/protoc:4.0.0
- docker save rvolosatovs/protoc:4.0.0 -o /tmp/protoc-image.tar
- if: ${{ steps.check-artifact.outputs.exists != 'true' && steps.cache-protoc.outputs.cache-hit != 'true' }}
+ - name: Install pinned native client generators
+ run: python3 packages/dapi-grpc/scripts/setup-codegen.py --install
+ if: ${{ steps.check-artifact.outputs.exists != 'true' }}
- name: Setup Node.JS
uses: ./.github/actions/nodejs
diff --git a/.github/workflows/tests-rs-wallet.yml b/.github/workflows/tests-rs-wallet.yml
index 386d6bfe7be..b18700d9ba3 100644
--- a/.github/workflows/tests-rs-wallet.yml
+++ b/.github/workflows/tests-rs-wallet.yml
@@ -20,10 +20,8 @@
#
# No coverage here: code coverage is collected only by the nightly run of the
# full workspace workflow (tests-rs-workspace.yml, `coverage` input), which
-# also measures the wallet crates. Known trade-offs: this fast path stays
-# pinned to the macOS runners (unlike the full workspace job, which schedules
-# onto any `rust-ci` self-hosted runner), so wallet PRs depend on a mac
-# runner being online; and the scoped `-p` builds feature-unify shared deps
+# also measures the wallet crates. Both workflows use the same Linux image
+# pool, including ARM64 Linux VMs on Macs. The scoped `-p` builds feature-unify shared deps
# differently than `--workspace` builds, so the shared target/ carries an
# extra artifact flavor.
on:
@@ -35,9 +33,35 @@ on:
default: false
jobs:
+ select-runner:
+ name: Select compatible runner image
+ if: >-
+ github.event_name != 'pull_request'
+ || github.event.pull_request.head.repo.full_name == github.repository
+ || github.event.pull_request.head.repo.owner.login == 'thepastaclaw'
+ runs-on: ubuntu-24.04
+ timeout-minutes: 135
+ permissions:
+ contents: read
+ pull-requests: read
+ statuses: read
+ actions: read
+ outputs:
+ labels: ${{ steps.select.outputs.labels }}
+ steps:
+ - uses: actions/checkout@v4
+ with:
+ persist-credentials: false
+ - name: Select provisioned pool or exact PR candidate
+ id: select
+ env:
+ GH_TOKEN: ${{ github.token }}
+ run: python3 .github/scripts/runner-image.py select --kind rust
+
test-mac:
- name: Wallet tests (macOS)
- runs-on: [self-hosted, macOS, ARM64]
+ name: Wallet tests (Linux image)
+ needs: select-runner
+ runs-on: ${{ fromJSON(needs.select-runner.outputs.labels) }}
if: >-
github.event_name != 'pull_request'
|| github.event.pull_request.head.repo.full_name == github.repository
@@ -58,7 +82,7 @@ jobs:
# shares the same runner workspace — wiping it on a wallet-only PR
# would force the next nightly into a ~2.5 min cold rebuild. The size
# guard below still removes the whole target/ if it outgrows the caps.
- - name: Prune macOS runner disk before tests
+ - name: Prune runner disk before tests
run: |
for path in ../target-backup-before-*-clean-*; do
if [ -e "$path" ]; then
@@ -77,6 +101,11 @@ jobs:
TARGET_MAX_MB=120000
MIN_FREE_MB=60000
+ TOTAL=$(df -m . | awk 'NR == 2 {print $2}')
+ if [ "${TOTAL:-0}" -gt 0 ]; then
+ [ $((TOTAL / 3)) -lt "$TARGET_MAX_MB" ] && TARGET_MAX_MB=$((TOTAL / 3))
+ [ $((TOTAL / 5)) -lt "$MIN_FREE_MB" ] && MIN_FREE_MB=$((TOTAL / 5))
+ fi
SIZE=$(du -sm target 2>/dev/null | awk '{print $1}' || echo 0)
FREE=$(df -m . | awk 'NR == 2 {print $4}')
SIZE=${SIZE:-0}
@@ -89,34 +118,15 @@ jobs:
rm -rf target
fi
- - name: Verify build dependencies (macOS)
- run: |
- # Persistent runners must be provisioned before accepting CI jobs.
- echo "/opt/homebrew/bin" >> "$GITHUB_PATH"
- echo "/opt/homebrew/opt/llvm/bin" >> "$GITHUB_PATH"
- export PATH="/opt/homebrew/opt/llvm/bin:/opt/homebrew/bin:$PATH"
-
- missing=0
- for tool in cmake llvm-config; do
- if ! "$tool" --version >/dev/null 2>&1; then
- echo "::error::Missing or unusable $tool. Provision this dependency on the runner before running CI."
- missing=1
- fi
- done
- for library in /opt/homebrew/opt/gmp/include/gmp.h /opt/homebrew/opt/gmp/lib/libgmp.dylib /opt/homebrew/opt/llvm/lib/libclang.dylib; do
- if [ ! -r "$library" ]; then
- echo "::error::Missing or unreadable $library. Provision this dependency on the runner before running CI."
- missing=1
- fi
- done
- exit "$missing"
-
- name: Setup Rust
uses: ./.github/actions/rust
with:
cache: false
components: rustfmt, clippy
+ - name: Verify repository-pinned cargo-nextest
+ run: test "$(cargo nextest --version | awk 'NR == 1 {print $2}')" = "$CI_CARGO_NEXTEST_VERSION"
+
# Enforce the invariant the scoped --package lists below rely on: the
# only workspace crate depending (transitively) on the wallet crates is
# rs-unified-sdk-ffi. The same check runs on the full workspace path,
@@ -131,7 +141,7 @@ jobs:
- name: Find unused dependencies
run: |
- cargo install cargo-machete 2>/dev/null || true
+ test "$(cargo machete --version | awk '{print $NF}')" = "$CI_CARGO_MACHETE_VERSION"
cargo machete
- name: Detect immutable structure changes
diff --git a/.github/workflows/tests-rs-workspace.yml b/.github/workflows/tests-rs-workspace.yml
index fb5632dc6da..19e21262574 100644
--- a/.github/workflows/tests-rs-workspace.yml
+++ b/.github/workflows/tests-rs-workspace.yml
@@ -1,6 +1,14 @@
on:
workflow_call:
inputs:
+ runner-architecture:
+ description: Optional native architecture for image validation (X64 or ARM64)
+ type: string
+ default: ''
+ validate-arm64-image:
+ description: Use validation-only ARM64 capacity before admission to the ordinary Rust pool
+ type: boolean
+ default: false
doctests-changed:
description: Whether doc comments with code examples have changed
type: boolean
@@ -26,14 +34,43 @@ on:
default: false
jobs:
+ select-runner:
+ name: Select compatible runner image
+ if: >-
+ github.event_name != 'pull_request'
+ || github.event.pull_request.head.repo.full_name == github.repository
+ || github.event.pull_request.head.repo.owner.login == 'thepastaclaw'
+ runs-on: ubuntu-24.04
+ # Allow the separate 90-minute build and publication/queue window to finish.
+ timeout-minutes: 135
+ permissions:
+ contents: read
+ pull-requests: read
+ statuses: read
+ actions: read
+ outputs:
+ labels: ${{ steps.select.outputs.labels }}
+ steps:
+ - uses: actions/checkout@v4
+ with:
+ persist-credentials: false
+ - name: Select provisioned pool or wait for the exact PR candidate
+ id: select
+ env:
+ GH_TOKEN: ${{ github.token }}
+ RUNNER_ARCHITECTURE: ${{ inputs.runner-architecture }}
+ VALIDATE_ARM64_IMAGE: ${{ inputs.validate-arm64-image }}
+ run: |
+ extra=()
+ if [ "$VALIDATE_ARM64_IMAGE" = true ]; then extra+=(--validation); fi
+ python3 .github/scripts/runner-image.py select --kind rust --arch "$RUNNER_ARCHITECTURE" "${extra[@]}"
+
test:
name: Tests
- # Scheduled onto whichever self-hosted runner is free — the macOS boxes or
- # the Linux one. `rust-ci` is a custom label applied to exactly those
- # runners; pairing it with `self-hosted` keeps the job off GitHub-hosted
- # runners entirely, so untrusted code never reaches a hosted Linux VM by
- # way of a label collision.
- runs-on: [self-hosted, rust-ci]
+ # Shared image pool: physical Linux hosts and ARM64 Linux VMs on Macs.
+ # Native macOS registrations remain available for Swift/Xcode.
+ needs: select-runner
+ runs-on: ${{ fromJSON(needs.select-runner.outputs.labels) }}
# Fork PRs must not execute on any persistent runner, macOS or Linux.
if: >-
github.event_name != 'pull_request'
@@ -130,43 +167,29 @@ jobs:
done
exit "$missing"
- # clang, llvm and libsnappy are installed by ./.github/actions/rust on
- # Linux; this covers what the rest of the job needs and what a bare
- # self-hosted image doesn't ship. Every branch is a no-op once the
- # persistent runner has been provisioned by the first run.
- - name: Install build dependencies (Linux)
+ # The persistent runner image is the versioned CI environment. Runtime
+ # apt/sudo would let a job mutate the runner and is unnecessary once the
+ # image contract is provisioned.
+ - name: Verify build dependencies (Linux)
if: runner.os == 'Linux'
run: |
set -euo pipefail
- MISSING=()
- for pkg in build-essential cmake libgmp-dev libssl-dev pkg-config jq zip; do
- dpkg -s "$pkg" >/dev/null 2>&1 || MISSING+=("$pkg")
+ missing=()
+ for tool in clang cmake gh jq llvm-config rustup zip; do
+ command -v "$tool" >/dev/null 2>&1 || missing+=("$tool")
done
- if [ ${#MISSING[@]} -gt 0 ]; then
- echo "Installing: ${MISSING[*]}"
- sudo apt-get update -qq
- sudo apt-get install -qq --yes "${MISSING[@]}"
- fi
-
- # Needed by the immutable-structure check below.
- if ! command -v gh >/dev/null 2>&1; then
- sudo apt-get install -qq --yes gh || {
- curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg \
- | sudo dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg
- echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" \
- | sudo tee /etc/apt/sources.list.d/github-cli.list > /dev/null
- sudo apt-get update -qq
- sudo apt-get install -qq --yes gh
- }
+ for pkg in build-essential clang libgmp-dev libsnappy-dev libssl-dev llvm pkg-config; do
+ dpkg -s "$pkg" >/dev/null 2>&1 || missing+=("$pkg")
+ done
+ if [ ${#missing[@]} -gt 0 ]; then
+ echo "::error::Self-hosted runner image is missing: ${missing[*]}"
+ echo "::error::Provision these in the pinned runner image; CI jobs must not install host packages."
+ exit 1
fi
- # dtolnay/rust-toolchain drives rustup; it must already exist.
- if ! command -v rustup >/dev/null 2>&1; then
- curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs \
- | sh -s -- -y --no-modify-path --default-toolchain none
- echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
- fi
+ # rustup is prebaked; Setup Rust may install the repository-selected
+ # toolchain under the runner user's home, without root.
- name: Setup Rust
uses: ./.github/actions/rust
@@ -174,26 +197,17 @@ jobs:
cache: false
components: llvm-tools, rustfmt, clippy
- # Install only when missing, and fail HERE, loudly, if the tool still
- # doesn't work afterwards — the previous `cargo install ... 2>/dev/null
- # || true` silently swallowed a failed nextest install on a freshly
- # provisioned runner, and the job then died 10 minutes later in the
- # test step with "no such command: nextest". cargo-llvm-cov is only
- # needed by the nightly coverage run.
- - name: Install cargo-llvm-cov
+ # These helpers are part of the pinned runner image. Version checks make
+ # image drift fail before the test phase instead of installing tools from
+ # a job or silently swallowing a failed installation. Coverage tooling
+ # is only needed when the caller requests an instrumented run.
+ - name: Verify repository-pinned cargo-llvm-cov
if: ${{ inputs.coverage }}
- run: |
- if ! cargo llvm-cov --version >/dev/null 2>&1; then
- cargo install cargo-llvm-cov --locked
- fi
- cargo llvm-cov --version
+ run: cargo llvm-cov --version | grep -Fx "cargo-llvm-cov $CI_CARGO_LLVM_COV_VERSION"
- - name: Install cargo-nextest
- run: |
- if ! cargo nextest --version >/dev/null 2>&1; then
- cargo install cargo-nextest --locked
- fi
- cargo nextest --version
+ - name: Verify repository-pinned cargo-nextest
+ # Release binaries include commit/host metadata after the version.
+ run: test "$(cargo nextest --version | awk 'NR == 1 {print $2}')" = "$CI_CARGO_NEXTEST_VERSION"
- name: Check formatting
run: cargo fmt --check --all
@@ -219,7 +233,7 @@ jobs:
- name: Find unused dependencies
run: |
- cargo install cargo-machete 2>/dev/null || true
+ test "$(cargo machete --version | awk '{print $NF}')" = "$CI_CARGO_MACHETE_VERSION"
cargo machete
# The transport-free cuts are how embedders with their own networking
@@ -623,11 +637,13 @@ jobs:
- name: Run doctests
if: ${{ inputs.doctests-changed }}
run: |
+ # Rustdoc compiles/links examples concurrently, independently of
+ # Cargo's build-job limit. Bound it too for memory-limited runners.
cargo test \
--workspace \
--all-features \
--locked \
- --doc
+ --doc -- --test-threads="${CARGO_BUILD_JOBS:-2}"
env:
CARGO_PROFILE_DEV_DEBUG: "0"
CARGO_PROFILE_DEV_CODEGEN_UNITS: "256"
diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
index f1ad788ecfc..0508af6eafb 100644
--- a/.github/workflows/tests.yml
+++ b/.github/workflows/tests.yml
@@ -61,6 +61,7 @@ jobs:
js-packages-direct: ${{ steps.override.outputs.js-packages-direct || steps.filter-js-direct.outputs.changes }}
rs-packages: ${{ steps.override.outputs.rs-packages || steps.filter-rs.outputs.changes }}
rs-workflows-changed: ${{ steps.filter-rs-workflows.outputs.rs-workflows }}
+ arm64-image-changed: ${{ steps.filter-rs-workflows.outputs.arm64-image }}
rs-scope: ${{ steps.rs-scope.outputs.scope }}
shielded-changed: ${{ steps.override.outputs.shielded-changed || steps.filter-shielded.outputs.shielded-changed }}
doctests-changed: ${{ steps.override.outputs.doctests-changed || steps.filter-doctests.outputs.doctests-changed }}
@@ -76,6 +77,11 @@ jobs:
- name: Verify self-hosted Swift runner policy
run: python3 .github/scripts/check-swift-self-hosted-runner.py
+ - name: Verify candidate-image routing and manifest inputs
+ run: |
+ python3 .github/scripts/runner-image.py env
+ python3 -m unittest discover -s .github/scripts/tests -v
+
- uses: dorny/paths-filter@v4
id: filter-js
if: ${{ github.event_name != 'workflow_dispatch' }}
@@ -104,6 +110,12 @@ jobs:
- .github/workflows/tests-rs-wallet.yml
- .github/workflows/tests.yml
- .github/scripts/check-wallet-closure.py
+ - .github/runner-requirements.json
+ - .github/runner-requirements.arm64.json
+ - .github/scripts/runner-image.py
+ - .github/actions/rust/**
+ arm64-image:
+ - .github/runner-requirements.arm64.json
- uses: dorny/paths-filter@v4
id: filter-e2e
@@ -286,7 +298,7 @@ jobs:
exit 0
fi
- if echo "$CHANGED" | grep -qE '(^|/)Cargo\.(toml|lock)$|^rust-toolchain\.toml$|^\.github/actions/rust/|^\.github/workflows/tests\.yml$|^\.github/workflows/tests-rs-workspace\.yml$'; then
+ if echo "$CHANGED" | grep -qE '(^|/)Cargo\.(toml|lock)$|^rust-toolchain\.toml$|^\.github/runner-requirements(\.arm64)?\.json$|^\.github/scripts/runner-image\.py$|^\.github/actions/rust/|^\.github/workflows/tests\.yml$|^\.github/workflows/tests-rs-workspace\.yml$'; then
echo "shielded-changed=true" >> "$GITHUB_OUTPUT"
echo "Build configuration changed — shielded tests will run"
exit 0
@@ -469,6 +481,23 @@ jobs:
# test phases without instrumentation and upload nothing to Codecov.
coverage: ${{ github.event_name == 'schedule' || (github.event_name == 'workflow_dispatch' && inputs.coverage == true) }}
+ # The AMD64 candidate publisher cannot prove an ARM64 image. Changes to
+ # its separate manifest need a real full workspace job on that architecture,
+ # in addition to any ordinary/AMD64 candidate job above. Provision the exact
+ # ARM64 image before merging; a hosted build or skipped fork job is not proof.
+ rs-arm64-image-tests:
+ name: ARM64 runner image validation
+ needs: changes
+ if: ${{ needs.changes.outputs.arm64-image-changed == 'true' }}
+ secrets: inherit
+ uses: ./.github/workflows/tests-rs-workspace.yml
+ with:
+ runner-architecture: ARM64
+ validate-arm64-image: true
+ doctests-changed: true
+ shielded-changed: true
+ coverage: false
+
# Fast path: only wallet crates changed, so run the scoped wallet suite
# instead of the full workspace job above (the two are mutually exclusive
# via `rs-scope`).
diff --git a/.pnp.cjs b/.pnp.cjs
index ddb8eee1876..c1f40457d98 100755
--- a/.pnp.cjs
+++ b/.pnp.cjs
@@ -2664,7 +2664,8 @@ const RAW_RUNTIME_STATE =
["mocha", "npm:11.1.0"],\
["mocha-sinon", "virtual:595d7482cc8ddf98ee6aef33fc48b46393554ab5f17f851ef62e6e39315e53666c3e66226b978689aa0bc7f1e83a03081511a21db1c381362fe67614887077f9#npm:2.1.2"],\
["sinon", "npm:18.0.1"],\
- ["sinon-chai", "virtual:5066f1efd4c78a5ddf1dc175fd2039811919d09bb6f7aa5f2b46141ac45f2e6a675ff6260802f91c4f0e827a9565804d3931db690e7aa741774d17536ffb79fb#npm:3.7.0"]\
+ ["sinon-chai", "virtual:5066f1efd4c78a5ddf1dc175fd2039811919d09bb6f7aa5f2b46141ac45f2e6a675ff6260802f91c4f0e827a9565804d3931db690e7aa741774d17536ffb79fb#npm:3.7.0"],\
+ ["ts-protoc-gen", "npm:0.15.0"]\
],\
"linkType": "SOFT"\
}]\
@@ -12883,6 +12884,13 @@ const RAW_RUNTIME_STATE =
["google-protobuf", "npm:3.19.1"]\
],\
"linkType": "HARD"\
+ }],\
+ ["npm:3.21.4", {\
+ "packageLocation": "./.yarn/cache/google-protobuf-npm-3.21.4-48c47540d3-0d87fe8ef2.zip/node_modules/google-protobuf/",\
+ "packageDependencies": [\
+ ["google-protobuf", "npm:3.21.4"]\
+ ],\
+ "linkType": "HARD"\
}]\
]],\
["gopd", [\
@@ -21667,6 +21675,16 @@ const RAW_RUNTIME_STATE =
"linkType": "HARD"\
}]\
]],\
+ ["ts-protoc-gen", [\
+ ["npm:0.15.0", {\
+ "packageLocation": "./.yarn/cache/ts-protoc-gen-npm-0.15.0-4bb1076a19-de1d526b47.zip/node_modules/ts-protoc-gen/",\
+ "packageDependencies": [\
+ ["google-protobuf", "npm:3.21.4"],\
+ ["ts-protoc-gen", "npm:0.15.0"]\
+ ],\
+ "linkType": "HARD"\
+ }]\
+ ]],\
["tsconfck", [\
["npm:3.0.0", {\
"packageLocation": "./.yarn/cache/tsconfck-npm-3.0.0-f54c83f135-25789acde6.zip/node_modules/tsconfck/",\
diff --git a/.yarn/cache/google-protobuf-npm-3.21.4-48c47540d3-0d87fe8ef2.zip b/.yarn/cache/google-protobuf-npm-3.21.4-48c47540d3-0d87fe8ef2.zip
new file mode 100644
index 00000000000..e65708da399
Binary files /dev/null and b/.yarn/cache/google-protobuf-npm-3.21.4-48c47540d3-0d87fe8ef2.zip differ
diff --git a/.yarn/cache/ts-protoc-gen-npm-0.15.0-4bb1076a19-de1d526b47.zip b/.yarn/cache/ts-protoc-gen-npm-0.15.0-4bb1076a19-de1d526b47.zip
new file mode 100644
index 00000000000..f7a25dfe1b8
Binary files /dev/null and b/.yarn/cache/ts-protoc-gen-npm-0.15.0-4bb1076a19-de1d526b47.zip differ
diff --git a/AGENTS.md b/AGENTS.md
index 316d53322ad..26ede667802 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -78,7 +78,8 @@ Platform uses data contracts to define application data schemas:
- Run linters: `yarn lint`
## Coding Style & Naming Conventions
-- Follow `.editorconfig`: 2-space indent by default; 4 spaces for `*.rs` and
+- Follow `.editorconfig`: 2-space indent by default; 4 spaces for `*.rs`,
+ Swift and Kotlin (`*.swift`, `*.kt`, `*.kts`), and
`packages/swift-sdk/scripts/*.py`, preserving the existing Python script style.
Use LF, UTF‑8, and a final newline.
- JS/TS: ESLint (Airbnb/TypeScript rules via package configs). Use camelCase for variables/functions, PascalCase for classes; prefer kebab-case filenames within JS packages.
diff --git a/CHANGELOG.md b/CHANGELOG.md
index d4ae8423857..72d965e81cc 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,3 +1,217 @@
+## [4.2.0-beta.6](https://github.com/dashpay/platform/compare/v4.2.0-beta.5...v4.2.0-beta.6) (2026-09-28)
+
+
+### ⚠ BREAKING CHANGES
+
+* **platform:** a preallocated agreement source must fit a tree key (PV14) (#5123)
+* **sdk:** countOf and sumOf totals in the rule descriptors of the JS, Swift and Kotlin SDKs (#5121)
+* **platform:** refuse own-type totals on contested types and fail loudly on unread totals (PV14) (#5115)
+* **drive:** subscription filters match generated properties a transition leaves out (#5114)
+* **platform:** generatedFrom, string properties the platform generates with a system function (PV14) (#5099)
+* **platform:** countOf and sumOf totals from count and sum trees in propertyConstraints rules (PV14) (#5109)
+* **dpp:** propertyConstraints read empty objects as absent and follow $defs refs (PV14) (#5101)
+* **platform:** elected moderation windows may be 0 off mainnet, mainnet keeps one day (PV14) (#5108)
+* **platform:** ifThen, ifThenElse, notIn, min, max and abs in propertyConstraints rules (PV14) (#5100)
+* **swift-sdk:** new propertyConstraints read kinds and system reads in the Swift SDK and iOS example app (#5098)
+* **kotlin-sdk:** new propertyConstraints read kinds and system reads in the Kotlin SDK and Android example app (#5097)
+* **dpp:** size estimates of strings of 16384 or more characters no longer overflow (PV14) (#5086)
+* **platform:** startsWith and endsWith in propertyConstraints rules (PV14) (#5085)
+* **platform:** contains in propertyConstraints rules (PV14) (#5083)
+* **dpp:** report contracts refused by parser generation 3 as consensus errors (PV14) (#5076)
+* **platform:** creation, update and transfer times and heights in propertyConstraints rules (PV14) (#5078)
+* **platform:** string length, byte length and array count operands in propertyConstraints rules (PV14) (#5071)
+* **dpp:** propertyConstraints compare identifier properties that declare refersTo (PV14) (#5073)
+* **dpp:** accept a reordered entryPayload and refuse unruled keywords as incompatible (PV14) (#5074)
+* **dpp:** refuse token cost and unruled keyword changes on update with a consensus error (PV14) (#5069)
+* **platform:** $ownerId comparisons in propertyConstraints rules (PV14) (#5048)
+* **platform:** identifier comparisons in propertyConstraints rules (PV14) (#5047)
+* **platform:** string ifAbsent defaults in propertyConstraints rules (PV14) (#5046)
+* **platform:** compare two string properties in propertyConstraints rules (PV14) (#5045)
+
+### Features
+
+* **kotlin-sdk:** new propertyConstraints read kinds and system reads in the Kotlin SDK and Android example app ([#5097](https://github.com/dashpay/platform/issues/5097))
+* **platform:** $ownerId comparisons in propertyConstraints rules (PV14) ([#5048](https://github.com/dashpay/platform/issues/5048))
+* **platform:** compare two string properties in propertyConstraints rules (PV14) ([#5045](https://github.com/dashpay/platform/issues/5045))
+* **platform:** contains in propertyConstraints rules (PV14) ([#5083](https://github.com/dashpay/platform/issues/5083))
+* **platform:** countOf and sumOf totals from count and sum trees in propertyConstraints rules (PV14) ([#5109](https://github.com/dashpay/platform/issues/5109))
+* **platform:** creation, update and transfer times and heights in propertyConstraints rules (PV14) ([#5078](https://github.com/dashpay/platform/issues/5078))
+* **platform:** elected moderation windows may be 0 off mainnet, mainnet keeps one day (PV14) ([#5108](https://github.com/dashpay/platform/issues/5108))
+* **platform:** generatedFrom, string properties the platform generates with a system function (PV14) ([#5099](https://github.com/dashpay/platform/issues/5099))
+* **platform:** identifier comparisons in propertyConstraints rules (PV14) ([#5047](https://github.com/dashpay/platform/issues/5047))
+* **platform:** ifThen, ifThenElse, notIn, min, max and abs in propertyConstraints rules (PV14) ([#5100](https://github.com/dashpay/platform/issues/5100))
+* **platform:** startsWith and endsWith in propertyConstraints rules (PV14) ([#5085](https://github.com/dashpay/platform/issues/5085))
+* **platform:** string ifAbsent defaults in propertyConstraints rules (PV14) ([#5046](https://github.com/dashpay/platform/issues/5046))
+* **platform:** string length, byte length and array count operands in propertyConstraints rules (PV14) ([#5071](https://github.com/dashpay/platform/issues/5071))
+* **sdk:** countOf and sumOf totals in the rule descriptors of the JS, Swift and Kotlin SDKs ([#5121](https://github.com/dashpay/platform/issues/5121))
+* **sdk:** propertyConstraints discovery and pre-check in the JS SDK ([#5051](https://github.com/dashpay/platform/issues/5051))
+* **sdk:** propertyConstraints rules and pre-check in the Kotlin SDK and Android example app ([#5066](https://github.com/dashpay/platform/issues/5066))
+* **sdk:** propertyConstraints rules and pre-check in the Swift SDK and iOS example app ([#5064](https://github.com/dashpay/platform/issues/5064))
+* **swift-sdk:** new propertyConstraints read kinds and system reads in the Swift SDK and iOS example app ([#5098](https://github.com/dashpay/platform/issues/5098))
+
+
+### Bug Fixes
+
+* **ci:** build release clients natively on unprivileged runners
+* **ci:** isolate release runners from PR build state
+* **dpp:** accept a reordered entryPayload and refuse unruled keywords as incompatible (PV14) ([#5074](https://github.com/dashpay/platform/issues/5074))
+* **dpp:** parse nested required and transient entries by prefix ([#5050](https://github.com/dashpay/platform/issues/5050))
+* **dpp:** pass the contract's $defs to the countOf and sumOf key enum check ([#5110](https://github.com/dashpay/platform/issues/5110))
+* **dpp:** propertyConstraints compare identifier properties that declare refersTo (PV14) ([#5073](https://github.com/dashpay/platform/issues/5073))
+* **dpp:** propertyConstraints read empty objects as absent and follow $defs refs (PV14) ([#5101](https://github.com/dashpay/platform/issues/5101))
+* **dpp:** refuse token cost and unruled keyword changes on update with a consensus error (PV14) ([#5069](https://github.com/dashpay/platform/issues/5069))
+* **dpp:** report contracts refused by parser generation 3 as consensus errors (PV14) ([#5076](https://github.com/dashpay/platform/issues/5076))
+* **dpp:** restore the shipped order of basic consensus errors ([#5053](https://github.com/dashpay/platform/issues/5053))
+* **dpp:** size estimates of strings of 16384 or more characters no longer overflow (PV14) ([#5086](https://github.com/dashpay/platform/issues/5086))
+* **drive-abci:** finalize a block accepted in an earlier round after a later proposal was refused ([#5081](https://github.com/dashpay/platform/issues/5081))
+* **drive-abci:** sign a locked block when a later proposal left no execution context ([#5079](https://github.com/dashpay/platform/issues/5079))
+* **drive-abci:** sign vote extensions only for blocks this node accepted ([#5084](https://github.com/dashpay/platform/issues/5084))
+* **drive:** subscription filters match generated properties a transition leaves out ([#5114](https://github.com/dashpay/platform/issues/5114))
+* **platform:** a preallocated agreement source must fit a tree key (PV14) ([#5123](https://github.com/dashpay/platform/issues/5123))
+* **platform:** refuse own-type totals on contested types and fail loudly on unread totals (PV14) ([#5115](https://github.com/dashpay/platform/issues/5115))
+* **release:** require all generated clients in packed archives
+* **sdk:** bound each DAPI request attempt, including the response body ([#4973](https://github.com/dashpay/platform/issues/4973))
+* **sdk:** consensus errors reach JS with their code ([#5112](https://github.com/dashpay/platform/issues/5112))
+* **sdk:** consensus errors reach Swift and Kotlin apps with their code ([#5116](https://github.com/dashpay/platform/issues/5116))
+* **swift-sdk:** documentTransfer handles a missing document and signs once ([#5120](https://github.com/dashpay/platform/issues/5120))
+* **swift-sdk:** free FFI errors in state-transition wrappers ([#5117](https://github.com/dashpay/platform/issues/5117))
+* **swift-sdk:** take migration copies out of WAL mode
+* **wasm-sdk:** keep StateTransitionResult.ownerBalance exact in JSON ([#5059](https://github.com/dashpay/platform/issues/5059))
+* **wasm-sdk:** leave price tier validation to rs-dpp ([#5058](https://github.com/dashpay/platform/issues/5058))
+
+
+### Miscellaneous Chores
+
+* **swift-sdk:** freeze App Store schema 3.0.0
+
+
+### Continuous Integration
+
+* bootstrap PR-first runner images on v4.2-dev
+* reconcile rootless runner workflows with v4.2-dev, closes [#4702](https://github.com/dashpay/platform/issues/4702)
+
+
+### Code Refactoring
+
+* **dpp:** restore shipped registration_cost v1 index parsing ([#5055](https://github.com/dashpay/platform/issues/5055))
+* **drive:** create once-per-identity claim trees in insert_contract v2 ([#5056](https://github.com/dashpay/platform/issues/5056))
+* **platform:** fold DRIVE_ABCI_QUERY_VERSIONS_V3 into V2 ([#5057](https://github.com/dashpay/platform/issues/5057))
+* **platform:** import instead of inline crate paths in 4.1 and 4.2 code ([#5065](https://github.com/dashpay/platform/issues/5065))
+
+
+### Documentation
+
+* add a contract keywords reference page to the book ([#5067](https://github.com/dashpay/platform/issues/5067))
+* encrypt the 69-byte compact xpub in the contact-request guide ([#5087](https://github.com/dashpay/platform/issues/5087))
+* give every contract keyword its own chapter in the book ([#5075](https://github.com/dashpay/platform/issues/5075))
+* list the complete contract language in the keywords overview ([#5082](https://github.com/dashpay/platform/issues/5082))
+* **platform:** add Parameters and Returns sections to 4.1 and 4.2 dispatchers ([#5063](https://github.com/dashpay/platform/issues/5063))
+* **platform:** document the genesis protocol version exception and Swift/Kotlin indentation ([#5061](https://github.com/dashpay/platform/issues/5061))
+* **platform:** say why in-place edits to shipped generations are inert ([#5054](https://github.com/dashpay/platform/issues/5054))
+* remove committed working specs and plans ([#5060](https://github.com/dashpay/platform/issues/5060))
+* **swift-sdk:** say the migration copy is switched out of WAL mode
+
+
+### Tests
+
+* **drive:** pin that a cached contract read after an in-block update bills like a cold read ([#5052](https://github.com/dashpay/platform/issues/5052))
+* follow the test conventions in tests added in 4.1 and 4.2 ([#5062](https://github.com/dashpay/platform/issues/5062))
+* **sdk:** ifThen and ifThenElse rules through rs-sdk-ffi and the Swift and Kotlin SDKs ([#5106](https://github.com/dashpay/platform/issues/5106))
+
+## [4.2.0-beta.5](https://github.com/dashpay/platform/compare/v4.2.0-beta.4...v4.2.0-beta.5) (2026-09-27)
+
+
+### ⚠ BREAKING CHANGES
+
+* **platform:** string equality for enums in propertyConstraints rules (PV14) (#5042)
+* **platform:** pay document ttl storage fees to the epochs the documents live in (PV14) (#5033)
+* **platform:** contenders state the most they pay and are charged the join price (PV14) (#5039)
+* **platform:** boolean operands in propertyConstraints rules (PV14) (#5040)
+* **platform:** in, value membership in propertyConstraints rules (PV14) (#5038)
+* **platform:** present and absent tests in propertyConstraints rules (PV14) (#5037)
+* **platform:** anyOf, allOf and not in propertyConstraints rules (PV14) (#5036)
+* **platform:** a contender's fund doubles for every 50 contenders a contest holds past 250 (PV14) (#5034)
+* **drive-abci:** cap a contest at 1,000 contenders and tally every one (PV14) (#5029)
+* **platform:** documents with a time to live, deleted by the platform (PV14) (#5007)
+* **drive:** an evonode's token claim covers only the epochs it read (PV14) (#5015)
+* **drive-abci:** claw a storage refund back from the epochs it was priced for (PV14) (#5013)
+* **drive-abci:** refuse bytes after a state transition (PV14) (#5011)
+* **drive-abci:** refuse a masternode vote for an identity that is not a contender (PV14) (#5002)
+* **drive:** delete an ended vote poll end date only once none of its polls remain (PV14) (#4996)
+* **drive-abci:** refuse a token mint or direct purchase past the i64::MAX supply ceiling (PV14) (#5000)
+* **drive-abci:** allow contested documents before epoch 4 (#4995)
+* **drive:** merge repeated writes of one balance in a batch so an action fee no longer loses a purchase price (#4987)
+* **platform:** credit repaid identity debt to the processing fee pool (#4985)
+* **dpp:** refuse immutableAllowSetting on a deletableDocument reference (PV14) (#4983)
+* **drive-abci:** re-check a contract reference's owner requirement on every replace of a transferable document (PV14) (#4982)
+* **drive-abci:** refuse a $creatorId key reference on a document without a creator id (PV14) (#4984)
+
+### Features
+
+* **drive-abci:** allow contested documents before epoch 4 ([#4995](https://github.com/dashpay/platform/issues/4995))
+* **platform:** a contender's fund doubles for every 50 contenders a contest holds past 250 (PV14) ([#5034](https://github.com/dashpay/platform/issues/5034))
+* **platform:** anyOf, allOf and not in propertyConstraints rules (PV14) ([#5036](https://github.com/dashpay/platform/issues/5036))
+* **platform:** boolean operands in propertyConstraints rules (PV14) ([#5040](https://github.com/dashpay/platform/issues/5040))
+* **platform:** documents with a time to live, deleted by the platform (PV14) ([#5007](https://github.com/dashpay/platform/issues/5007))
+* **platform:** in, value membership in propertyConstraints rules (PV14) ([#5038](https://github.com/dashpay/platform/issues/5038))
+* **platform:** pay document ttl storage fees to the epochs the documents live in (PV14) ([#5033](https://github.com/dashpay/platform/issues/5033))
+* **platform:** present and absent tests in propertyConstraints rules (PV14) ([#5037](https://github.com/dashpay/platform/issues/5037))
+* **platform:** string equality for enums in propertyConstraints rules (PV14) ([#5042](https://github.com/dashpay/platform/issues/5042))
+* **rs-dapi:** refuse shielded broadcasts from addresses that keep sending invalid proofs ([#5001](https://github.com/dashpay/platform/issues/5001))
+
+
+### Bug Fixes
+
+* **dpp:** refuse immutableAllowSetting on a deletableDocument reference (PV14) ([#4983](https://github.com/dashpay/platform/issues/4983))
+* **drive-abci:** cap a contest at 1,000 contenders and tally every one (PV14) ([#5029](https://github.com/dashpay/platform/issues/5029))
+* **drive-abci:** check the address input limit before verifying witnesses ([#5005](https://github.com/dashpay/platform/issues/5005))
+* **drive-abci:** claw a storage refund back from the epochs it was priced for (PV14) ([#5013](https://github.com/dashpay/platform/issues/5013))
+* **drive-abci:** re-check a contract reference's owner requirement on every replace of a transferable document (PV14) ([#4982](https://github.com/dashpay/platform/issues/4982))
+* **drive-abci:** refuse a $creatorId key reference on a document without a creator id (PV14) ([#4984](https://github.com/dashpay/platform/issues/4984))
+* **drive-abci:** refuse a masternode vote for an identity that is not a contender (PV14) ([#5002](https://github.com/dashpay/platform/issues/5002))
+* **drive-abci:** refuse a token mint or direct purchase past the i64::MAX supply ceiling (PV14) ([#5000](https://github.com/dashpay/platform/issues/5000))
+* **drive-abci:** refuse bytes after a state transition (PV14) ([#5011](https://github.com/dashpay/platform/issues/5011))
+* **drive-abci:** sign and verify vote extensions of a block accepted in another round ([#5028](https://github.com/dashpay/platform/issues/5028))
+* **drive-abci:** verify vote extensions against the withdrawals of their own round ([#5010](https://github.com/dashpay/platform/issues/5010))
+* **drive:** an evonode's token claim covers only the epochs it read (PV14) ([#5015](https://github.com/dashpay/platform/issues/5015))
+* **drive:** delete an ended vote poll end date only once none of its polls remain (PV14) ([#4996](https://github.com/dashpay/platform/issues/4996))
+* **drive:** merge repeated writes of one balance in a batch so an action fee no longer loses a purchase price ([#4987](https://github.com/dashpay/platform/issues/4987))
+* **drive:** read stored group actions without the proof decoding budget ([#5006](https://github.com/dashpay/platform/issues/5006))
+* **platform:** contenders state the most they pay and are charged the join price (PV14) ([#5039](https://github.com/dashpay/platform/issues/5039))
+* **platform:** credit repaid identity debt to the processing fee pool ([#4985](https://github.com/dashpay/platform/issues/4985))
+* **rs-sdk-ffi:** stop probing google.com and testnet quorums on every SDK build ([#5008](https://github.com/dashpay/platform/issues/5008))
+* **sdk:** don't panic in DapiClient::new on an empty address list ([#4964](https://github.com/dashpay/platform/issues/4964))
+
+
+### Performance Improvements
+
+* **drive-abci:** read shielded encrypted notes in one chunk-aligned range read ([#5030](https://github.com/dashpay/platform/issues/5030))
+* **drive:** build batch deletes without copying the pending batch ([#5004](https://github.com/dashpay/platform/issues/5004))
+
+
+### Tests
+
+* **drive:** keep setup_drive's temp directory alive while the drive is open ([#5003](https://github.com/dashpay/platform/issues/5003))
+* **swift-sdk:** run SDKMethodTests against the offline mock SDK instead of live testnet ([#5009](https://github.com/dashpay/platform/issues/5009))
+
+
+### Miscellaneous Chores
+
+* open every pr-description output with a basic explanation section ([#5031](https://github.com/dashpay/platform/issues/5031))
+* sync the pr-description skill with the PR template and title check ([#5032](https://github.com/dashpay/platform/issues/5032))
+
+
+### Continuous Integration
+
+* build release SDKs and NPM packages on self-hosted runners ([#4562](https://github.com/dashpay/platform/issues/4562))
+* re-pin PR Hygiene ([#4975](https://github.com/dashpay/platform/issues/4975))
+* re-pin PR Hygiene ([#4979](https://github.com/dashpay/platform/issues/4979))
+* re-pin PR Hygiene ([#4989](https://github.com/dashpay/platform/issues/4989))
+* re-pin PR Hygiene ([#5016](https://github.com/dashpay/platform/issues/5016))
+* re-pin PR Hygiene ([#5023](https://github.com/dashpay/platform/issues/5023))
+* re-pin PR Hygiene and wake it on a hand-over ([#5020](https://github.com/dashpay/platform/issues/5020))
+* **release:** raise NPM and Swift SDK release job timeouts ([#4974](https://github.com/dashpay/platform/issues/4974))
+
## [4.2.0-beta.4](https://github.com/dashpay/platform/compare/v4.2.0-beta.3...v4.2.0-beta.4) (2026-09-24)
diff --git a/Cargo.lock b/Cargo.lock
index a7f35d1b352..e7d97484fc3 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -159,7 +159,7 @@ checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c"
[[package]]
name = "app-connect-contract"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"base58",
"platform-value",
@@ -1055,7 +1055,7 @@ dependencies = [
[[package]]
name = "check-features"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"toml 0.8.23",
]
@@ -1543,7 +1543,7 @@ dependencies = [
[[package]]
name = "dapi-grpc"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"dash-platform-macros",
"futures-core",
@@ -1630,7 +1630,7 @@ dependencies = [
[[package]]
name = "dash-async"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"futures",
"thiserror 2.0.18",
@@ -1641,7 +1641,7 @@ dependencies = [
[[package]]
name = "dash-context-provider"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"dash-async",
"dpp",
@@ -1672,7 +1672,7 @@ dependencies = [
[[package]]
name = "dash-platform-balance-checker"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"anyhow",
"clap",
@@ -1687,7 +1687,7 @@ dependencies = [
[[package]]
name = "dash-platform-macros"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"heck 0.5.0",
"quote",
@@ -1696,7 +1696,7 @@ dependencies = [
[[package]]
name = "dash-platform-queries"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"dapi-grpc",
"dash-context-provider",
@@ -1713,7 +1713,7 @@ dependencies = [
[[package]]
name = "dash-sdk"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"arc-swap",
"assert_matches",
@@ -1858,7 +1858,7 @@ dependencies = [
[[package]]
name = "dashpay-contract"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"platform-value",
"platform-version",
@@ -1868,7 +1868,7 @@ dependencies = [
[[package]]
name = "data-contracts"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"app-connect-contract",
"base58",
@@ -2073,7 +2073,7 @@ dependencies = [
[[package]]
name = "document-history-contract"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"platform-value",
"platform-version",
@@ -2095,7 +2095,7 @@ checksum = "1435fa1053d8b2fbbe9be7e97eca7f33d37b28409959813daefc1446a14247f1"
[[package]]
name = "dpns-contract"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"platform-value",
"platform-version",
@@ -2105,7 +2105,7 @@ dependencies = [
[[package]]
name = "dpp"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"anyhow",
"assert_matches",
@@ -2164,7 +2164,7 @@ dependencies = [
[[package]]
name = "dpp-json-convertible-derive"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"proc-macro2",
"quote",
@@ -2173,7 +2173,7 @@ dependencies = [
[[package]]
name = "drive"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"arc-swap",
"assert_matches",
@@ -2216,7 +2216,7 @@ dependencies = [
[[package]]
name = "drive-abci"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"arc-swap",
"assert_matches",
@@ -2278,7 +2278,7 @@ dependencies = [
[[package]]
name = "drive-proof-verifier"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"dapi-grpc",
"dash-context-provider",
@@ -4080,7 +4080,7 @@ dependencies = [
[[package]]
name = "json-schema-compatibility-validator"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"assert_matches",
"json-patch",
@@ -4223,7 +4223,7 @@ dependencies = [
[[package]]
name = "keyword-search-contract"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"base58",
"platform-value",
@@ -4414,7 +4414,7 @@ dependencies = [
[[package]]
name = "masternode-reward-shares-contract"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"platform-value",
"platform-version",
@@ -4615,7 +4615,7 @@ dependencies = [
[[package]]
name = "moderation-charters-contract"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"base58",
"platform-value",
@@ -5198,7 +5198,7 @@ checksum = "19f132c84eca552bf34cab8ec81f1c1dcc229b811638f9d283dceabe58c5569e"
[[package]]
name = "platform-encryption"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"aes",
"cbc",
@@ -5210,7 +5210,7 @@ dependencies = [
[[package]]
name = "platform-serialization"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"grovedb-bincode",
"platform-version",
@@ -5218,7 +5218,7 @@ dependencies = [
[[package]]
name = "platform-serialization-derive"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"proc-macro2",
"quote",
@@ -5228,7 +5228,7 @@ dependencies = [
[[package]]
name = "platform-value"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"base64 0.22.1",
"bs58",
@@ -5247,7 +5247,7 @@ dependencies = [
[[package]]
name = "platform-value-convertible"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"quote",
"syn 2.0.117",
@@ -5255,7 +5255,7 @@ dependencies = [
[[package]]
name = "platform-version"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"grovedb-bincode",
"grovedb-version",
@@ -5265,7 +5265,7 @@ dependencies = [
[[package]]
name = "platform-versioning"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"proc-macro2",
"quote",
@@ -5274,7 +5274,7 @@ dependencies = [
[[package]]
name = "platform-wallet"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"arc-swap",
"async-trait",
@@ -5314,7 +5314,7 @@ dependencies = [
[[package]]
name = "platform-wallet-ffi"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"anyhow",
"async-trait",
@@ -5342,7 +5342,7 @@ dependencies = [
[[package]]
name = "platform-wallet-storage"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"apple-native-keyring-store",
"argon2",
@@ -6359,7 +6359,7 @@ dependencies = [
[[package]]
name = "rs-dapi"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"async-trait",
"axum 0.8.9",
@@ -6409,7 +6409,7 @@ dependencies = [
[[package]]
name = "rs-dapi-client"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"backon",
"chrono",
@@ -6417,9 +6417,11 @@ dependencies = [
"futures",
"getrandom 0.2.17",
"gloo-timers",
+ "h2",
"hex",
"http",
"http-serde",
+ "hyper-util",
"lru",
"rand 0.8.6",
"serde",
@@ -6428,13 +6430,14 @@ dependencies = [
"thiserror 2.0.18",
"tokio",
"tonic-web-wasm-client",
+ "tower 0.5.3",
"tracing",
"wasm-bindgen-futures",
]
[[package]]
name = "rs-dash-event-bus"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"metrics",
"tokio",
@@ -6467,7 +6470,7 @@ dependencies = [
[[package]]
name = "rs-sdk-ffi"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"async-trait",
"bs58",
@@ -6501,7 +6504,7 @@ dependencies = [
[[package]]
name = "rs-sdk-trusted-context-provider"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"arc-swap",
"dash-async",
@@ -6521,7 +6524,7 @@ dependencies = [
[[package]]
name = "rs-unified-sdk-ffi"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"dash-network",
"key-wallet-ffi",
@@ -6531,7 +6534,7 @@ dependencies = [
[[package]]
name = "rs-unified-sdk-jni"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"android_logger",
"dash-network",
@@ -7273,7 +7276,7 @@ checksum = "e3a9fe34e3e7a50316060351f37187a3f546bce95496156754b601a5fa71b76e"
[[package]]
name = "simple-signer"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"async-trait",
"base64 0.22.1",
@@ -7410,7 +7413,7 @@ dependencies = [
[[package]]
name = "strategy-tests"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"dpp",
"drive",
@@ -7812,7 +7815,7 @@ checksum = "1f3ccbac311fea05f86f61904b462b55fb3df8837a366dfc601a0161d0532f20"
[[package]]
name = "token-history-contract"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"platform-value",
"platform-version",
@@ -8655,7 +8658,7 @@ dependencies = [
[[package]]
name = "wallet-utils-contract"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"platform-value",
"platform-version",
@@ -8798,7 +8801,7 @@ checksum = "a8145dd1593bf0fb137dbfa85b8be79ec560a447298955877804640e40c2d6ea"
[[package]]
name = "wasm-dpp"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"anyhow",
"async-trait",
@@ -8822,7 +8825,7 @@ dependencies = [
[[package]]
name = "wasm-dpp2"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"anyhow",
"async-trait",
@@ -8841,7 +8844,7 @@ dependencies = [
[[package]]
name = "wasm-drive-verify"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"base64 0.22.1",
"bs58",
@@ -8896,7 +8899,7 @@ dependencies = [
[[package]]
name = "wasm-sdk"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"base64 0.22.1",
"bip39",
@@ -9407,7 +9410,7 @@ dependencies = [
[[package]]
name = "withdrawals-contract"
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
dependencies = [
"num_enum 0.5.11",
"platform-value",
diff --git a/Cargo.toml b/Cargo.toml
index e8727211b7b..030eaee0c49 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -149,5 +149,5 @@ opt-level = 3
[workspace.package]
-version = "4.2.0-beta.4"
+version = "4.2.0-beta.6"
rust-version = "1.98"
diff --git a/book/src/SUMMARY.md b/book/src/SUMMARY.md
index c41f5122fa5..abc3286f5e0 100644
--- a/book/src/SUMMARY.md
+++ b/book/src/SUMMARY.md
@@ -34,6 +34,7 @@
- [Fee System Overview](fees/overview.md)
- [Platform Address Fees](fees/platform-address-fees.md)
- [Shielded Transaction Fees](fees/shielded-fees.md)
+- [What a Document Costs](fees/document-cost.md)
# Error Handling
@@ -57,10 +58,46 @@
- [Contract Groups](data-model/contract-groups.md)
- [Contract Moderation](data-model/contract-moderation.md)
- [Documents](data-model/documents.md)
+- [Document Time To Live](data-model/document-ttl.md)
- [Contested Documents](data-model/contested-documents.md)
- [Identities](data-model/identities.md)
- [Key Budgets and Expiry](data-model/key-limits.md)
+# Contract Keywords
+
+- [Overview](contract-keywords.md)
+- [Document Shape](contract-keywords/document-shape.md)
+- [Property Schemas](contract-keywords/property-schemas.md)
+- [Typed Arrays](contract-keywords/typed-arrays.md)
+- [System Properties](contract-keywords/system-properties.md)
+- [requiredSince](contract-keywords/required-since.md)
+- [transient](contract-keywords/transient.md)
+- [Mutability](contract-keywords/mutability.md)
+- [Deletion](contract-keywords/deletion.md)
+- [Time To Live (ttl)](contract-keywords/ttl.md)
+- [Creation, Transfers and Trading](contract-keywords/ownership-and-trading.md)
+- [History](contract-keywords/history.md)
+- [Signing and Keys](contract-keywords/signing-keys.md)
+- [References (refersTo)](contract-keywords/refers-to.md)
+ - [Lookups](contract-keywords/refers-to-lookup.md)
+ - [Expressions](contract-keywords/refers-to-expressions.md)
+ - [List Elements](contract-keywords/refers-to-list-element.md)
+ - [Writer and Creator References](contract-keywords/owner-refers-to.md)
+- [distinctFrom](contract-keywords/distinct-from.md)
+- [maxBytes](contract-keywords/max-bytes.md)
+- [generatedFrom](contract-keywords/generated-from.md)
+- [encryptedFor](contract-keywords/encrypted-for.md)
+- [propertyConstraints](contract-keywords/property-constraints.md)
+- [Token Costs (tokenCost)](contract-keywords/token-cost.md)
+- [Action Fees (actionFees)](contract-keywords/action-fees.md)
+- [Indexes (indices)](contract-keywords/indexes.md)
+ - [Contested Indexes](contract-keywords/contested.md)
+ - [Counts, Sums and Averages](contract-keywords/aggregates.md)
+ - [Ranked Indexes](contract-keywords/ranked.md)
+ - [Time-Range Indexes](contract-keywords/time-range.md)
+ - [Index-Only Types](contract-keywords/index-only.md)
+- [Contract-Level Keys and config](contract-keywords/contract-config.md)
+
# Drive
- [The GroveDB Structure](drive/grovedb-structure.md)
diff --git a/book/src/architecture/component-pipeline.md b/book/src/architecture/component-pipeline.md
index c292e1a908f..e7cc066216a 100644
--- a/book/src/architecture/component-pipeline.md
+++ b/book/src/architecture/component-pipeline.md
@@ -94,7 +94,9 @@ where
```
The `FullAbciApplication` struct wires everything together. It holds a reference
-to `Platform`, a GroveDB transaction, and the current block execution context:
+to `Platform`, a GroveDB transaction, the current block execution context, and
+the withdrawal transactions of every proposal accepted at the current height,
+which vote extensions are verified against:
```rust
// From packages/rs-drive-abci/src/abci/app/full.rs
@@ -102,6 +104,7 @@ pub struct FullAbciApplication<'a, C> {
pub platform: &'a Platform,
pub transaction: RwLock>>,
pub block_execution_context: RwLock>,
+ pub unsigned_withdrawal_txs_by_round: RwLock,
}
```
diff --git a/book/src/contract-keywords.md b/book/src/contract-keywords.md
new file mode 100644
index 00000000000..eb81f0d1431
--- /dev/null
+++ b/book/src/contract-keywords.md
@@ -0,0 +1,317 @@
+# Contract Keywords
+
+A data contract describes its documents with a JSON schema per document type, and Platform reads a set of keywords in those schemas: some from JSON Schema, most of its own. The chapters of this part take each keyword, or a small group that works together, and say what it does, how to write it, what is checked when a contract is registered and when a document is written, what a later contract update may do with it, and which errors it produces. The chapters of the Data Model and Drive parts explain the internals behind them and are linked from each chapter.
+
+Everything here follows the document meta-schema of protocol version 14, `packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json`. Every document type schema is validated against it when a contract is registered or updated, and the parser (`try_from_schema`) checks the rules a JSON schema cannot express. When a chapter and the meta-schema disagree, the meta-schema is right and the chapter is out of date.
+
+## Where keywords go
+
+A contract's `documentSchemas` maps each document type name to its schema. Keywords sit at three levels:
+
+```json
+"post": {
+ "type": "object",
+ "documentsMutable": true,
+ "canBeDeleted": true,
+ "indices": [
+ { "name": "byOwner", "properties": [{ "$ownerId": "asc" }, { "$createdAt": "asc" }] }
+ ],
+ "properties": {
+ "text": { "type": "string", "maxLength": 280, "maxBytes": 560, "position": 0 },
+ "replyTo": {
+ "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "refersTo": { "type": "deletableDocument", "documentType": "post" },
+ "position": 1
+ }
+ },
+ "required": ["$createdAt", "text"],
+ "additionalProperties": false
+}
+```
+
+- **Document type keywords** sit at the top of the schema (`documentsMutable`, `canBeDeleted`, `indices`, `required`). They say what may happen to a document of the type and who may do it.
+- **Index keywords** sit inside an entry of `indices` (`name`, `properties`).
+- **Property keywords** sit inside a property's schema (`type`, `maxLength`, `maxBytes`, `position`, `refersTo`). Most are ordinary JSON Schema; the rest are Platform's own.
+
+The contract around the document types has keys of its own, and a `config` object: see [Contract-Level Keys and config](contract-keywords/contract-config.md).
+
+## Reading the chapters
+
+Each chapter opens with a short table for each keyword:
+
+- **Since** is the first protocol version at which the keyword can be used.
+- **On update** is what a contract update may do with the keyword on a document type that already exists. *Fixed* means adding, removing and changing it are all refused. A document type the update adds may use any keyword, as a new contract may.
+- **Errors** are consensus errors, written `ErrorName` (code). See [Error Codes](error-handling/error-codes.md) for the code ranges.
+
+A contract update that breaks an update rule is refused with one of two errors, depending on which check catches it. `DocumentTypeUpdateError` (40212) comes from the comparison of the parsed document types, which judges flags such as `documentsMutable` by their meaning. `IncompatibleDocumentTypeSchemaError` (10246) comes from the comparison of the two JSON schemas, which judges property keywords such as `refersTo` or `maxLength` by their text. Top-level `required` and `indices` have errors of their own (10276 and 10217). Because the schema comparison reads text, an edit that changes how a keyword is written but not what it means, such as writing out a default or switching to the `documentsAverageable` shorthand, is refused with 10246.
+
+## Protocol versions
+
+The document meta-schema has changed three times:
+
+| Meta-schema | Protocol versions | What it added |
+|---|---|---|
+| v0 | 1 to 11 | The original keywords. A document type key the meta-schema did not know was ignored. |
+| v1 | 12 | Unknown document type keys are refused. The count, sum and average keywords. |
+| v2 | 13 | `keepsTransferHistory`, `keepsPurchaseHistory`, `keepsPricingHistory`. |
+| v3 | 14 | References, typed arrays, `requiredSince`, `immutable`, `ttl`, `propertyConstraints`, `actionFees`, moderation deletion, ranked and time-range indexes, index-only types, and the rest marked 14 in these chapters. |
+
+Most keywords of v0 took effect at protocol version 1. The exceptions are `tokenCost` (9) and the index keyword `countable` (12).
+
+## The complete language
+
+Every key a contract can write, grouped by where it goes. **Since** is the protocol version from which the key works. **Read more** links to the key's section in this part and, where there is one, to the chapter with the internals. A dotted name such as `tokenCost..amount` is a key written inside the ones before it.
+
+### The contract
+
+| Key | Takes | What it does | Since | Read more |
+|---|---|---|---|---|
+| `$formatVersion` | `"0"` or `"1"` | The contract's serialization format. `"1"`, the default from 9, carries `groups`, `tokens`, `keywords`, `description` and the timestamps. | 1 | [Contract keys](contract-keywords/contract-config.md#contract-keys) |
+| `id` | identifier | The contract's id: a hash of `ownerId` and the identity nonce of the create transition. | 1 | [Contract keys](contract-keywords/contract-config.md#contract-keys) |
+| `ownerId` | identifier | The identity that registers the contract, and the only one that may update it. | 1 | [Contract keys](contract-keywords/contract-config.md#contract-keys) |
+| `version` | integer | 1 at creation; every update raises it by exactly one. | 1 | [Contract keys](contract-keywords/contract-config.md#contract-keys) |
+| `config` | object | Contract-wide settings, [below](#config). | 1 | [config](contract-keywords/contract-config.md#config) |
+| `documentSchemas` | object of document types | The document types by name, each written with the [document type keys](#document-type). | 1 | [documentSchemas](contract-keywords/contract-config.md#documentschemas) |
+| `schemaDefs` | object | Definitions any property may point at with `$ref`. | 1 | [Document Shape](contract-keywords/document-shape.md#schema-and-defs) |
+| `groups` | object | Sets of identities, each member with a voting power, whose approval some token actions need. | 9 | [Data Contracts](data-model/data-contracts.md#what-v1-added) |
+| `tokens` | object | The contract's tokens, by position. Their configuration is not covered in this part. | 9 | [Data Contracts](data-model/data-contracts.md#what-v1-added) · [Creating a Basic Token](evo-sdk/tutorials/basic-token.md) |
+| `keywords` | up to 50 strings of 3 to 50 bytes | Search keywords, for the keyword search contract. | 9 | [keywords and description](contract-keywords/contract-config.md#keywords-and-description) |
+| `description` | string of 3 to 100 bytes | A short description, for the keyword search contract. | 9 | [keywords and description](contract-keywords/contract-config.md#keywords-and-description) |
+| `createdAt`, `updatedAt`, `createdAtBlockHeight`, `updatedAtBlockHeight`, `createdAtEpoch`, `updatedAtEpoch` | numbers | When the contract was created and last updated. Set by the platform, never written. | 9 | [Contract keys](contract-keywords/contract-config.md#contract-keys) |
+| `contractGroup` | `{ "admins", "name", "description" }` | On the create transition, beside the contract: registers a contract group, a set of contracts the signer owns, with up to 16 `admins`, a `name` of 1 to 64 characters and a `description` of 1 to 256. | 14 | [Contract Groups](data-model/contract-groups.md) |
+| `contractGroupMemberships` | up to 16 `{ "contractGroupId", "member" }` | On the create transition: enrols the new contract (`"contract"`), one of its document types (`{ "documentType": ... }`) or one of its tokens (`{ "token": ... }`) in contract groups. | 14 | [Contract Groups](data-model/contract-groups.md) |
+
+### config
+
+| Key | Takes | What it does | Since | Read more |
+|---|---|---|---|---|
+| `canBeDeleted` | boolean, default `false` | Whether the contract may ever be deleted. No transition deletes a contract today. | 1 | [canBeDeleted](contract-keywords/contract-config.md#canbedeleted) |
+| `readonly` | boolean, default `false` | `true`: the contract can never be updated. | 1 | [readonly](contract-keywords/contract-config.md#readonly) |
+| `keepsHistory` | boolean, default `false` | Drive keeps every version of the contract. | 1 | [keepsHistory](contract-keywords/contract-config.md#keepshistory) |
+| `documentsKeepHistoryContractDefault` | boolean, default `false` | `documentsKeepHistory` for a document type that does not say. | 1 | [Document type defaults](contract-keywords/contract-config.md#document-type-defaults) |
+| `documentsMutableContractDefault` | boolean, default `true` | `documentsMutable` for a document type that does not say. | 1 | [Document type defaults](contract-keywords/contract-config.md#document-type-defaults) |
+| `documentsCanBeDeletedContractDefault` | boolean, default `true` | `canBeDeleted` for a document type that does not say. | 1 | [Document type defaults](contract-keywords/contract-config.md#document-type-defaults) |
+| `requiresIdentityEncryptionBoundedKey`, `requiresIdentityDecryptionBoundedKey` | `0` unique, `1` multiple, `2` multiple with a pointer to the latest | Lets identities bind encryption or decryption keys to the whole contract, and says how they are kept. | 1 | [Bounded key requirements](contract-keywords/contract-config.md#bounded-key-requirements) · [Contract Bounds](sdk/identity-keys.md#contract-bounds) |
+| `sizedIntegerTypes` | boolean, default `true` | Stores each integer in the smallest width its bounds allow, instead of 8 bytes. | 9 | [sizedIntegerTypes](contract-keywords/contract-config.md#sizedintegertypes) |
+| `moderation` | object | Makes the contract moderated: which lists it keeps and who moderates. | 14 | [moderation](contract-keywords/contract-config.md#moderation) · [Contract Moderation](data-model/contract-moderation.md) |
+| `moderation.banlist`, `.suspensions`, `.warnings` | boolean, default `false` | Keeps a banlist, a suspension list, a warning list. A banned or suspended identity cannot act on the contract's documents; a warning bars nothing. | 14 | [The Model](data-model/contract-moderation.md#the-model) |
+| `moderation.moderators` | `{ "$type": ... }` | Who moderates: `"contractOwner"`, `"appointedModerators"` with `identities` (1 to 16), or `"elected"` with the keys below. | 14 | [moderation](contract-keywords/contract-config.md#moderation) |
+| `moderators.seatContestable` | boolean, required when elected | Whether a seated team may later be challenged. | 14 | [Elected Moderation](data-model/contract-moderation.md#elected-moderation) |
+| `moderators.challengeCoolDown` | seconds, two weeks to three years | How long a seated team is safe from a challenge after a seat change. Required when the seat is contestable, refused when it is not. | 14 | [Elected Moderation](data-model/contract-moderation.md#elected-moderation) |
+| `moderators.moderatedDocumentTypes` | object: document type → abilities | The document types the team moderates, each with its abilities: `ban`, `suspend`, `warn`, `deleteDocuments`. | 14 | [Elected Moderation](data-model/contract-moderation.md#elected-moderation) |
+| `moderators.interim` | `{ "$type": ... }` | Who moderates until a team is seated: `"contractOwner"`, `"appointedModerators"`, `"notYetUsable"` (the moderated types cannot be used yet) or `"noModeration"`. | 14 | [Elected Moderation](data-model/contract-moderation.md#elected-moderation) |
+| `moderators.joinWindow`, `.voteWindow` | seconds | How long applicants may join an election, and how long masternodes then vote. | 14 | [Elected Moderation](data-model/contract-moderation.md#elected-moderation) |
+| `moderators.electionDelay` | seconds | How long after the contract's creation the first election may be called. | 14 | [Elected Moderation](data-model/contract-moderation.md#elected-moderation) |
+| `moderators.maxAddedModerators` | 0 to 15, default 0 | How many members the seated leader may add after the election. | 14 | [Elected Moderation](data-model/contract-moderation.md#elected-moderation) |
+| `moderators.ownerProtected` | boolean, default `false` | Protects the contract owner from the seated team. | 14 | [Elected Moderation](data-model/contract-moderation.md#elected-moderation) |
+
+### Document type
+
+| Key | Takes | What it does | Since | Read more |
+|---|---|---|---|---|
+| `type` | `"object"` | Required. A document is an object. | 1 | [type](contract-keywords/document-shape.md#type) |
+| `properties` | object of 1 to 100 properties | The document's properties, each written with the [property keys](#property). | 1 | [properties](contract-keywords/document-shape.md#properties) |
+| `required` | array of names | The properties every document holds. A system time or height listed here is recorded. | 1 | [required](contract-keywords/document-shape.md#required) |
+| `additionalProperties` | `false` | Required: a document holds only the declared properties. | 1 | [additionalProperties](contract-keywords/document-shape.md#additionalproperties) |
+| `minProperties`, `maxProperties` | integer | How many properties a document holds. | 1 | [minProperties and maxProperties](contract-keywords/document-shape.md#minproperties-and-maxproperties) |
+| `dependentRequired` | object | A property that requires others when present. | 1 | [dependentRequired](contract-keywords/document-shape.md#dependentrequired) |
+| `$comment`, `description` | string | Notes; consensus ignores them. | 1 | [$comment and description](contract-keywords/document-shape.md#comment-and-description) |
+| `$schema`, `$defs` | added by the platform | The meta-schema URL and the contract's `schemaDefs`. A document type writing either is refused. | 1 | [$schema and $defs](contract-keywords/document-shape.md#schema-and-defs) |
+| `transient` | array of top-level names | Properties validated on the transition but never stored. | 1 | [transient](contract-keywords/transient.md) · [internals](data-model/documents.md#transient-properties) |
+| `documentsMutable` | boolean, default `true` | `false`: documents cannot be replaced. | 1 | [documentsMutable](contract-keywords/mutability.md#documentsmutable) |
+| `immutable` | array of top-level names | Properties frozen at creation on a mutable type. | 14 | [immutable](contract-keywords/mutability.md#immutable) · [internals](data-model/documents.md#immutable-properties-on-mutable-document-types) |
+| `immutableAllowSetting` | array of names from `immutable` | Immutable properties a replace may still set once, while they have no value. | 14 | [immutableAllowSetting](contract-keywords/mutability.md#immutableallowsetting) |
+| `canBeDeleted` | boolean, default `true` | `false`: a document's owner cannot delete it. | 1 | [canBeDeleted](contract-keywords/deletion.md#canbedeleted) |
+| `canBeDeletedByModerators` | boolean | The contract's moderators may delete documents of the type. | 14 | [canBeDeletedByModerators](contract-keywords/deletion.md#canbedeletedbymoderators) · [internals](data-model/contract-moderation.md#deleting-documents) |
+| `canBeDeletedByModeratorsFor` | seconds | Limits moderator deletion to this long after a document's last change. | 14 | [canBeDeletedByModeratorsFor](contract-keywords/deletion.md#canbedeletedbymoderatorsfor) |
+| `ttl` | seconds, 3600 to 31536000 | The platform deletes each document this long after its creation. | 14 | [Time To Live](contract-keywords/ttl.md) · [internals](data-model/document-ttl.md) |
+| `creationRestrictionMode` | `0` anyone, `1` contract owner, `2` nobody | Who may create documents. | 1 | [creationRestrictionMode](contract-keywords/ownership-and-trading.md#creationrestrictionmode) |
+| `transferable` | `0` never, `1` always | Whether an owner may give a document to another identity. | 1 | [transferable](contract-keywords/ownership-and-trading.md#transferable) |
+| `tradeMode` | `0` none, `1` direct purchase | Whether an owner may set a price and anyone buy at it. | 1 | [tradeMode](contract-keywords/ownership-and-trading.md#trademode) |
+| `documentsKeepHistory` | boolean, default `false` | Drive keeps every revision of every document. | 1 | [documentsKeepHistory](contract-keywords/history.md#documentskeephistory) |
+| `keepsTransferHistory`, `keepsPurchaseHistory`, `keepsPricingHistory` | boolean, default `false` | Records every transfer, purchase or price update in the document history contract. | 13 | [History](contract-keywords/history.md#keepstransferhistory) |
+| `signatureSecurityLevelRequirement` | `1` critical, `2` high (default), `3` medium | The weakest key level that may sign a transition on the type. | 1 | [signatureSecurityLevelRequirement](contract-keywords/signing-keys.md#signaturesecuritylevelrequirement) · [Security Level](sdk/identity-keys.md#security-level) |
+| `requiresIdentityEncryptionBoundedKey`, `requiresIdentityDecryptionBoundedKey` | `0` unique, `1` multiple, `2` multiple with a pointer to the latest | Lets identities bind encryption or decryption keys to the type, and says how they are kept. | 1 | [Signing and Keys](contract-keywords/signing-keys.md#requiresidentityencryptionboundedkey) · [Contract Bounds](sdk/identity-keys.md#contract-bounds) |
+| `ownerRefersTo` | a [`refersTo`](#refersto) declaration | A reference the writer must meet, on types whose documents are never transferred or traded. | 14 | [ownerRefersTo](contract-keywords/owner-refers-to.md#ownerrefersto) · [internals](data-model/documents.md#on-the-writer-or-the-creator-ownerrefersto-creatorrefersto) |
+| `creatorRefersTo` | a [`refersTo`](#refersto) declaration | A reference the creator must meet, on types whose documents can be transferred or traded. | 14 | [creatorRefersTo](contract-keywords/owner-refers-to.md#creatorrefersto) |
+| `propertyConstraints` | object of named rules, at most 16 | Rules over several properties every created or replaced document meets. See the [operators](#propertyconstraints). | 14 | [propertyConstraints](contract-keywords/property-constraints.md) · [internals](data-model/documents.md#property-constraints-propertyconstraints) |
+| `tokenCost` | object keyed by action | Token payments for actions on documents. See the [keys](#tokencost-and-actionfees). | 9 | [Token Costs](contract-keywords/token-cost.md) · [Fees](fees/overview.md#gas-paid-by-the-contract-owner) |
+| `actionFees` | object keyed by action | Credit fees for actions on documents, paid to the owner's and moderators' pots. See the [keys](#tokencost-and-actionfees). | 14 | [Action Fees](contract-keywords/action-fees.md) · [Fees](fees/overview.md#document-action-fees) |
+| `indices` | array of 1 to 10 indexes | The indexes documents are queried by, each written with the [index keys](#index). | 1 | [Indexes](contract-keywords/indexes.md#indices) · [internals](drive/indexes.md) |
+| `documentsCountable` | boolean | Keeps a count of the type's documents. | 12 | [documentsCountable](contract-keywords/aggregates.md#documentscountable) · [internals](drive/document-count-trees.md#primary-key-tree-flags) |
+| `documentsSummable` | property name | Keeps the sum of one integer property over the type's documents. | 12 | [documentsSummable](contract-keywords/aggregates.md#documentssummable) · [internals](drive/document-sum-trees.md#primary-key-tree-flags) |
+| `documentsAverageable` | property name | Shorthand for `documentsCountable` plus `documentsSummable`. | 12 | [documentsAverageable](contract-keywords/aggregates.md#documentsaverageable) |
+| `rangeCountable`, `rangeSummable`, `rangeAverageable` | boolean | Provable counts, sums or averages over ranges of document ids. | 12 | [Document type range keys](contract-keywords/aggregates.md#document-type-rangecountable-rangesummable-rangeaverageable) |
+| `indexOnly` | boolean | Documents are never stored whole: the index entries are the rows. | 14 | [indexOnly](contract-keywords/index-only.md#indexonly) · [internals](drive/index-only-document-types.md) |
+| `entryPayload` | array of 1 to 16 names | On an index-only type, properties carried in each entry's value instead of a key. | 14 | [entryPayload](contract-keywords/index-only.md#entrypayload) |
+
+### Property
+
+| Key | Takes | What it does | Since | Read more |
+|---|---|---|---|---|
+| `type` | `string`, `integer`, `number`, `boolean`, `object`, `array` | The kind of value. An array is a byte array or a typed array. | 1 | [type](contract-keywords/property-schemas.md#type) |
+| `position` | integer | The property's place in the stored document. Required; top-level positions run 0, 1, 2 with no gap. | 1 | [position](contract-keywords/property-schemas.md#position) · [Document Serialization](serialization/document-serialization.md) |
+| `minLength`, `maxLength` | integer | A string's length in characters. | 1 | [Strings](contract-keywords/property-schemas.md#strings) |
+| `pattern` | regular expression | A string must match it. Needs `maxLength` of at most 50000. | 1 | [Strings](contract-keywords/property-schemas.md#strings) |
+| `format` | `date-time`, `date`, `time`, `email`, `idn-email`, `hostname`, `ipv4`, `ipv6`, `uri`, `regex` | A string must have this format. Needs `maxLength` of at most 50000. | 1 | [Strings](contract-keywords/property-schemas.md#strings) |
+| `maxBytes` | 1 to 65535 | The most UTF-8 bytes a string may take. | 14 | [maxBytes](contract-keywords/max-bytes.md) · [internals](data-model/documents.md#byte-caps-on-strings-maxbytes) |
+| `minimum`, `maximum`, `exclusiveMinimum`, `exclusiveMaximum`, `multipleOf` | number | Numeric bounds. `minimum` and `maximum` also decide an integer's stored width. | 1 | [Numbers](contract-keywords/property-schemas.md#numbers) |
+| `enum`, `const` | values | The values allowed, or the one value allowed. | 1 | [enum and const](contract-keywords/property-schemas.md#enum-and-const) |
+| `byteArray` | `true` | Makes an array a string of bytes, stored raw. | 1 | [Byte arrays and identifiers](contract-keywords/property-schemas.md#byte-arrays-and-identifiers) |
+| `contentMediaType` | `"application/x.dash.dpp.identifier"` | Makes a 32-byte array an identifier. | 1 | [Byte arrays and identifiers](contract-keywords/property-schemas.md#byte-arrays-and-identifiers) |
+| `minItems`, `maxItems`, `uniqueItems`, `contains` | integer, boolean, schema | A byte array's length in bytes, or a typed array's number of elements (at most 1024); no repeats; an element that matches. | 1 | [Arrays](contract-keywords/property-schemas.md#arrays) |
+| `items` | an element schema | Makes an array a typed array whose elements all follow this schema. | 14 | [Typed Arrays](contract-keywords/typed-arrays.md) · [internals](data-model/documents.md#typed-arrays) |
+| `properties`, `required`, `additionalProperties`, `minProperties`, `maxProperties`, `dependentRequired` | as on a document type | A nested object's members and its bounds. | 1 | [Objects](contract-keywords/property-schemas.md#objects) |
+| `$ref` | `"#/$defs/"` | Uses a definition from the contract's `schemaDefs`. | 1 | [$ref](contract-keywords/property-schemas.md#ref) |
+| `$id`, `$comment`, `description`, `examples` | annotations | Notes; consensus ignores them. | 1 | [Annotations](contract-keywords/property-schemas.md#annotations) |
+| `requiredSince` | contract version | Lets an update add a required property that older documents may leave out. | 14 | [requiredSince](contract-keywords/required-since.md) · [internals](data-model/data-contracts.md#evolving-a-contract-adding-required-fields) |
+| `distinctFrom` | a property path or `"$ownerId"` | An identifier must differ from another identifier of the document, or from the owner. | 14 | [distinctFrom](contract-keywords/distinct-from.md) · [internals](data-model/documents.md#distinct-identifier-properties) |
+| `encryptedFor` | `{ "recipient", "recipientKey", "senderKey", "scheme" }` | Declares how an encrypted byte array was made: whose keys, which scheme. | 14 | [encryptedFor](contract-keywords/encrypted-for.md) · [internals](data-model/documents.md#encrypted-properties-encryptedfor) |
+| `encryptedFor.recipient` | identifier property path or `"$ownerId"` | The identity the value is encrypted to. | 14 | [encryptedFor](contract-keywords/encrypted-for.md#example) |
+| `encryptedFor.recipientKey`, `.senderKey` | integer property paths | The properties holding the recipient's and the sender's key ids. | 14 | [encryptedFor](contract-keywords/encrypted-for.md#example) |
+| `encryptedFor.scheme` | `"ecdh-secp256k1-aes256-cbc"` | How the ciphertext is made. | 14 | [The scheme](contract-keywords/encrypted-for.md#the-scheme) |
+| `generatedFrom` | `{ "function", "params" }` | The platform generates the string from other properties of the document; on arrival when a document leaves it out. | 14 | [generatedFrom](contract-keywords/generated-from.md) · [internals](data-model/documents.md#generated-properties-generatedfrom) |
+| `generatedFrom.function` | `"sys.stringTransformations.homographSafeASCII"` | The system function that generates the value: `sys.stringTransformations.` `lowercase`, `uppercase`, `capitalize`, `camelCase`, `snakeCase` or `homographSafeASCII`. | 14 | [Functions](contract-keywords/generated-from.md#functions) |
+| `generatedFrom.params` | property paths | The properties the function reads, in order. | 14 | [Params](contract-keywords/generated-from.md#params) |
+| `refersTo` | a declaration | What an identifier points at, checked when a document is written. See the [keys](#refersto). | 14 | [References](contract-keywords/refers-to.md) · [internals](data-model/documents.md#document-references-refersto) |
+
+A typed array's element (`items`) takes `type`, `enum`, `minimum`, `maximum`, `exclusiveMinimum`, `exclusiveMaximum`, `multipleOf`, `minLength`, `maxLength`, `pattern`, `format`, `minItems` and `maxItems` (bytes of a byte array element), `byteArray`, `contentMediaType`, `maxBytes`, `distinctFrom`, `refersTo`, `$comment` and `description`. It takes no `position`, `const`, `uniqueItems` or `examples`.
+
+### refersTo
+
+| Key | Takes | What it does | Since | Read more |
+|---|---|---|---|---|
+| `type` | a target below | What the value points at. | 14 | [Targets](contract-keywords/refers-to.md#targets) |
+| `type: "identity"` | | The value is the id of an existing identity. | 14 | [identity](contract-keywords/refers-to.md#identity) |
+| `type: "contract"` | | The value is the id of an existing data contract. | 14 | [contract](contract-keywords/refers-to.md#contract) |
+| `type: "token"` | | The value is the id of an existing token. | 14 | [token](contract-keywords/refers-to.md#token) |
+| `type: "permanentDocument"` | | The value is the id of a document whose type can never lose its documents. | 14 | [permanentDocument](contract-keywords/refers-to.md#permanentdocument) |
+| `type: "deletableDocument"` | | The value is the id of a document that can be deleted; checked again on every replace. | 14 | [deletableDocument](contract-keywords/refers-to.md#deletabledocument) |
+| `type: "identityPublicKey"` | | The value names an identity key that exists and is not disabled. | 14 | [identityPublicKey](contract-keywords/refers-to.md#identitypublickey) |
+| `type: "listElement"` | | The value is one of the identifiers a list on another document holds. | 14 | [List Elements](contract-keywords/refers-to-list-element.md) · [internals](data-model/documents.md#an-element-of-a-list-listelement) |
+| `documentType` | document type name | The referenced document type. | 14 | [documentType](contract-keywords/refers-to.md#documenttype) |
+| `contractId` | identifier | The contract holding `documentType`, when it is not this one. | 14 | [contractId](contract-keywords/refers-to.md#contractid) |
+| `propertyAgreement` | 1 to 10 pairs `{ "": "" }` | Properties of the two documents that must be equal. `$ownerId` here makes a write gate. | 14 | [propertyAgreement](contract-keywords/refers-to.md#propertyagreement) |
+| `lookup` | `{ "index", "keys" }` | Finds the document through a unique index, with the value as one part of the key. | 14 | [Lookups](contract-keywords/refers-to-lookup.md) · [internals](data-model/documents.md#resolved-through-a-unique-index-lookup) |
+| `lookup.index` | index name | A unique index of `documentType`. | 14 | [Lookups](contract-keywords/refers-to-lookup.md#assembling-the-key) |
+| `lookup.keys` | index property → `"."`, `"$ownerId"` or a path | Where each part of the key comes from; `"."` is the value itself. | 14 | [Lookups](contract-keywords/refers-to-lookup.md#assembling-the-key) |
+| `inList` | typed array path | The list on the referenced document the value must be in. | 14 | [List Elements](contract-keywords/refers-to-list-element.md) |
+| `keyIdProperty` | integer property path | On an identity property: the property holding the key id. | 14 | [keyIdProperty and identityProperty](contract-keywords/refers-to.md#keyidproperty-and-identityproperty) |
+| `identityProperty` | `"$ownerId"`, `"$creatorId"` or a path | On a key id property: whose key it is. | 14 | [keyIdProperty and identityProperty](contract-keywords/refers-to.md#keyidproperty-and-identityproperty) |
+| `keyRequirements.purpose` | `authentication`, `encryption`, `decryption`, `transfer`, `voting`, `owner` | The key's purpose. | 14 | [keyRequirements](contract-keywords/refers-to.md#keyrequirements) |
+| `keyRequirements.boundTo` | document type name | The key must be bound to that document type of this contract. | 14 | [keyRequirements](contract-keywords/refers-to.md#keyrequirements) |
+| `contractRequirements.moderation` | `"elected"`, `"electionOpen"` | The contract declares an elected team, or one whose election may be called. | 14 | [contractRequirements](contract-keywords/refers-to.md#contractrequirements) · [Elected Moderation](data-model/contract-moderation.md#elected-moderation) |
+| `contractRequirements.minimumAgeSeconds`, `.minimumSecondsSinceUpdate` | seconds | The contract was created, or last changed, at least this long ago. | 14 | [contractRequirements](contract-keywords/refers-to.md#contractrequirements) |
+| `contractRequirements.owner` | `"self"`, `"other"` | The contract is owned by the writer, or by someone else. | 14 | [contractRequirements](contract-keywords/refers-to.md#contractrequirements) |
+| `contractRequirements.readonly`, `.keepsHistory` | `true` | The contract can never be updated, or keeps history. | 14 | [contractRequirements](contract-keywords/refers-to.md#contractrequirements) |
+| `contractRequirements.ownerProtected` | boolean | The contract's elected team does, or does not, protect its owner. | 14 | [contractRequirements](contract-keywords/refers-to.md#contractrequirements) |
+| `anyOf`, `allOf` | 2 to 4 operands | In place of `type`: at least one, or every, operand holds. Nest at most 4 deep. | 14 | [Expressions](contract-keywords/refers-to-expressions.md) · [internals](data-model/documents.md#reference-expressions-anyof-allof) |
+
+### tokenCost and actionFees
+
+`` is one of `create`, `replace`, `delete`, `transfer`, `update_price` and `purchase`.
+
+| Key | Takes | What it does | Since | Read more |
+|---|---|---|---|---|
+| `tokenCost..tokenPosition` | 0 to 65535, required | Which token is charged. | 9 | [Token Costs](contract-keywords/token-cost.md) |
+| `tokenCost..amount` | at least 1, required | How many tokens the action costs. | 9 | [Token Costs](contract-keywords/token-cost.md) |
+| `tokenCost..contractId` | identifier | The contract whose token is charged, when it is not this one. | 9 | [contractId](contract-keywords/token-cost.md#tokens-of-another-contract-contractid) |
+| `tokenCost..effect` | `0` to the contract owner (default), `1` burn | What happens to the tokens paid. | 9 | [effect](contract-keywords/token-cost.md#effect-transfer-or-burn) |
+| `tokenCost..gasFeesPaidBy` | `0` document owner (default), `1` contract owner, `2` prefer contract owner | Who the contract owner offers to have pay the gas. Accepted from 9, acted on from 14. | 14 | [gasFeesPaidBy](contract-keywords/token-cost.md#who-pays-the-gas-gasfeespaidby) · [Fees](fees/overview.md#gas-paid-by-the-contract-owner) |
+| `tokenCost..optional` | boolean, default `false` | A transition may skip the token and pay in credits. | 14 | [Optional costs](contract-keywords/token-cost.md#optional-costs) · [Fees](fees/overview.md#optional-token-costs) |
+| `actionFees.pricing` | `"feeMultiplier"` (default), `"fixed"` | Whether the amounts scale with the epoch's fee multiplier. | 14 | [Action Fees](contract-keywords/action-fees.md#how-it-works) |
+| `actionFees..owner` | credits | Paid into the contract owner's pot. | 14 | [The pots and the claim](contract-keywords/action-fees.md#the-pots-and-the-claim) |
+| `actionFees..moderators` | credits | Paid into the moderators' pot. Needs `moderation`. | 14 | [The pots and the claim](contract-keywords/action-fees.md#the-pots-and-the-claim) · [Fee Pots](data-model/contract-moderation.md#fee-pots-and-the-claim) |
+
+### propertyConstraints
+
+A rule is one condition. Conditions:
+
+| Key | Takes | Holds when | Since | Read more |
+|---|---|---|---|---|
+| `equal`, `notEqual` | `[a, b]` | The two sides are equal, or differ: integer expressions, strings or identifiers. | 14 | [Conditions](contract-keywords/property-constraints.md#conditions) |
+| `lessThan`, `lessThanOrEqual`, `greaterThan`, `greaterThanOrEqual` | `[a, b]` | The integer comparison holds. | 14 | [Conditions](contract-keywords/property-constraints.md#conditions) |
+| `in` | `[a, [values]]` | `a` takes one of two or more listed integers, strings or identifiers. | 14 | [Conditions](contract-keywords/property-constraints.md#conditions) |
+| `present`, `absent` | a path | The document holds the property, or leaves it out (or null, or an object with no member present). | 14 | [Conditions](contract-keywords/property-constraints.md#conditions) |
+| `anyOf`, `allOf` | two or more conditions | At least one, or every, condition holds, checked in order. | 14 | [Evaluation order](contract-keywords/property-constraints.md#evaluation-order-and-short-circuiting) |
+| `not` | a condition | The condition does not hold. | 14 | [Conditions](contract-keywords/property-constraints.md#conditions) |
+| `ifThen`, `ifThenElse` | `[if, then]`, `[if, then, else]` | The second condition holds when the first does (and, for `ifThenElse`, the third when it does not); only the branch taken is evaluated. | 14 | [Conditions](contract-keywords/property-constraints.md#conditions) |
+| `notIn` | `[a, [values]]` | `a` takes none of the listed values. | 14 | [Conditions](contract-keywords/property-constraints.md#conditions) |
+| `startsWith`, `endsWith` | `[text, affix]` | A string starts or ends with another, byte for byte. | 14 | [Conditions](contract-keywords/property-constraints.md#conditions) |
+| `contains` | `[array, value]` | A typed array holds an element equal to the value. | 14 | [Conditions](contract-keywords/property-constraints.md#conditions) |
+
+Expressions:
+
+| Key | Takes | Value | Since | Read more |
+|---|---|---|---|---|
+| an integer | `100` | Itself. | 14 | [Expressions](contract-keywords/property-constraints.md#expressions) |
+| a path | `"price"`, `"meta.total"` | An integer or boolean property's value; 0 when left out. | 14 | [Expressions](contract-keywords/property-constraints.md#expressions) |
+| `add`, `multiply` | two or more operands | The sum or product. | 14 | [Arithmetic](contract-keywords/property-constraints.md#arithmetic) |
+| `subtract`, `divide`, `modulo`, `power` | `[a, b]` | The difference, Euclidean quotient or remainder, or power. | 14 | [Arithmetic](contract-keywords/property-constraints.md#arithmetic) |
+| `min`, `max`, `abs` | two or more operands, or one for `abs` | The least, the greatest, or the absolute value. | 14 | [Expressions](contract-keywords/property-constraints.md#expressions) |
+| `countOf`, `sumOf` | `[type, filter?]`, `[type, property, filter?]` | How many documents of a type of the contract match the filter, or the total of an integer property over them, from its count or sum trees. | 14 | [Totals of other documents](contract-keywords/property-constraints.md#totals-of-other-documents) |
+| `ifAbsent` | `[path, default]` | The property's value, or the default when left out (an integer, or a string for a string property). | 14 | [Expressions](contract-keywords/property-constraints.md#expressions) |
+| `length`, `byteLength` | a string path | A string's length in characters, or in UTF-8 bytes. | 14 | [Expressions](contract-keywords/property-constraints.md#expressions) |
+| `count` | an array path | The elements of a typed array, or the bytes of a byte array. | 14 | [Expressions](contract-keywords/property-constraints.md#expressions) |
+| `$createdAt`, `$updatedAt`, `$transferredAt`, `$createdAtBlockHeight`, `$updatedAtBlockHeight`, `$transferredAtBlockHeight`, `$createdAtCoreBlockHeight`, `$updatedAtCoreBlockHeight`, `$transferredAtCoreBlockHeight` | a path | A time or height the document records, when listed in `required`. | 14 | [Times and heights](contract-keywords/property-constraints.md#times-and-heights) |
+| `const` | a string | A string constant, or a base58 identifier, as one side of `equal` or `notEqual`. | 14 | [Strings](contract-keywords/property-constraints.md#strings) |
+| `$ownerId` | | The document's owner, as an identifier side. | 14 | [Identifiers and $ownerId](contract-keywords/property-constraints.md#identifiers-and-ownerid) |
+
+### Index
+
+| Key | Takes | What it does | Since | Read more |
+|---|---|---|---|---|
+| `name` | 1 to 32 characters, required | The index's name, unique in the type. | 1 | [name](contract-keywords/indexes.md#name) |
+| `properties` | 1 to 10 `{ "": "asc" }` | The indexed properties, in order. A flat index of an index-only type leaves it out. | 1 | [properties](contract-keywords/indexes.md#properties) · [internals](drive/indexes.md) |
+| `unique` | boolean | No two documents share the indexed values. | 1 | [unique](contract-keywords/indexes.md#unique) |
+| `nullSearchable` | boolean, default `true` | `false` leaves out documents whose indexed values are all null. | 1 | [nullSearchable](contract-keywords/indexes.md#nullsearchable) |
+| `contested` | object | Matching values are decided by a masternode vote, not first come. | 1 | [Contested Indexes](contract-keywords/contested.md) · [internals](data-model/contested-documents.md) |
+| `contested.resolution` | `0` vote with lock, `1` vote without lock | How the contest is decided. `1` from 14. | 1 | [The keys](contract-keywords/contested.md#the-keys) |
+| `contested.fieldMatches` | `[{ "field", "regexPattern" }]` | Which values are contested. | 1 | [The keys](contract-keywords/contested.md#the-keys) |
+| `contested.description` | string | A note; consensus ignores it. | 1 | [The keys](contract-keywords/contested.md#the-keys) |
+| `countable` | `"notCountable"`, `"countable"`, `"countableAllowingOffset"` or boolean | Keeps a document count per indexed value. | 12 | [countable](contract-keywords/aggregates.md#countable) · [internals](drive/document-count-trees.md#per-index-countable-flag) |
+| `summable` | property name | Keeps the sum of an integer property per indexed value. | 12 | [summable](contract-keywords/aggregates.md#summable) · [internals](drive/document-sum-trees.md#per-index-summable-flag) |
+| `averageable` | property name | Shorthand for `countable` plus `summable`. | 12 | [averageable](contract-keywords/aggregates.md#averageable) |
+| `rangeCountable`, `rangeSummable`, `rangeAverageable` | boolean | Provable counts, sums or averages over ranges of the indexed value. | 12 | [Index range keys](contract-keywords/aggregates.md#index-rangecountable-rangesummable-rangeaverageable) |
+| `rankedCountable` | boolean or `{ "at": ... }` | Orders the indexed values by document count, for "top K" queries; `at` names the levels ranked. | 14 | [rankedCountable](contract-keywords/ranked.md#rankedcountable) · [internals](drive/document-ranked-trees.md#contract-grammar) |
+| `rankedSummable`, `rankedAverageable` | boolean | Orders them by sum, or by average. | 14 | [Ranked Indexes](contract-keywords/ranked.md#rankedsummable) |
+| `timeRange` | `{ "on", "range", "step", "phase", "ttl" }` | Buckets a system timestamp into time windows, for trending queries. | 14 | [Time-Range Indexes](contract-keywords/time-range.md) · [internals](drive/time-range-ttl.md) |
+| `timeRange.on` | `"$createdAt"`, `"$updatedAt"`, `"$transferredAt"` | The timestamp to bucket: the index's first property. | 14 | [The keys](contract-keywords/time-range.md#the-keys) |
+| `timeRange.range`, `.step` | seconds | Each window's length, and the time between window starts. | 14 | [The keys](contract-keywords/time-range.md#the-keys) |
+| `timeRange.phase` | seconds, default 0 | Shifts the window boundaries. | 14 | [The keys](contract-keywords/time-range.md#the-keys) |
+| `timeRange.ttl` | seconds, at most one week | Expires the index's entries after their window; on an index-only type, the rows leave this index. | 14 | [The keys](contract-keywords/time-range.md#the-keys) · [internals](drive/time-range-ttl.md#cleanup) |
+| `terminal` | property name or list | On an index-only type, what keys each entry in place of the document id. | 14 | [terminal](contract-keywords/index-only.md#terminal) |
+| `preallocated` | boolean | On an index-only type, creates the index's trees with the referenced document. | 14 | [preallocated](contract-keywords/index-only.md#preallocated) · [internals](drive/index-only-document-types.md#preallocated-index-paths) |
+| `skipIfAbsent` | boolean | On an index-only type, a document without the first property writes no entry. | 14 | [skipIfAbsent](contract-keywords/index-only.md#skipifabsent) · [internals](drive/index-only-document-types.md#conditional-participation-skipifabsent) |
+
+### System properties
+
+| Property | Holds | Recorded | Since | Read more |
+|---|---|---|---|---|
+| `$id` | The document's id. | always | 1 | [$id](contract-keywords/system-properties.md#id) |
+| `$ownerId` | The identity that owns the document. | always | 1 | [$ownerId](contract-keywords/system-properties.md#ownerid) |
+| `$revision` | 1 at creation, raised by every replace, transfer, price update and purchase. | on types whose documents can change hands or content | 1 | [$revision](contract-keywords/system-properties.md#revision) |
+| `$createdAt`, `$updatedAt`, `$transferredAt` | Block times, in milliseconds, of the creation, the last replace or price update, and the last transfer or purchase. | when listed in `required` | 1 | [Timestamps](contract-keywords/system-properties.md#timestamps) |
+| `$createdAtBlockHeight`, `$updatedAtBlockHeight`, `$transferredAtBlockHeight` | Platform block heights of the same events. | when listed in `required` | 1 | [Block heights](contract-keywords/system-properties.md#block-heights) |
+| `$createdAtCoreBlockHeight`, `$updatedAtCoreBlockHeight`, `$transferredAtCoreBlockHeight` | Core chain block heights of the same events. | when listed in `required` | 1 | [Block heights](contract-keywords/system-properties.md#block-heights) |
+| `$creatorId` | The identity that created the document. | on transferable or tradeable types of format-1 contracts | 10 | [$creatorId](contract-keywords/system-properties.md#creatorid) |
+
+## Limits
+
+The first three limits come from the meta-schema, the rest from protocol version 14's `SystemLimits`. A contract over a limit is refused at registration.
+
+| Limit | Value | Applies to |
+|---|---|---|
+| Properties per object | 100 | `properties`, at the top and in each nested object |
+| Indexes per document type | 10 | `indices` |
+| Properties per index | 10 | an index's `properties` |
+| `max_field_value_size` | 5120 bytes | any one value a document stores (`DocumentFieldMaxSizeExceededError`, 10417) |
+| `max_typed_array_items` | 1024 | a typed array's `maxItems` |
+| `max_references_per_document` | 256 | references one document carries |
+| `max_reference_operands` | 4 | operands in one `anyOf` or `allOf` of a reference |
+| `max_reference_expression_depth` | 4 | nesting of reference expressions |
+| `max_property_constraints` | 16 | rules in one `propertyConstraints` |
+| `max_property_constraint_nodes` | 32 | nodes in one rule |
+| `min_document_ttl_seconds`, `max_document_ttl_seconds` | 3600, 31536000 | `ttl` |
+| `max_time_range_ttl_seconds` | 604800 | a `timeRange` index's `ttl` |
diff --git a/book/src/contract-keywords/action-fees.md b/book/src/contract-keywords/action-fees.md
new file mode 100644
index 00000000000..e5e4087b085
--- /dev/null
+++ b/book/src/contract-keywords/action-fees.md
@@ -0,0 +1,102 @@
+# Action Fees (actionFees)
+
+`actionFees` charges a fixed fee in credits, on top of the gas, for an action on a document of the type. Each fee has two parts: one for the contract owner and one for the contract's moderators, each collected in a pot that its recipients claim. Reach for it when an app wants to earn from the documents written under it, or to pay the people who moderate it. The transition that pays must state the fee it agrees to, so a fee can never surprise a signer.
+
+| | |
+|---|---|
+| **Where** | Document type |
+| **Value** | An object with an optional `pricing`, and one or more of the actions `create`, `replace`, `delete`, `transfer`, `update_price`, `purchase`, each an object with `owner` and/or `moderators` (below) |
+| **Default** | Absent: no action charges a fee |
+| **Since** | protocol version 14 |
+| **On update** | Fixed (`DocumentTypeUpdateError`, 40212): an update may not add, change or remove the fees of an existing document type, nor switch their pricing. A document type the update adds may declare its own |
+| **Errors** | On a document transition: `DocumentActionFeeAgreementNotSetError` (40132), `DocumentActionFeeAgreementMismatchError` (40133), `DocumentActionFeeMultiplierNotToleratedError` (40134), `DocumentActionFeeModeratorsShareMismatchError` (40139). At registration: `DocumentActionFeesWithoutModerationError` (10902), `JsonSchemaError` (10101), `InvalidContractStructure` (10231) |
+
+The keys:
+
+| Key | Value | Default | Meaning |
+|---|---|---|---|
+| `pricing` | `"feeMultiplier"` or `"fixed"` | `"feeMultiplier"` | Whether the amounts follow the network's fee multiplier or are charged as written |
+| `.owner` | credits, 0 to 9223372036854775807 | 0 | Added to the contract's owner pot, which the contract owner claims |
+| `.moderators` | credits, 0 to 9223372036854775807 | 0 | Added to the contract's moderators pot, which the moderation team shares. Needs `moderation` in the contract config |
+
+Amounts are in credits: 1 Dash is 100,000,000,000 credits (1000 credits per duff).
+
+## Example
+
+```json
+"post": {
+ "type": "object",
+ "actionFees": {
+ "pricing": "feeMultiplier",
+ "create": { "moderators": 100000000, "owner": 10000000 }
+ },
+ "properties": {
+ "text": { "type": "string", "minLength": 1, "maxLength": 280, "maxBytes": 560, "position": 0 }
+ },
+ "required": ["text"],
+ "additionalProperties": false
+}
+```
+
+In a contract that declares moderation, creating a post costs an extra 0.001 Dash for the moderation team and 0.0001 Dash for the contract owner, at a fee multiplier of 1. Replacing, deleting and every other action cost only their gas.
+
+## How it works
+
+- **What is charged.** With `fixed` pricing, the declared amounts. With `feeMultiplier`, the declared amounts scaled by the fee multiplier of the epoch the action executes in (`declared * multiplier_permille / 1000`, rounded down), so a fee follows the network's fees. A scaled amount is held at the maximum number of credits instead of overflowing; such a fee refuses the action for an insufficient balance.
+- **Who pays.** Whoever pays the gas pays the fee: the signer, or the contract owner when they pay the gas of a token-paid action (see [Token Costs](token-cost.md#who-pays-the-gas-gasfeespaidby)). The contract owner never pays the `owner` part, which would only travel through their pot back to them: a contract owner who pays, as the signer or as the gas sponsor, pays the `moderators` part only. A contract that sponsors gas should price that in: a sponsor's balance must cover the gas and those fees, or the transition falls back to the signer or is refused, as the token cost's rules say.
+- **Only executed actions pay.** A transition that fails, at any stage, owes no fee.
+- **Where it goes.** One removal from the payer's balance, and one addition to each pot the fee has a part for. The fee is not part of the gas: the fee pools and block proposers get none of it.
+
+## The agreement: `$actionFeeAgreement`
+
+The contract is read when the transition executes, not when it was signed. So every transition on an action that charges a fee carries an action fee agreement in its base, naming the fee its signer saw (version 2 of the document base transition, the default from protocol version 14):
+
+```json
+"$actionFeeAgreement": {
+ "$formatVersion": "0",
+ "owner": 10000000,
+ "moderators": 100000000,
+ "feeMultiplier": { "knownPermille": 1000, "increaseTolerancePercent": 20 }
+}
+```
+
+| Field | Meaning |
+|---|---|
+| `owner`, `moderators` | The amounts the document type declares for the action, before any multiplier. They must match exactly, each part on its own |
+| `feeMultiplier` | Present for a `feeMultiplier` fee, left out for a `fixed` one. `knownPermille` is the fee multiplier the signer priced the fee with, in thousandths (1000 is 1x). `increaseTolerancePercent` is how far above it the executing epoch's multiplier may be, in percent of the known one: 20 accepts up to 1.2 times |
+
+A transition on an action that charges a fee is refused:
+
+- without an agreement (`DocumentActionFeeAgreementNotSetError`, 40132), whoever pays, a sponsored transition included;
+- with other amounts, parts moved between the pots, or the other pricing (`DocumentActionFeeAgreementMismatchError`, 40133). The signer reads the contract again;
+- when the executing epoch's multiplier is above what the agreement tolerates (`DocumentActionFeeMultiplierNotToleratedError`, 40134). A multiplier that fell is always accepted, and what is charged follows the epoch's multiplier, never the known one.
+
+Each refusal bumps the signer's nonce, and no action fee is charged. The mempool applies the same checks on arrival and on every recheck, so a transition whose agreement no longer holds leaves the mempool with the same error. An agreement on an action that charges nothing is ignored.
+
+A client should build the agreement from the contract it showed its user, never from a contract fetched behind their back at signing time. In Rust, `DocumentActionFeeAgreement::for_document_type_action` builds it from a document type, and the SDK's document transition builders take it with `with_action_fee_agreement`.
+
+**A seated team's discount.** On a document type that an elected contract moderates, the `moderators` part of an agreement may name less than the declared amount: exactly the share the contract's seated moderation charter takes (its `moderatorsShare`, in percent, rounded down to the credit). Everything else must still match. The action is then charged the agreed amount. Any other amount below the declared one, including a discount on a contract with no seated charter yet, is refused (`DocumentActionFeeModeratorsShareMismatchError`, 40139). A lower amount anywhere else, on a type the contract does not moderate or a contract that is not elected, is the plain mismatch (40133). See [Elected Moderation](../data-model/contract-moderation.md#elected-moderation).
+
+## The pots and the claim
+
+The `owner` parts collect in the contract's **owner pot** and the `moderators` parts in its **moderators pot**. A `ContractFeeClaim` state transition pays a pot out:
+
+- the owner pot goes whole to the contract owner, the only identity that may claim it;
+- the moderators pot is split equally between the moderation team (the identities the contract appoints, or the owner alone when it appoints none), and any member of the team may claim it for all of them. A seated elected team splits it by its charter's reward split instead. What a split leaves over, less than a credit per member, stays in the pot;
+- each pot is paid out at most once per epoch, and the two are independent.
+
+A claim is refused when the signer is not a recipient of the pot (`ContractFeeClaimNotAllowedError`, 41113), when the pot was already paid out this epoch (`ContractFeesAlreadyClaimedThisEpochError`, 41111), or when a recipient would get less than a credit (`ContractFeesNothingToClaimError`, 41112). The JavaScript SDK reads the pots with `contracts.feePots` and claims with `contracts.claimFees`.
+
+## Rules at registration
+
+- The meta-schema checks the shape (`JsonSchemaError`, 10101): only `pricing` and the six action keys; `pricing` one of the two values; each action an object with `owner`, `moderators` or both, each an integer from 0 to 9223372036854775807.
+- At least one action is priced, and a priced action charges something: an action whose parts are all 0 is refused, leave it out instead. The two parts of an action may not add up to more than 9223372036854775807 credits (`InvalidContractStructure`, 10231).
+- A nonzero `moderators` part needs a contract whose config declares `moderation` (`DocumentActionFeesWithoutModerationError`, 10902): the moderation team is who that pot is for. This is checked when the contract is created and when it is updated.
+
+## See also
+
+- [Document action fees](../fees/overview.md#document-action-fees), the deep dive
+- [Fee Pots and the Claim](../data-model/contract-moderation.md#fee-pots-and-the-claim), for the pots, the claim and its proofs
+- [Token Costs (tokenCost)](token-cost.md), which prices the same six actions in tokens
+- [Contract-Level Keys and config](contract-config.md), for `moderation`
+- [Contract Keywords overview](../contract-keywords.md), for the conventions of these tables
diff --git a/book/src/contract-keywords/aggregates.md b/book/src/contract-keywords/aggregates.md
new file mode 100644
index 00000000000..35ca0a89946
--- /dev/null
+++ b/book/src/contract-keywords/aggregates.md
@@ -0,0 +1,178 @@
+# Counts, Sums and Averages
+
+Counting documents normally means fetching them and counting what comes back, which grows with the number of documents and proves every one of them. The keywords in this chapter make Drive keep running totals inside its trees instead, so a count, a sum or an average is read without visiting the documents, and proved with a short proof. The document type keywords keep totals over all of a type's documents. The index keywords keep totals per indexed value, and with their `range*` forms over ranges of values. An average is never stored as such: the platform returns the count and the sum of the same documents, and the client divides.
+
+Every total is updated by every write that touches it, so each flag adds to the cost of writing documents of the type. The flags choose the layout of the type's trees, so all of them are fixed when the document type is created.
+
+## Example
+
+```json
+"tip": {
+ "type": "object",
+ "documentsMutable": false,
+ "documentsAverageable": "amount",
+ "indices": [
+ { "name": "byRecipient", "properties": [{ "recipient": "asc" }], "averageable": "amount" },
+ { "name": "byDay", "properties": [{ "day": "asc" }], "summable": "amount", "rangeSummable": true }
+ ],
+ "properties": {
+ "recipient": {
+ "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "position": 0
+ },
+ "amount": { "type": "integer", "minimum": 1, "maximum": 4294967295, "position": 1 },
+ "day": { "type": "integer", "minimum": 0, "maximum": 4294967295, "position": 2 }
+ },
+ "required": ["recipient", "amount", "day"],
+ "additionalProperties": false
+}
+```
+
+`documentsAverageable` keeps the number of tips and their total, so the average tip over the whole type is one read. `byRecipient` keeps the same pair per recipient, so "how many tips did this identity get, and how much on average" is one read. `byDay` keeps the sum per day and, with `rangeSummable`, answers "total tipped between day 100 and day 130" without visiting each day.
+
+## `documentsCountable`
+
+| | |
+|---|---|
+| **Where** | document type |
+| **Value** | boolean |
+| **Default** | `false` |
+| **Since** | protocol version 12 |
+| **On update** | Fixed (`DocumentTypeUpdateError`, 40212) |
+
+Keeps the number of the type's documents in its primary tree, so a count query with no `where` clause is one read, with a proof. Without it, such a query is refused rather than answered slowly.
+
+## `documentsSummable`
+
+| | |
+|---|---|
+| **Where** | document type |
+| **Value** | the name of an integer property, 1 to 64 characters |
+| **Default** | absent |
+| **Since** | protocol version 12 |
+| **On update** | Fixed (40212) |
+
+Keeps the sum of the named property over all the type's documents, so a sum query with no `where` clause is one read. On a type with `documentsKeepHistory`, the sum counts each document's current revision only.
+
+The property must exist on the type, be listed in `required` (a missing value would leave the sum wrong when the document is deleted), and hold values that fit a signed 64-bit sum. Integers are stored in the smallest width their bounds allow when the contract uses [`sizedIntegerTypes`](contract-config.md#sizedintegertypes), and a property that becomes an unsigned 64-bit integer is refused: `"minimum": 0` with no `maximum` does, so add a `maximum` of at most 4294967295, or give no bounds at all.
+
+## `documentsAverageable`
+
+| | |
+|---|---|
+| **Where** | document type |
+| **Value** | the name of an integer property, 1 to 64 characters |
+| **Default** | absent |
+| **Since** | protocol version 12 |
+| **On update** | Fixed (40212) |
+
+Shorthand for `documentsCountable: true` plus `documentsSummable` on the named property: the count and the sum an average is computed from. The storage is exactly that of the two flags. When `documentsSummable` is also written it must name the same property, and an explicit `documentsCountable: false` beside it is refused as a contradiction.
+
+## Document type `rangeCountable`, `rangeSummable`, `rangeAverageable`
+
+| | |
+|---|---|
+| **Where** | document type |
+| **Value** | boolean each |
+| **Default** | `false` |
+| **Since** | protocol version 12 |
+| **On update** | Fixed (40212) |
+
+These keep the count, the sum, or both at every node of the primary tree rather than only at its root, which is what range and offset aggregates over the primary key need. They are rarely useful: a range count or sum is almost always wanted over an indexed property, which is what the [index keywords of the same names](#index-rangecountable-rangesummable-rangeaverageable) give.
+
+- `rangeCountable` implies `documentsCountable`.
+- `rangeSummable` needs `documentsSummable` or `documentsAverageable`.
+- `rangeAverageable` is shorthand for `rangeCountable` plus `rangeSummable`, and needs `documentsAverageable`. An explicit `false` for either of the two beside it is refused as a contradiction.
+
+## `countable`
+
+| | |
+|---|---|
+| **Where** | index |
+| **Value** | `"notCountable"`, `"countable"`, `"countableAllowingOffset"`, or a boolean (`true` is `"countable"`, `false` is `"notCountable"`) |
+| **Default** | `"notCountable"` |
+| **Since** | protocol version 12 |
+| **On update** | Fixed, like every index (`DataContractInvalidIndexDefinitionUpdateError`, 10217) |
+
+Keeps a document count for each value of the index, so "how many documents have these values" is one read per value, with a proof. A count query is served by a countable index whose properties exactly match its `where` clauses, each an equality or an `in`: a countable index on `[brand, color]` answers `brand == "a" AND color == "red"`, and `color in [...]` under a fixed brand with one count per color, but not `brand == "a"` alone. Declare one countable index per shape of count the application needs.
+
+`"countableAllowingOffset"` keeps a count at every node of the index's trees, not only per value. It costs more on every write and prepares the index for offset queries; use `"countable"` unless you need it. The boolean form is kept for contracts written before the string form existed.
+
+On a unique index the flag changes almost nothing, since each value holds at most one document; it only counts the documents that leave an indexed property out.
+
+## `summable`
+
+| | |
+|---|---|
+| **Where** | index |
+| **Value** | the name of an integer property, 1 to 64 characters |
+| **Default** | absent |
+| **Since** | protocol version 12 |
+| **On update** | Fixed (10217) |
+
+Keeps the sum of the named property for each value of the index, the way `countable` keeps a count, and serves sum queries under the same exact-match rule. The property follows the rules of `documentsSummable`: it exists, is required, and fits a signed 64-bit sum.
+
+## `averageable`
+
+| | |
+|---|---|
+| **Where** | index |
+| **Value** | the name of an integer property, 1 to 64 characters |
+| **Default** | absent |
+| **Since** | protocol version 12 |
+| **On update** | Fixed (10217) |
+
+Shorthand for `countable: "countable"` plus `summable` on the named property, which is what an average query per value needs. When `summable` is also written it must name the same property. An explicit `countable: "countableAllowingOffset"` beside it is kept; an explicit `"notCountable"` (or `false`) is refused as a contradiction.
+
+## Index `rangeCountable`, `rangeSummable`, `rangeAverageable`
+
+| | |
+|---|---|
+| **Where** | index |
+| **Value** | boolean each |
+| **Default** | `false` |
+| **Since** | protocol version 12 |
+| **On update** | Fixed (10217) |
+
+These answer aggregates over a **range** of the index's last property: "how many reviews between grade 60 and 80", "total tipped from day 100 to day 130". The count or sum is kept at every node of the tree of that property's values, so a range total is read by walking the edges of the range, with a proof whose size grows with the logarithm of the number of values rather than with the documents. The same index also returns one total per distinct value in a range. The query fixes the properties before the last one, with equalities or an `in`.
+
+- `rangeCountable` makes the index countable: an omitted `countable` becomes `"countable"`, an explicit `"countableAllowingOffset"` is kept, and an explicit `"notCountable"` is refused. Before protocol version 14, `countable` had to be written out beside it.
+- `rangeSummable` needs `summable` or `averageable`.
+- `rangeAverageable` is shorthand for `rangeCountable` plus `rangeSummable`, and needs `averageable`. An explicit `false` for either of the two beside it is refused.
+
+A range total costs more on each write than a per-value total, since every node of the tree carries it. The ranked keywords build on these flags: see [Ranked Indexes](ranked.md).
+
+## How they combine
+
+- **Count and sum on one tree.** An index with both a count and a sum (by `averageable`, or by `countable` plus `summable`) keeps both in one tree, and one proof returns the pair. The same holds for the document type flags.
+- **One summed property per type.** Every `summable`, `averageable`, `documentsSummable` and `documentsAverageable` of a document type must name the same property: the sums share their trees, which have no room to tell two properties apart. To sum two properties, use two document types.
+- **Type and index flags are independent.** `documentsCountable` gives the unfiltered total; a countable index gives filtered ones. A type may have both.
+- **Index-only types** take the index keywords but refuse the document type ones, since they have no primary tree. See [Index-Only Types](index-only.md).
+- **Queries must match.** A count or sum query that no index serves exactly is refused; there is no slow fallback. Pick the indexes for the queries the application will make.
+
+## Rules at registration
+
+- A summed property exists on the type, is an integer that fits a signed 64-bit sum (not an unsigned 64-bit integer), and is listed in `required`.
+- All summed properties of a type are the same property.
+- The shorthands agree with their longhand where both are written, and no explicit `false` or `"notCountable"` contradicts them.
+- Each `range*` flag has its prerequisite, as listed above.
+- The meta-schema refuses a malformed value (`JsonSchemaError`, 10101).
+
+## Choosing what to set
+
+| The application needs | Set |
+|---|---|
+| The number of documents of the type | `documentsCountable: true` |
+| The sum, or the average, of a property over the whole type | `documentsSummable` or `documentsAverageable` |
+| A count per value (`where author == x`) | `countable: "countable"` on an index whose properties are exactly those of the query |
+| A sum or an average per value | `summable` or `averageable` on such an index |
+| A count, sum or average over a range (`where grade > 60`) | the matching `range*` flag on an index whose last property is the ranged one |
+| The top or bottom values by count, sum or average | the `range*` flags plus a ranking: see [Ranked Indexes](ranked.md) |
+
+## See also
+
+- [Document Count Trees](../drive/document-count-trees.md) and [Document Sum Trees](../drive/document-sum-trees.md) for the tree variants, the query endpoints and what each flag costs.
+- [Count Index Examples](../drive/count-index-examples.md), [Sum Index Examples](../drive/sum-index-examples.md) and [Average Index Examples](../drive/average-index-examples.md) for worked queries and proof sizes.
+- [Indexes](indexes.md) for the index keywords every index uses.
+- [Ranked Indexes](ranked.md) for ordering values by these totals.
diff --git a/book/src/contract-keywords/contested.md b/book/src/contract-keywords/contested.md
new file mode 100644
index 00000000000..4d54eb46578
--- /dev/null
+++ b/book/src/contract-keywords/contested.md
@@ -0,0 +1,81 @@
+# Contested Indexes
+
+A unique index normally works first come, first served: the first document to take a value keeps it, and every later one is refused as a duplicate. `contested` changes that for the values a contract considers valuable. A document whose values fall in the contested range opens a **contest**, or joins one already open for the same values, and masternodes and evonodes vote on which identity gets them. DPNS uses it so that a short name such as `alice.dash` cannot simply be taken by whoever submits first. A contract author reaches for it when a unique value is scarce and should be awarded rather than raced for.
+
+| | |
+|---|---|
+| **Where** | a unique index |
+| **Value** | object: `resolution` (required), `fieldMatches`, `description` |
+| **Since** | protocol version 1; `resolution: 1` from protocol version 14 |
+| **On update** | Fixed, like every index (`DataContractInvalidIndexDefinitionUpdateError`, 10217) |
+| **Errors** | `DocumentContestNotPaidForError` (40114), `DocumentContestCurrentlyLockedError` (40110), `DocumentContestNotJoinableError` (40111), `DocumentContestIdentityAlreadyContestantError` (40112), `DocumentContestIndexMismatchError` (40118), `DocumentContestNotRequiredError` (40119), `DocumentContestMaximumContendersReachedError` (40141); `DuplicateUniqueIndexError` (40105) for values outside the contested range |
+
+## Example
+
+The `domain` document type of the DPNS contract:
+
+```json
+{
+ "name": "parentNameAndLabel",
+ "properties": [
+ { "normalizedParentDomainName": "asc" },
+ { "normalizedLabel": "asc" }
+ ],
+ "unique": true,
+ "contested": {
+ "fieldMatches": [
+ { "field": "normalizedLabel", "regexPattern": "^[a-zA-Z01-]{3,19}$" }
+ ],
+ "resolution": 0,
+ "description": "If the normalized label part of this index is less than 20 characters (all alphabet a-z, A-Z, 0, 1, and -) then a masternode vote contest takes place to give out the name"
+ }
+}
+```
+
+A name is unique under its parent domain. A label of 3 to 19 characters made of letters, `0`, `1` and `-` is contested: registering it opens a masternode vote, which may give it to a contender or lock it. Any other label, a longer one or one with other digits, is registered first come, first served, and a second registration of it is a duplicate.
+
+## The keys
+
+| Key | Value | What it does |
+|---|---|---|
+| `resolution` | `0` or `1`, required | How the contest is decided. `0`: masternodes vote for a contender, abstain, or **lock** the value so nobody gets it. `1` (from protocol version 14): masternodes vote for a contender or abstain; there is no lock, so the contest always ends with a winner. |
+| `fieldMatches` | array of at least one `{ "field", "regexPattern" }` | Which values are contested. `field` is a property path of the document; `regexPattern` a regular expression its value must match. Each is 1 to 256 characters. |
+| `description` | string, 1 to 256 characters | A note for readers. Consensus does not read it. |
+
+## How it works
+
+**Which documents are contested.** A document is contested when, for every `fieldMatches` entry, the document holds a string at `field` and `regexPattern` matches it. A missing value, a value that is not a string, or one entry that does not match makes the document an ordinary unique-index document. Without `fieldMatches`, every document the index covers is contested. The pattern uses the syntax of Rust's `regex` crate and matches anywhere in the value, so write `^` and `$` to match the whole of it, as DPNS does.
+
+**Values outside the contested range** behave exactly like any unique index: the first document takes the value, and a later create with the same values is refused with `DuplicateUniqueIndexError` (40105).
+
+**Opening or joining a contest.** A create whose values are contested must carry `$prefundedVotingBalance`, a pair `[indexName, amount]`: the contested index's name and the most credits the contender will pay to fund the vote. The document is not stored under the index yet; it is held as a contender until the contest ends. A create is refused:
+
+- with `DocumentContestNotPaidForError` (40114) when it carries no prefunded balance, or less than the contest's fund. The fund is the contested document fund, 0.1 Dash. From protocol version 14 it doubles once the contest holds 250 contenders and again for every 50 more, and a contender is charged the fund and keeps what it stated beyond it; before 14 every contender stated and paid exactly the fund.
+- with `DocumentContestIndexMismatchError` (40118, from protocol version 14) when the pair names another index than the contested one its values match.
+- with `DocumentContestNotRequiredError` (40119, from protocol version 14) when the document is not contested but carries a prefunded balance.
+- with `DocumentContestNotJoinableError` (40111) when the contest's join window (one week on mainnet) has passed.
+- with `DocumentContestIdentityAlreadyContestantError` (40112) when its owner is already a contender.
+- with `DocumentContestMaximumContendersReachedError` (40141, from protocol version 14) when the contest already holds 1,000 contenders.
+- with `DocumentContestCurrentlyLockedError` (40110) when an earlier contest for these values ended locked.
+
+**The vote.** Masternodes and evonodes vote with `MasternodeVote` transitions for the length of the poll (two weeks on mainnet). Under `resolution: 0` the contender with the most votes wins unless the lock tally beats it, and the contest always runs its full length, even with one contender. Under `resolution: 1` a contender always wins, and a contest that still has a single contender when its join window closes is awarded to it at once. From protocol version 14 a tie goes to the earliest contender.
+
+**After the contest.** The winning document is stored and held by the index like any unique-index document; the other contenders' documents are removed. The contest's result stays readable.
+
+The fund, the windows, the tallies and the special case of moderation elections are described in [Contested Documents](../data-model/contested-documents.md).
+
+## Rules at registration
+
+- The index is `unique: true`. A contested index that is not unique is refused (`InvalidContractStructure`, 10231).
+- The document type's documents cannot be replaced: `documentsMutable: false` (`ContestedUniqueIndexOnMutableDocumentTypeError`, 10248). They may still be transferred and sold, as DPNS domains are.
+- A document type has at most one contested index, and no other unique index beside it (`ContestedUniqueIndexWithUniqueIndexError`, 10249).
+- `resolution` is present and is `0` or `1`; `1` is refused before protocol version 14.
+- `fieldMatches`, when present, holds at least one entry, and each `regexPattern` is a valid regular expression (`RegexError`, 10247).
+- A contested index cannot carry a [`timeRange`](time-range.md) or a ranking (a ranking needs a non-unique index), and an [index-only type](index-only.md) cannot have one.
+- A document type with a contested index cannot set [`ttl`](ttl.md), nor `canBeDeletedByModerators` (see [Deletion](deletion.md)): a moderator's restore puts a document back by an ordinary insert, and a contested value is only awarded through a vote.
+
+## See also
+
+- [Contested Documents](../data-model/contested-documents.md) for the contest's lifecycle, fund, resolutions, ties and storage.
+- [Indexes](indexes.md) for `unique` and the other index keywords.
+- [Mutability](mutability.md) for `documentsMutable`, which a contested type sets to `false`.
diff --git a/book/src/contract-keywords/contract-config.md b/book/src/contract-keywords/contract-config.md
new file mode 100644
index 00000000000..42d952abca7
--- /dev/null
+++ b/book/src/contract-keywords/contract-config.md
@@ -0,0 +1,206 @@
+# Contract-Level Keys and config
+
+The document types sit inside a data contract, which has keys of its own: its id, its owner, its version, the shared definitions its types may point at, its tokens and groups, and a search description. One of them, `config`, holds contract-wide settings: whether the contract can ever change, whether it keeps its history, the defaults its document types inherit, how integers are stored, and whether and how it is moderated. Almost every `config` setting is chosen once, at registration, and kept for the contract's life.
+
+## Example
+
+```json
+{
+ "$formatVersion": "1",
+ "id": "AY6xWncZUFv2GCrS5seqKthUfbW9yYyUXtF8diSuHQ4g",
+ "ownerId": "AtirhSVpAWF7dEt6dLAmesC4Sr1MsJ9bFC1nLAoNnq2S",
+ "version": 1,
+ "config": {
+ "$formatVersion": "2",
+ "canBeDeleted": false,
+ "readonly": false,
+ "keepsHistory": false,
+ "documentsKeepHistoryContractDefault": false,
+ "documentsMutableContractDefault": true,
+ "documentsCanBeDeletedContractDefault": true,
+ "sizedIntegerTypes": true,
+ "moderation": {
+ "banlist": true,
+ "suspensions": true,
+ "moderators": { "$type": "contractOwner" }
+ }
+ },
+ "documentSchemas": {
+ "post": {
+ "type": "object",
+ "properties": {
+ "text": { "type": "string", "minLength": 1, "maxLength": 280, "position": 0 }
+ },
+ "required": ["text"],
+ "additionalProperties": false
+ }
+ },
+ "keywords": ["social", "microblog"],
+ "description": "Short public posts"
+}
+```
+
+A first version of a small social contract. Its config states the defaults explicitly, stores integers in the smallest width their bounds allow, and keeps a banlist and a suspension list that the owner edits. `keywords` and `description` make it findable through the keyword search contract.
+
+## Contract keys
+
+| Key | Value | What it is | Since |
+|---|---|---|---|
+| `$formatVersion` | `"0"` or `"1"` | The contract's serialization format. Format 1 is the default from protocol version 9 and is the one that carries `groups`, `tokens`, `keywords`, `description` and the timestamps. | 1 |
+| `id` | identifier | The contract's id: a hash of `ownerId` and the identity nonce of the create transition. A create whose id is not that hash is refused (`InvalidDataContractIdError`, 10204). | 1 |
+| `ownerId` | identifier | The identity that registers the contract, and the only one that can update it. | 1 |
+| `version` | integer | 1 when the contract is created; each update must raise it by exactly one (`InvalidDataContractVersionError`, 10212). | 1 |
+| `config` | object | Contract-wide settings: see [`config`](#config) below. Absent means the defaults. | 1 |
+| `documentSchemas` | object | The document types, by name. See [`documentSchemas`](#documentschemas). | 1 |
+| `schemaDefs` | object | Definitions every document type may point at with `$ref`. An update may add definitions, not remove them (`IncompatibleDataContractSchemaError`, 10213). | 1 |
+| `groups` | object | Groups of identities that act together, each member with a voting power, whose approval some token actions need. See [Data Contracts](../data-model/data-contracts.md#what-v1-added). Not the same thing as a [contract group](../data-model/contract-groups.md), a set of contracts. | 9 |
+| `tokens` | object | The contract's tokens, keyed by position `0`, `1`, and so on. Document types may charge them with [`tokenCost`](token-cost.md). | 9 |
+| `keywords` | array of strings | Search keywords. See [`keywords` and `description`](#keywords-and-description). | 9 |
+| `description` | string | A short description for search. See [`keywords` and `description`](#keywords-and-description). | 9 |
+| `createdAt`, `updatedAt`, `createdAtBlockHeight`, `updatedAtBlockHeight`, `createdAtEpoch`, `updatedAtEpoch` | numbers | When the contract was created and last updated. The platform sets them; a contract does not write them. | 9 |
+
+### `documentSchemas`
+
+| | |
+|---|---|
+| **Where** | contract |
+| **Value** | object mapping each document type name to its schema |
+| **Since** | protocol version 1 |
+| **On update** | Document types may be added; none may be removed (`DocumentTypeUpdateError`, 40212). Each existing type follows the update rules of its keywords. |
+| **Errors** | `DocumentTypesAreMissingError` (10214), `InvalidDocumentTypeNameError` (10415), at registration |
+
+A contract has at least one document type, unless it defines tokens (`DocumentTypesAreMissingError`, 10214). A name is 1 to 64 ASCII letters, digits, `_` or `-`; from protocol version 14 a name may not contain `-` (`InvalidDocumentTypeNameError`, 10415). The keywords a schema takes are the subject of the rest of this part: see [Contract Keywords](../contract-keywords.md).
+
+### `keywords` and `description`
+
+| | |
+|---|---|
+| **Where** | contract |
+| **Value** | `keywords`: array of at most 50 strings; `description`: string |
+| **Default** | no keywords, no description |
+| **Since** | protocol version 9 |
+| **On update** | May be changed; the search entries are replaced |
+| **Errors** | `TooManyKeywordsError` (10262), `InvalidKeywordLengthError` (10270), `InvalidKeywordCharacterError` (10269), `DuplicateKeywordsError` (10263), `InvalidDescriptionLengthError` (10264) |
+
+The keyword search system contract indexes each contract by its keywords and description, so applications can find contracts by topic. The rules, checked on every create and update:
+
+- At most 50 keywords (10262).
+- Each keyword is 3 to 50 bytes of UTF-8 (10270) and contains no whitespace or control character (10269).
+- No keyword appears twice (10263).
+- A description is 3 to 100 bytes of UTF-8 (10264).
+
+## `config`
+
+| | |
+|---|---|
+| **Where** | contract |
+| **Value** | object: `$formatVersion` and the keys below |
+| **Default** | absent: every key takes its default |
+| **Since** | protocol version 1 |
+| **On update** | The keys below are fixed, with the exceptions each one names (`DataContractConfigUpdateError`, 40002) |
+| **Errors** | `DataContractConfigUpdateError` (40002), `DataContractIsReadonlyError` (40001) |
+
+`$formatVersion` is the config's own version: `"0"` before protocol version 9, `"1"` from 9, and `"2"` from 14, which adds `moderation`. From protocol version 14 every new contract carries config version 2, moderated or not, and an older contract moves to it with its next update.
+
+### `canBeDeleted`
+
+| | |
+|---|---|
+| **Where** | `config` |
+| **Value** | boolean |
+| **Default** | `false` |
+| **Since** | protocol version 1 |
+| **On update** | Fixed (40002) |
+
+Whether the contract itself may ever be deleted. No transition deletes a contract today, so the flag has no effect yet beyond being recorded.
+
+### `readonly`
+
+| | |
+|---|---|
+| **Where** | `config` |
+| **Value** | boolean |
+| **Default** | `false` |
+| **Since** | protocol version 1 |
+| **On update** | Cannot be set by an update (40002) |
+| **Errors** | `DataContractIsReadonlyError` (40001) |
+
+`true` freezes the contract at its first version: every update of it is refused with `DataContractIsReadonlyError` (40001). Only a create can set it, so a contract is read-only from the start or never. A document reference can require its target contract to be read-only (`contractRequirements.readonly`, see [References](refers-to.md)).
+
+### `keepsHistory`
+
+| | |
+|---|---|
+| **Where** | `config` |
+| **Value** | boolean |
+| **Default** | `false` |
+| **Since** | protocol version 1 |
+| **On update** | Fixed (40002) |
+
+`true` makes Drive keep every version of the contract, not only the latest, so a client can read and prove the contract as it was at an earlier version.
+
+### Document type defaults
+
+| | |
+|---|---|
+| **Where** | `config` |
+| **Value** | `documentsKeepHistoryContractDefault`, `documentsMutableContractDefault`, `documentsCanBeDeletedContractDefault`: boolean each |
+| **Default** | `false`, `true`, `true` |
+| **Since** | protocol version 1 |
+| **On update** | Fixed (40002) |
+
+The value of `documentsKeepHistory`, `documentsMutable` and `canBeDeleted` for each document type that does not set it itself. A type that sets the keyword overrides the default. Changing a default would change every type that relies on it, so all three are fixed. See [History](history.md), [Mutability](mutability.md) and [Deletion](deletion.md).
+
+### Bounded key requirements
+
+| | |
+|---|---|
+| **Where** | `config` |
+| **Value** | `requiresIdentityEncryptionBoundedKey`, `requiresIdentityDecryptionBoundedKey`: `0` unique, `1` multiple, `2` multiple with a pointer to the latest |
+| **Default** | absent |
+| **Since** | protocol version 1 |
+| **On update** | Fixed (40002) |
+
+Let identities add encryption, or decryption, keys bound to the whole contract, and say how they are kept: one key that cannot be replaced, several, or several with a pointer to the latest. The document type keywords of the same names do this for keys bound to one document type. See [Signing and Keys](signing-keys.md) and [Contract Bounds](../sdk/identity-keys.md#contract-bounds).
+
+### `sizedIntegerTypes`
+
+| | |
+|---|---|
+| **Where** | `config` |
+| **Value** | boolean |
+| **Default** | `true` in config version 1 and later; config version 0 has no such key and behaves as `false` |
+| **Since** | protocol version 9 |
+| **On update** | May be turned on, not off (40002) |
+
+With `true`, each integer property is stored in the smallest width its `minimum`, `maximum` or `enum` allow: `"minimum": 0, "maximum": 100` takes one byte. With `false`, every integer is a signed 8-byte value. Turning it off would make stored documents unreadable, so it is refused. Turning it on is allowed by the config check, but it changes the width of every bounded integer of the existing document types, and an update that changes how an existing property's values are stored is refused (`DocumentTypeUpdateError`, 40212); in practice it can only be turned on when it leaves the width of every existing integer property unchanged.
+
+The width also decides what a summed property may be: see [Counts, Sums and Averages](aggregates.md#documentssummable).
+
+### `moderation`
+
+| | |
+|---|---|
+| **Where** | `config` (config version 2) |
+| **Value** | object: `banlist`, `suspensions`, `warnings` (booleans, default `false`) and `moderators` |
+| **Default** | absent: the contract is not moderated |
+| **Since** | protocol version 14 |
+| **On update** | Which lists are kept is fixed, and an elected team can be neither declared, changed nor left; otherwise the moderators may change (40002) |
+| **Errors** | `InvalidContractModerationConfigError` (10900), `ContractModeratorIdentityNotFoundError` (41110) |
+
+Declares which moderation lists the contract keeps and who edits them. An identity on the banlist, or suspended, cannot act on the contract's documents; a warning is a record that bars nothing. `moderators` is one of:
+
+- `{ "$type": "contractOwner" }`: the owner moderates alone.
+- `{ "$type": "appointedModerators", "identities": [...] }`: the owner and 1 to 16 named identities, each acting alone. Every named identity must exist (`ContractModeratorIdentityNotFoundError`, 41110).
+- `{ "$type": "elected", ... }`: a team elected by masternodes moderates, with the abilities the declaration gives it. See [Elected Moderation](../data-model/contract-moderation.md#elected-moderation).
+
+A declaration keeps at least one list, unless a document type sets `canBeDeletedByModerators`, and is refused otherwise (`InvalidContractModerationConfigError`, 10900). An unknown key is refused rather than ignored, so a misspelled list name cannot silently leave the contract without it. Because the lists are fixed, a contract that will ever need moderation declares it when it is created.
+
+`moderation` is what `canBeDeletedByModerators` (see [Deletion](deletion.md)) and the moderators' share of [`actionFees`](action-fees.md) require.
+
+## See also
+
+- [Data Contracts](../data-model/data-contracts.md) for the contract structure and its versions.
+- [Contract Moderation](../data-model/contract-moderation.md) for the lists, the moderation transition and elected teams.
+- [Contract Groups](../data-model/contract-groups.md) for contract groups, sets of contracts, which a create transition may register or join (not the contract's `groups`).
+- [Contract Keywords](../contract-keywords.md) for the document type keywords.
diff --git a/book/src/contract-keywords/deletion.md b/book/src/contract-keywords/deletion.md
new file mode 100644
index 00000000000..ed7189bc0f8
--- /dev/null
+++ b/book/src/contract-keywords/deletion.md
@@ -0,0 +1,148 @@
+# Deletion
+
+A document can leave the state three ways: its owner deletes it, the contract's moderators delete it, or the platform deletes it when its time to live runs out. `canBeDeleted` rules the first, `canBeDeletedByModerators` and `canBeDeletedByModeratorsFor` the second, and `ttl` the third (see [Time To Live](ttl.md)). Each is independent of the others: a type may let moderators remove what its authors cannot retract, or expire documents that nobody may delete by hand.
+
+## `canBeDeleted`
+
+Whether a document's owner may delete it. Set it to `false` for records that other documents or other people rely on staying put.
+
+| | |
+|---|---|
+| **Where** | document type |
+| **Value** | boolean |
+| **Default** | the contract config's `documentsCanBeDeletedContractDefault`, which is `true` unless the contract says otherwise |
+| **Since** | protocol version 1 |
+| **On update** | Fixed (`DocumentTypeUpdateError`, 40212), except that a type which keeps history before and after the update may change it from `true` to `false`. Adding or removing the key without changing its value is refused too, as a schema change (`IncompatibleDocumentTypeSchemaError`, 10246). |
+| **Errors** | `InvalidDocumentTransitionActionError` (10404) for a delete of a type set to `false`, or, from protocol version 14, of a type that keeps history; `DocumentOwnerIdMismatchError` (40102) for a delete by anyone but the owner |
+
+### Example
+
+```json
+"comment": {
+ "type": "object",
+ "canBeDeleted": true,
+ "properties": {
+ "postId": {
+ "type": "array",
+ "byteArray": true,
+ "minItems": 32,
+ "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "position": 0
+ },
+ "text": { "type": "string", "maxLength": 500, "position": 1 }
+ },
+ "required": ["postId", "text"],
+ "additionalProperties": false
+}
+```
+
+A commenter may take a comment down at any time. Since `true` is the usual default, the key could be left out; writing it makes the intent plain.
+
+### How it works
+
+- The owner deletes a document with a delete transition that names its id. Anyone else is refused (`DocumentOwnerIdMismatchError`, 40102), and a document that does not exist is `DocumentNotFoundError` (40101).
+- The owner is refunded the part of the document's storage fee that has not yet been paid out to past epochs. A document of a type with a `ttl` refunds nothing. See [Refunds](../fees/overview.md#refunds).
+- A delete may carry a token cost or an action fee, like any document action. See [Token Costs](token-cost.md) and [Action Fees](action-fees.md).
+- An identity that is banned or suspended on a moderated contract may still delete its own documents. See [Contract Moderation](../data-model/contract-moderation.md#the-model).
+- `false` binds only the owner. The contract's moderators, when the type allows them, and the platform, when the type has a `ttl`, still delete such documents.
+- Drive never deletes a document whose type keeps history (`documentsKeepHistory`). From protocol version 14 a delete of such a document is refused with 10404 whatever `canBeDeleted` says; before it, the delete failed inside Drive as an internal error.
+- Documents of an `indexOnly` type are deleted with an index-only delete transition that carries their values, since there is no stored row to name by id. A delete by id of such a document is refused (10404). See [Index-Only Types](index-only.md).
+
+### Rules at registration
+
+- From protocol version 14, a type with `documentsKeepHistory: true` must set `canBeDeleted: false` (`InvalidContractStructure`, 10231). The default is `true`, so it has to be written out. A contract registered earlier with both flags on stays readable, but its next update is checked like a new contract, so that update must turn `canBeDeleted` off on the type. That is the one change to `canBeDeleted` an update may make.
+- For references, a type whose owner may delete its documents is deletable: a `permanentDocument` or `listElement` reference may not point at it (`ReferencedDocumentTypeDeletableError`, 40122), and a `deletableDocument` reference may. See [References](refers-to.md).
+
+## `canBeDeletedByModerators`
+
+Lets the contract's moderators delete documents of the type, whoever owns them. It is how an application takes down content that breaks its rules, where a ban only stops an identity from writing more.
+
+| | |
+|---|---|
+| **Where** | document type, in a contract whose config declares `moderation` |
+| **Value** | boolean |
+| **Default** | `false` |
+| **Since** | protocol version 14 |
+| **On update** | Fixed (`DocumentTypeUpdateError`, 40212) |
+| **Errors** | `DocumentTypeNotDeletableByModeratorsError` (41115), `IdentityNotContractModeratorError` (41101), `ContractModerationTargetNotAllowedError` (41102), `DocumentNotFoundError` (40101), and `DocumentModerationWindowElapsedError` (41116) with a window |
+
+### Example
+
+```json
+"post": {
+ "type": "object",
+ "documentsMutable": true,
+ "canBeDeleted": false,
+ "canBeDeletedByModerators": true,
+ "canBeDeletedByModeratorsFor": 604800,
+ "properties": {
+ "text": { "type": "string", "maxLength": 280, "position": 0 }
+ },
+ "required": ["$createdAt", "$updatedAt", "text"],
+ "additionalProperties": false
+}
+```
+
+Authors cannot retract a post, but the contract's moderators can remove one for a week (604,800 seconds) after it was written or last edited. The contract around it must declare `moderation` in its config.
+
+### How it works
+
+- A moderator deletes a document with the contract user moderation transition, naming the document type, the document id and a reason. The moderators are the ones the contract's `moderation` config declares; see [Contract Moderation](../data-model/contract-moderation.md#the-model).
+- The transition is checked in this order, each refusal paid: the document type exists (`InvalidDocumentTypeError`, 10406); it carries the keyword (41115); the signer is the contract owner or a moderator (41101); the document exists (40101); its owner is neither the contract owner nor a moderator (41102); and, when the type sets `canBeDeletedByModeratorsFor`, the window has not passed (41116).
+- The document and all its index entries are deleted as an owner's delete would delete them, without the `canBeDeleted` check. A removal record is written under the contract: whose document it was, which moderator removed it, the reason, the block time and a hash of the document. The record is never deleted.
+- The document's owner gets no storage refund, and the moderator pays neither the type's delete token cost nor its delete action fee.
+- For a week after the deletion a moderator may restore the document exactly as it was. See [Restoring Documents](../data-model/contract-moderation.md#restoring-documents).
+
+### Rules at registration
+
+All refusals below are `InvalidContractStructure` (10231).
+
+- The contract's config must declare `moderation`. Moderation cannot be added by a later update, so without it nobody could ever delete anything. In return the `moderation` block may keep no banlist, suspension list or warning list at all when a document type carries this keyword.
+- Refused on a type that keeps history (Drive never deletes those documents), on an `indexOnly` type (there is no stored row to name), on a type with `creationRestrictionMode` 1 or 2 (its documents are the contract owner's or the platform's), and on a type with a contested index (a restore could not go through the vote the index requires).
+- For references, the type is deletable even with `canBeDeleted: false`: a `permanentDocument` or `listElement` reference may not point at it (40122), and a `deletableDocument` reference may.
+
+## `canBeDeletedByModeratorsFor`
+
+Limits the moderators' deletion to a window after a document's last change. Once the window has passed the document is settled: moderation acts on what was just written and does not reach back into what has stood unchallenged.
+
+| | |
+|---|---|
+| **Where** | document type, with `canBeDeletedByModerators: true` |
+| **Value** | integer, seconds, 1 to 4294967295 |
+| **Default** | absent: no limit |
+| **Since** | protocol version 14 |
+| **On update** | Fixed (`DocumentTypeUpdateError`, 40212), in both directions: a longer window would reopen documents that had settled |
+| **Errors** | `DocumentModerationWindowElapsedError` (41116) |
+
+### How it works
+
+- The window is measured from the document's `$updatedAt`, or from its `$createdAt` on a type that does not record `$updatedAt`. A deletion at exactly that time plus the window still passes; after it, every moderator is refused, the contract owner included.
+- A replace or a price update moves `$updatedAt`, so new content opens the window again. A transfer or a purchase does not move it.
+- A restored document comes back with its old `$updatedAt`, so it may already be settled.
+- The window says nothing about the document's own owner, whose deletion `canBeDeleted` rules at any age.
+
+### Rules at registration
+
+- Needs `canBeDeletedByModerators: true` (`InvalidContractStructure`, 10231).
+- A type whose documents can be replaced must list `$updatedAt` in `required`: measured from creation alone, an author could wait the window out and then rewrite a post into something no moderator can remove. A type with `documentsMutable: false` must list `$updatedAt` or `$createdAt`. Both refusals are 10231.
+- A window of 0 is refused by the meta-schema (`JsonSchemaError`, 10101). A type that moderators may never delete from simply leaves `canBeDeletedByModerators` out.
+
+## How they combine
+
+| Who deletes | Allowed by | Refund to the owner |
+|---|---|---|
+| The document's owner | `canBeDeleted: true`, on a type that does not keep history | Yes, except on a type with a `ttl` |
+| The contract's moderators | `canBeDeletedByModerators: true`, within `canBeDeletedByModeratorsFor` when set | No |
+| The platform | `ttl`, once it has passed | No |
+
+A type that allows any of the three counts as deletable for references. Only a type that allows none of them can be the target of a `permanentDocument` or `listElement` reference.
+
+## See also
+
+- [Deleting Documents](../data-model/contract-moderation.md#deleting-documents) and [Restoring Documents](../data-model/contract-moderation.md#restoring-documents), for the moderation transition and the removal record
+- [Time To Live](ttl.md), the third way a document leaves the state
+- [History](history.md), for why a type that keeps history can never delete
+- [Mutability](mutability.md) and [Creation, Transfers and Trading](ownership-and-trading.md)
+- [References](refers-to.md), for `permanentDocument` and `deletableDocument`
+- [Contract-Level Keys and config](contract-config.md), for `documentsCanBeDeletedContractDefault` and `moderation`
diff --git a/book/src/contract-keywords/distinct-from.md b/book/src/contract-keywords/distinct-from.md
new file mode 100644
index 00000000000..0c21e801c1b
--- /dev/null
+++ b/book/src/contract-keywords/distinct-from.md
@@ -0,0 +1,82 @@
+# distinctFrom
+
+`distinctFrom` says that an identifier property must hold a different identity (or document) than another identifier of the same document, or than the document's owner. Reach for it when a document names two parties that must not be the same: a delegate who is not the delegator, a buyer who is not the seller, a team member who is not the team's leader. The check reads only the document being written, so it costs no state reads.
+
+| | |
+|---|---|
+| **Where** | An identifier property, at the top level or inside an object; or the `items` of a typed array of identifiers, where it binds every element |
+| **Value** | `"$ownerId"`, or the dotted path of another identifier property of the same document type (`"meta.reviewerId"` for a nested one), 1 to 256 characters |
+| **Default** | Absent: no rule |
+| **Since** | protocol version 14 |
+| **On update** | Fixed: adding, removing or changing it is refused (`IncompatibleDocumentTypeSchemaError`, 10246) |
+| **Errors** | `DocumentPropertyNotDistinctError` (10419) on a document; at registration `JsonSchemaError` (10101) or `InvalidContractStructure` (10231) |
+
+An identifier here is a 32-byte id, declared as a byte array with the identifier `contentMediaType` (see [Property Schemas](property-schemas.md)).
+
+## Example
+
+```json
+"delegation": {
+ "type": "object",
+ "properties": {
+ "delegateId": {
+ "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "distinctFrom": "$ownerId",
+ "position": 0
+ },
+ "backupId": {
+ "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "distinctFrom": "delegateId",
+ "position": 1
+ }
+ },
+ "required": ["delegateId"],
+ "additionalProperties": false
+}
+```
+
+An identity cannot delegate to itself: `delegateId` must differ from the document's owner. The optional `backupId` must differ from `delegateId`; a delegation without a backup passes, since there is nothing to compare.
+
+On a typed array the keyword goes on `items`, and every element must differ from the named value. The moderation charters system contract uses this for a team's members, none of whom may be the leader who writes the document:
+
+```json
+"members": {
+ "type": "array",
+ "minItems": 0,
+ "maxItems": 15,
+ "uniqueItems": true,
+ "items": {
+ "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "distinctFrom": "$ownerId"
+ },
+ "position": 2
+}
+```
+
+## How it works
+
+- **Create and replace.** After the document's properties pass the JSON schema, each declaring property is compared with what it names: the other property's value in the same document, or the writer's identity for `$ownerId`. Equal values refuse the transition with `DocumentPropertyNotDistinctError` (10419), which names the document type, the property and what it collided with. For a typed array each element is compared on its own.
+- **Absent values pass.** When the declaring property or the property it names is left out of the document, there is nothing to compare, and the rule holds. Add the property to `required` when it must be there.
+- **Transfer and purchase.** These change the owner and nothing else. The stored document is judged against its new owner, so a transfer to, or a purchase by, the identity held in a property that must differ from `$ownerId` is refused with the same error. A delegation, say, cannot be transferred to its own delegate. A price update changes neither owner nor data and is not judged.
+- **Deletes** are never judged.
+
+The check reads the transition (or, for a transfer or purchase, the stored document) and never looks anything else up, and a type without declarations pays nothing for it.
+
+## Rules at registration
+
+- The keyword is allowed only on an identifier property or on the identifier `items` of a typed array. On any other property, the typed array itself included, the meta-schema refuses it (`JsonSchemaError`, 10101).
+- The value must be `"$ownerId"` or name a property of the same document type. No other system property is accepted.
+- A named property must exist, must itself be an identifier (an identifier that carries `refersTo` counts), must not be an object, and must not be the declaring property.
+
+The parser refuses a value that breaks the last two rules with `InvalidContractStructure` (10231).
+
+## See also
+
+- [Distinct Identifier Properties](../data-model/documents.md#distinct-identifier-properties), the deep dive
+- [Typed Arrays](typed-arrays.md), for keywords on `items`
+- [propertyConstraints](property-constraints.md): a rule such as `{ "notEqual": ["buyerId", "sellerId"] }` says the same, and can be combined with other conditions
+- [Writer and Creator References](owner-refers-to.md), for rules that look the writer up in state
+- [Contract Keywords overview](../contract-keywords.md), for the conventions of these tables
diff --git a/book/src/contract-keywords/document-shape.md b/book/src/contract-keywords/document-shape.md
new file mode 100644
index 00000000000..d0fb2ac160f
--- /dev/null
+++ b/book/src/contract-keywords/document-shape.md
@@ -0,0 +1,169 @@
+# Document Shape
+
+A document type's schema is a JSON Schema object with some Platform keywords added. The keywords in this chapter give a document its outline: it is an object, it has these properties and no others, some of them must be present, and a few JSON Schema rules hold over the document as a whole. What a single property may hold is described in [Property Schemas](property-schemas.md); what may happen to a document (replace, delete, transfer) has chapters of its own.
+
+## Example
+
+The DashPay `profile` type, with its indexes left out:
+
+```json
+"profile": {
+ "type": "object",
+ "properties": {
+ "avatarUrl": { "type": "string", "format": "uri", "minLength": 1, "maxLength": 2048, "position": 0 },
+ "avatarHash": { "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32, "position": 1 },
+ "avatarFingerprint": { "type": "array", "byteArray": true, "minItems": 8, "maxItems": 8, "position": 2 },
+ "publicMessage": { "type": "string", "minLength": 1, "maxLength": 140, "position": 3 },
+ "displayName": { "type": "string", "minLength": 1, "maxLength": 25, "position": 4 }
+ },
+ "minProperties": 1,
+ "dependentRequired": {
+ "avatarUrl": ["avatarHash", "avatarFingerprint"],
+ "avatarHash": ["avatarUrl", "avatarFingerprint"],
+ "avatarFingerprint": ["avatarUrl", "avatarHash"]
+ },
+ "required": ["$createdAt", "$updatedAt"],
+ "additionalProperties": false
+}
+```
+
+A profile holds at least one of its five properties, and never any other. The three avatar properties come together or not at all. No property of its own is required, but the platform records the time each profile was created and last updated, because `required` lists `$createdAt` and `$updatedAt`.
+
+## `type`
+
+| | |
+|---|---|
+| **Where** | The top of a document type. Required. |
+| **Value** | `"object"`, the only value allowed |
+| **Since** | protocol version 1 |
+| **On update** | Fixed (`IncompatibleDocumentTypeSchemaError`, 10246) |
+| **Errors** | `JsonSchemaError` (10101) at registration |
+
+A document is always an object: a set of named properties. Properties inside a document have their own `type`, described in [Property Schemas](property-schemas.md#type).
+
+## `properties`
+
+| | |
+|---|---|
+| **Where** | The top of a document type. Required. |
+| **Value** | An object mapping 1 to 100 property names to property schemas |
+| **Since** | protocol version 1 |
+| **On update** | Properties may be added, never removed (`IncompatibleDocumentTypeSchemaError`, 10246). An added property is optional, or required with [`requiredSince`](required-since.md). |
+| **Errors** | `MissingPositionsInDocumentTypePropertiesError` (10411), `JsonSchemaError` (10101), both at registration |
+
+`properties` declares every property a document of the type may hold. Each value is the property's schema: its type, its bounds and the Platform keywords that apply to it (see [Property Schemas](property-schemas.md)).
+
+Rules at registration:
+
+- A document type has 1 to 100 properties. A nested object has the same limit.
+- A property name is 1 to 64 letters, digits or underscores (`^[a-zA-Z0-9_]{1,64}$`). Protocol versions 1 to 13 also admitted `-`; from protocol version 14 it is refused.
+- Every property has a [`position`](property-schemas.md#position), a number. The top-level positions must run 0, 1, 2 and so on, with no gap and no number used twice (`MissingPositionsInDocumentTypePropertiesError`, 10411). A document is stored with its properties in `position` order and without their names, so the numbers are what tie the stored bytes to the schema.
+
+On update, a new property takes the next free position. It is optional unless it is listed in `required` with `requiredSince` set to the version the update creates. A property that already exists can never be removed, since stored documents may hold it.
+
+## `additionalProperties`
+
+| | |
+|---|---|
+| **Where** | The top of a document type (required), and every property of type `object` |
+| **Value** | `false`, the only value allowed |
+| **Since** | protocol version 1 |
+| **On update** | Fixed (10246) |
+| **Errors** | `JsonSchemaError` (10101): a document holding a property its type does not declare |
+
+A document holds only the properties its type declares. Writing `additionalProperties: false` says so; Platform accepts no other value.
+
+## `required`
+
+| | |
+|---|---|
+| **Where** | The top of a document type. A property of type `object` has its own `required` for its members (see [Objects](property-schemas.md#objects)). |
+| **Value** | An array of names, none repeated: properties of the type, and the system timestamps and block heights |
+| **Default** | Absent: every property is optional and no timestamp is recorded |
+| **Since** | protocol version 1 |
+| **On update** | May gain only a property the same update adds, annotated with `requiredSince`; may lose nothing (`DataContractInvalidRequiredFieldsUpdateError`, 10276) |
+| **Errors** | `JsonSchemaError` (10101): a created or replaced document leaves out a required property |
+
+`required` names two kinds of thing:
+
+- **The type's own properties.** Every created or replaced document must hold them. A required property is also stored without the presence byte an optional one carries (unless it is [transient](transient.md)), which is why the list is so hard to change later.
+- **System timestamps and block heights**: `$createdAt`, `$updatedAt`, `$transferredAt`, and their `BlockHeight` and `CoreBlockHeight` forms. The writer does not supply these. Listing one tells the platform to record it on every document of the type; one that is not listed is never recorded. See [System Properties](system-properties.md).
+
+Some keywords only work with a timestamp in the list: [`ttl`](ttl.md) needs `$createdAt`, and a [time-range index](time-range.md) needs the timestamp it buckets.
+
+On a contract update, from protocol version 14:
+
+- A property the update adds may be required if it carries `requiredSince` equal to the contract version the update creates. See [requiredSince](required-since.md).
+- An existing optional property may not become required.
+- Nothing may be removed from the list.
+- A system timestamp or block height may not be added. So the set of values a type records is fixed once the type exists.
+
+Each of these is refused with `DataContractInvalidRequiredFieldsUpdateError` (10276). The `required` list of a nested object is fixed (`IncompatibleDocumentTypeSchemaError`, 10246).
+
+## `minProperties` and `maxProperties`
+
+| | |
+|---|---|
+| **Where** | The top of a document type; also on properties of type `object` |
+| **Value** | An integer, 0 or more |
+| **Since** | protocol version 1 (declared in the meta-schema from 12) |
+| **On update** | Fixed (`IncompatibleDocumentTypeSchemaError`, 10246) |
+| **Errors** | `JsonSchemaError` (10101) |
+
+These are the JSON Schema keywords: a document must hold at least `minProperties` and at most `maxProperties` of its own properties. System properties are not counted. The DashPay `profile` above uses `minProperties: 1` so that an empty profile cannot be written.
+
+Meta-schema v0, which covered protocol versions 1 to 11, did not list these keywords at the top of a document type, but it did not refuse keys it did not know either. A contract of that time could use them, and its documents were validated against them: the DashPay contract, registered at protocol version 1, uses `minProperties` and `dependentRequired`. From protocol version 12 the meta-schema lists them and checks their values.
+
+## `dependentRequired`
+
+| | |
+|---|---|
+| **Where** | The top of a document type; also on properties of type `object` |
+| **Value** | An object mapping a property name to an array of property names |
+| **Since** | protocol version 1 (declared in the meta-schema from 12) |
+| **On update** | Entries, and names within an entry, may be removed, and so may the whole keyword; nothing may be added (`IncompatibleDocumentTypeSchemaError`, 10246) |
+| **Errors** | `JsonSchemaError` (10101) |
+
+The JSON Schema keyword: when a document holds the property named by a key, it must also hold every property in that key's array. In the example, a profile with an `avatarUrl` must also have an `avatarHash` and an `avatarFingerprint`. Removing an entry only lets more documents through, which is why removal is the only change allowed.
+
+## `$comment` and `description`
+
+| | |
+|---|---|
+| **Where** | The top of a document type; also on any property |
+| **Value** | A string |
+| **Since** | protocol version 1 |
+| **On update** | Free: may be added, changed or removed |
+| **Errors** | none |
+
+Notes for people reading the contract. Consensus does not act on them.
+
+## `$schema` and `$defs`
+
+| | |
+|---|---|
+| **Where** | Added by the platform; a document type does not write them |
+| **Value** | `$schema`: the document meta-schema's URL. `$defs`: the contract's `schemaDefs`. |
+| **Since** | protocol version 1 |
+| **Errors** | `InvalidContractStructure` (10231): a document type that writes either key |
+
+When Platform reads a document type, it adds two keys before validating the schema:
+
+- `$schema`, the URL of the document meta-schema, so the schema is checked against it.
+- `$defs`, holding the contract's `schemaDefs`: definitions shared by every document type of the contract. A property then refers to one with [`$ref`](property-schemas.md#ref), as `"$ref": "#/$defs/"`.
+
+A document type that writes either key itself is refused. Definitions live only at the contract level, in `schemaDefs`, and are checked as part of each document type: 1 to 100 of them, each named like a property and each a property schema. A contract update may add definitions but not remove them, and a change to a definition follows the update rules of the keywords it holds (`IncompatibleDataContractSchemaError`, 10213). See [Contract-Level Keys and config](contract-config.md).
+
+## Limits on the whole type
+
+- **Name.** A document type's name, its key in `documentSchemas`, is 1 to 64 letters, digits or underscores (`InvalidDocumentTypeNameError`, 10415). Up to protocol version 13 it could also hold `-`.
+- **Nesting.** The schema, with the definitions it reaches through `$ref`, nests at most 256 levels of objects and arrays (`DataContractMaxDepthExceedError`, 10200). A `$ref` that does not resolve, or that leads back to itself, is refused (`InvalidJsonSchemaRefError`, 10207).
+- **Known keys only.** From protocol version 12 a key the meta-schema does not know is refused at the top of a document type (`JsonSchemaError`, 10101). Before 12 such a key was accepted without being checked. The keys are listed in the [overview](../contract-keywords.md#every-keyword).
+
+## See also
+
+- [Property Schemas](property-schemas.md), for what goes inside `properties`
+- [System Properties](system-properties.md), for the timestamps `required` can record
+- [requiredSince](required-since.md), for adding a required property in an update
+- [Evolving a Contract: Adding Required Fields](../data-model/data-contracts.md#evolving-a-contract-adding-required-fields)
+- [Document Serialization](../serialization/document-serialization.md#user-defined-properties), for how `position` and `required` shape the stored bytes
diff --git a/book/src/contract-keywords/encrypted-for.md b/book/src/contract-keywords/encrypted-for.md
new file mode 100644
index 00000000000..945190a7ec2
--- /dev/null
+++ b/book/src/contract-keywords/encrypted-for.md
@@ -0,0 +1,115 @@
+# encryptedFor
+
+`encryptedFor` marks a byte array property as ciphertext that one identity can read, and writes the recipe into the contract: who the message is for, which identity keys were used, and which encryption scheme made the bytes. Wallets and SDKs read the recipe from the contract instead of from per-app documentation. Reach for it when a document carries a private message, a private note to self, or any other value only its recipient should read. Consensus cannot see inside the ciphertext: it checks only that the bytes have the length the scheme produces.
+
+| | |
+|---|---|
+| **Where** | A byte array property (`byteArray: true`) that is not an identifier, at the top level or inside an object. Not on the elements of a typed array |
+| **Value** | An object with exactly four keys, all required: `recipient`, `recipientKey`, `senderKey`, `scheme` (below) |
+| **Default** | Absent: the property is plain bytes |
+| **Since** | protocol version 14 |
+| **On update** | Fixed: adding, removing or changing it is refused (`IncompatibleDocumentTypeSchemaError`, 10246). Documents already written could not be read under another recipe |
+| **Errors** | `InvalidEncryptedPropertyShapeError` (10420) on a document; at registration `JsonSchemaError` (10101) or `InvalidContractStructure` (10231) |
+
+The four keys:
+
+| Key | Value |
+|---|---|
+| `recipient` | The dotted path of an identifier property of the same document type, whose value is the recipient identity's id; or `"$ownerId"` for a message the writer encrypts to themself |
+| `recipientKey` | The dotted path of an integer property of the same document type that carries the id of the recipient's identity key. Its schema must declare `minimum` of at least 0 and `maximum` of at most 4294967295 |
+| `senderKey` | The same for the sender's identity key, a key of the document's owner (`$ownerId`) |
+| `scheme` | `"ecdh-secp256k1-aes256-cbc"`, the only scheme today |
+
+The three paths are 1 to 256 characters each.
+
+## Example
+
+```json
+"directMessage": {
+ "type": "object",
+ "documentsMutable": false,
+ "properties": {
+ "recipientId": {
+ "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "position": 0
+ },
+ "recipientKeyId": { "type": "integer", "minimum": 0, "maximum": 4294967295, "position": 1 },
+ "senderKeyId": { "type": "integer", "minimum": 0, "maximum": 4294967295, "position": 2 },
+ "encryptedMessage": {
+ "type": "array", "byteArray": true, "minItems": 32, "maxItems": 1040,
+ "encryptedFor": {
+ "recipient": "recipientId",
+ "recipientKey": "recipientKeyId",
+ "senderKey": "senderKeyId",
+ "scheme": "ecdh-secp256k1-aes256-cbc"
+ },
+ "position": 3
+ }
+ },
+ "required": ["recipientId", "recipientKeyId", "senderKeyId", "encryptedMessage"],
+ "additionalProperties": false
+}
+```
+
+A message is written by its owner to the identity in `recipientId`. The owner used their key `senderKeyId` and the recipient's key `recipientKeyId`, and the ciphertext is 32 to 1040 bytes, room for a plaintext of up to 1023 bytes.
+
+## The scheme
+
+`ecdh-secp256k1-aes256-cbc` is the scheme the DashPay contact request already uses for its encrypted fields:
+
+1. The shared key is the secp256k1 ECDH of the sender's private key and the recipient's public key: the SHA-256 of the product point's parity byte and x coordinate. The recipient derives the same 32 bytes from their own private key and the sender's public key.
+2. The writer draws a random 16-byte IV.
+3. The stored value is the IV followed by the plaintext encrypted with AES-256-CBC under the shared key and that IV, with PKCS7 padding.
+
+A ciphertext is therefore `16 + 16 * ceil((plaintext length + 1) / 16)` bytes: at least 32, and always a multiple of 16. There is no authentication tag, so a reader with the wrong key usually fails the padding check, but about once in 256 attempts gets garbage instead. Readers should treat a value that does not decrypt as a bad message, not as a protocol error.
+
+The SDKs do this from the declaration. The Rust SDK's `dash_sdk::platform::encrypted_for` module has `encrypt_property` (which also fills in both key id properties) and `decrypt_property`; the JavaScript SDK has `sdk.encryptedFor.encrypt`, `decrypt` and `envelope`.
+
+## How it works
+
+- **Create and replace.** After the JSON schema validation, each property that declares `encryptedFor` and is present in the transition is checked for its shape: at least 32 bytes (the IV and one block) and a multiple of 16. A value that is not refuses the transition with `InvalidEncryptedPropertyShapeError` (10420), which names the property, the scheme and the lengths. The schema's own `minItems` and `maxItems` are checked first, so a value outside them gets the schema's error instead.
+- **Nothing else is checkable on chain.** Consensus does not know whether the bytes decrypt, whether the key ids exist on the identities, or whether those keys have an encryption purpose. A writer can store any 32 bytes.
+- **Checking the keys.** To have consensus check that the keys exist and are of the right kind, add [references](refers-to.md) of type `identityPublicKey` next to the declaration. The moderation charters system contract does this: the recipient's key must be a decryption key and the sender's an encryption key.
+
+```json
+"recipientId": {
+ "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "refersTo": {
+ "type": "identityPublicKey",
+ "keyIdProperty": "recipientKeyId",
+ "keyRequirements": { "purpose": "decryption" }
+ },
+ "position": 0
+},
+"senderKeyId": {
+ "type": "integer", "minimum": 0, "maximum": 4294967295,
+ "refersTo": {
+ "type": "identityPublicKey",
+ "identityProperty": "$ownerId",
+ "keyRequirements": { "purpose": "encryption" }
+ },
+ "position": 2
+}
+```
+
+`encryptedFor` neither requires nor duplicates these references: it describes the recipe, and the references hold the keys to it.
+
+## Rules at registration
+
+- The keyword is allowed only on a byte array that is not an identifier. On an identifier or any other property the meta-schema refuses it (`JsonSchemaError`, 10101), and so it does an unknown key, a missing key or another `scheme`.
+- `recipient` must name an identifier property of the document type (an identifier that carries `refersTo` counts) or be `"$ownerId"`. No other `$` name is accepted.
+- `recipientKey` and `senderKey` must name integer properties of the document type whose schemas declare `minimum` of at least 0 and `maximum` of at most 4294967295, the range of a key id. System properties are refused.
+- None of the three named properties may be `transient` or sit inside a transient object: a transient value is never stored, so a stored ciphertext would lose its recipe.
+- The byte array's `maxItems` must be at least 32, the shortest ciphertext the scheme produces.
+
+A registration refusal from the parser is `InvalidContractStructure` (10231).
+
+## See also
+
+- [Encrypted Properties](../data-model/documents.md#encrypted-properties-encryptedfor), the deep dive, with [the scheme's layout](../data-model/documents.md#the-ecdh-secp256k1-aes256-cbc-layout) and [what consensus checks, and what it cannot](../data-model/documents.md#what-consensus-checks-and-what-it-cannot)
+- [References (refersTo)](refers-to.md), for `identityPublicKey` references
+- [Signing and Keys](signing-keys.md), for encryption and decryption keys bound to a document type
+- [transient](transient.md)
+- [Contract Keywords overview](../contract-keywords.md), for the conventions of these tables
diff --git a/book/src/contract-keywords/generated-from.md b/book/src/contract-keywords/generated-from.md
new file mode 100644
index 00000000000..766537a582c
--- /dev/null
+++ b/book/src/contract-keywords/generated-from.md
@@ -0,0 +1,98 @@
+# generatedFrom
+
+`generatedFrom` says that the platform generates a string property's value with a system function of other properties of the same document, its params. The functions change the case of a string (`lowercase`, `uppercase`, `capitalize`, `camelCase`, `snakeCase`), or fold a name so that names differing only by case or by look-alike characters read the same (`homographSafeASCII`): reach for that one when a unique index must treat `Bob`, `BOB` and `B0B` as one name. A client may leave the property out, and the platform generates it when the document arrives; a client that sends it has it checked. The check reads only the document being written, so it costs no state reads.
+
+| | |
+|---|---|
+| **Where** | A string property, at the top level or inside an object. Not on a typed array or its `items`, and not beside `$ref` |
+| **Value** | `{ "function": , "params": [, ...] }`: a system function, and as many params as it takes, each the dotted path of a property of the same document type (`"profile.display"` for a nested one), 1 to 256 characters (ASCII, as property names are, so as many bytes) |
+| **Default** | Absent: no rule |
+| **Since** | protocol version 14 |
+| **On update** | Fixed: adding, removing or changing it is refused (`IncompatibleDocumentTypeSchemaError`, 10246); a property an update adds may declare it only when one of its params is new too (`DocumentTypeUpdateError`, 40212) |
+| **Errors** | `DocumentPropertyNotGeneratedError` (10424) on a document; at registration `JsonSchemaError` (10101) or `InvalidContractStructure` (10231); on update `IncompatibleDocumentTypeSchemaError` (10246) or `DocumentTypeUpdateError` (40212) |
+
+## Example
+
+```json
+"handle": {
+ "type": "object",
+ "indices": [
+ { "name": "byNormalizedLabel", "properties": [{ "normalizedLabel": "asc" }], "unique": true }
+ ],
+ "properties": {
+ "label": {
+ "type": "string", "pattern": "^[a-zA-Z0-9-]{3,63}$", "maxLength": 63,
+ "position": 0
+ },
+ "normalizedLabel": {
+ "type": "string", "maxLength": 63,
+ "generatedFrom": {
+ "function": "sys.stringTransformations.homographSafeASCII",
+ "params": ["label"]
+ },
+ "position": 1
+ }
+ },
+ "required": ["label", "normalizedLabel"],
+ "additionalProperties": false
+}
+```
+
+A client creates `{ "label": "Bob" }` and the stored document holds `{ "label": "Bob", "normalizedLabel": "b0b" }`. A second handle `{ "label": "B0B" }` generates the same `b0b` and is refused by the unique index. A client that sends `{ "label": "Bob", "normalizedLabel": "b0b" }` gets the same result; one that sends `"normalizedLabel": "bob"` is refused with `DocumentPropertyNotGeneratedError`.
+
+The generated property needs no `pattern` of its own. Every value it can hold is what the function generates from params that passed their own patterns: here `label` admits ASCII letters, digits and `-`, so `normalizedLabel` can only ever hold `a` to `z` without `i`, `l` and `o`, digits and `-`. It keeps `maxLength` because it is indexed, and an indexed string declares one of at most 63.
+
+This is the rule the DPNS `domain` type's data trigger checks today for `normalizedLabel` and `normalizedParentDomainName`, written into the schema.
+
+## Functions
+
+Functions are system functions, built into the platform and named under `sys.`; any other name is refused at registration. The prefix keeps them apart from functions a contract may bring in a later protocol version. They are grouped in namespaces, and each one declares how many params it takes. The `sys.stringTransformations` functions each take one string:
+
+| Function | Returns | Example |
+|---|---|---|
+| `sys.stringTransformations.lowercase` | The string with `A` to `Z` lowercased | `Hello World` to `hello world` |
+| `sys.stringTransformations.uppercase` | The string with `a` to `z` uppercased | `Hello World` to `HELLO WORLD` |
+| `sys.stringTransformations.capitalize` | The first character uppercased and every other lowercased | `hELLO wORLD` to `Hello world` |
+| `sys.stringTransformations.camelCase` | The words joined, the first lowercased and every later one capitalized | `Hello World`, `hello_world` and `HelloWorld` to `helloWorld` |
+| `sys.stringTransformations.snakeCase` | The words lowercased and joined with `_` | `Hello World`, `helloWorld` and `hello-world` to `hello_world` |
+| `sys.stringTransformations.homographSafeASCII` | The string with `A` to `Z` lowercased, then `o` turned into `0` and `i` and `l` into `1` | `Lil-Olive` to `111-011ve` |
+
+The string transformations change ASCII characters only and keep every other character as it is (`Olé` becomes `01é` under `homographSafeASCII`, `OLÉ` becomes `olÉ` under `lowercase`). They use no Unicode tables: Unicode case mappings change between releases of the tools a node is built with, and two nodes that lowercased a character differently would disagree about a document. On ASCII, `homographSafeASCII` is exactly what the DPNS trigger computes.
+
+`camelCase` and `snakeCase` split the string into words. Every ASCII character that is neither a letter nor a digit (a space, `-`, `_`, `.` and so on) separates words and is dropped. A word also starts at an ASCII uppercase letter that follows anything other than another ASCII uppercase letter (`helloWorld` is `hello`, `World`), or that follows one and is followed by an ASCII lowercase letter (`XMLHttpRequest` is `XML`, `Http`, `Request`, so it becomes `xmlHttpRequest` or `xml_http_request`). Characters outside ASCII belong to the word they are in. Every transformation gives back its own output unchanged.
+
+`lowercase`, `uppercase`, `capitalize` and `homographSafeASCII` keep the length of the value, and `camelCase` never lengthens it, but `snakeCase` can: `aBcDeF` becomes `a_bc_de_f`. Give a generated property a `maxLength` that admits what its function can generate from its params' longest value.
+
+A function refuses nothing. Which characters a value may hold is the job of each param's `pattern`: `homographSafeASCII` resists look-alike names only where that pattern admits ASCII alone, as DPNS's does. A contract that admits other scripts can hold two names that look alike but generate different values.
+
+## Params
+
+For now a param is always a property of the same document, written as its dotted path. The form leaves room to add, in a later protocol version, literals (`{ "const": ... }`), system values such as `"$ownerId"` and nested calls, without changing what parses today.
+
+## How it works
+
+- **Generated on arrival.** When a document create or replace, or the values of an [index-only](index-only.md) delete, leaves the property out and supplies every param, the platform writes the generated value into the document before anything reads it: contest detection, the schema validation, the indexes and the stored document all see it. A property the document sends is left as sent. A generated value then goes through the property's own schema like a sent one, so any bound it declares, such as `maxLength`, should admit every value the function can generate from its params.
+- **Checked after the JSON schema.** Wherever a document's properties are validated, on every create and replace included and in a client that validates a document before sending it, the property must hold what the function generates from its params, and must be absent when a param is. A document that repeats a key in an object on the way to the property or to a param is refused too, since the value it holds there would be ambiguous. A property that breaks this refuses the transition with `DocumentPropertyNotGeneratedError` (10424), which names the document type, the property, the function and its params. A schema error on any of the values is reported first.
+- **Absent params.** A property one of whose params is absent must be absent too. To make the params required in effect, list the generated property in `required`: a document without them then fails the schema.
+- **Replace.** A replace is judged on the whole new document. Leave the property out to have it generated from the new params; a stale value sent with changed params is refused.
+- **Transfers, purchases and deletes by id** do not change the data and are not judged. An index-only delete is: its values are generated and checked like a create's, so a stale value, or one without its params, refuses it.
+
+The SDK's transition builders generate the property from the document's params, replacing any value the document holds and leaving it out when a param is absent, so a transition built from a document carries the value the platform would generate, and contest detection sees it. A document fetched, edited and sent back through them therefore carries the value of its new params, not the stale one. The property-constraint pre-checks of the JavaScript and FFI SDKs judge the document the same way. A client that validates a document it built, before building a transition, should generate it first (in Rust, `DocumentTypeBasicMethods::regenerate_generated_properties`) or set the value; otherwise the local check reports the property missing. The proof a client verifies after a create or replace is checked against the document with the generated value, as the platform stored it.
+
+## Rules at registration
+
+- The keyword is allowed only on a string property, and not beside `$ref`, whose definition replaces every keyword written next to it (declare it in the definition instead). On any other property, a typed array and its `items` included, the meta-schema refuses it (`JsonSchemaError`, 10101).
+- `function` must name a system function, and `params` must list 1 to 16 params, as many as the function takes. The meta-schema refuses an unknown function or an empty or overlong list (`JsonSchemaError`, 10101).
+- Every param must name another string property of the same document type (not an object, not a system property, not the declaring property), and that property may not be generated itself.
+- Neither the declaring property nor a param may be [transient](transient.md) or sit inside a transient object: a transient value is never stored.
+- Every param must sit inside every object that holds the declaring property: a top-level property may take any param, but `profile.normalized` must take params inside `profile`. A document that supplies the params then always holds the object the platform writes the value into.
+- On a contract update, a new property may declare `generatedFrom` only when one of its params is new too. Documents stored before the update were never generated, so a new generated property whose params all existed is refused (`DocumentTypeUpdateError`, 40212).
+
+The meta-schema refuses the shape errors of the first two rules with `JsonSchemaError` (10101). The parser refuses a function with the wrong number of params, and a declaration that breaks the param rules (the third to fifth), with `InvalidContractStructure` (10231). The update rule refuses with `DocumentTypeUpdateError` (40212).
+
+## See also
+
+- [Generated Properties](../data-model/documents.md#generated-properties-generatedfrom), the deep dive
+- [Property Schemas](property-schemas.md), for `pattern` and `required`
+- [Indexes (indices)](indexes.md), for the unique index that usually reads the generated property
+- [Contract Keywords overview](../contract-keywords.md), for the conventions of these tables
diff --git a/book/src/contract-keywords/history.md b/book/src/contract-keywords/history.md
new file mode 100644
index 00000000000..9613be9b572
--- /dev/null
+++ b/book/src/contract-keywords/history.md
@@ -0,0 +1,143 @@
+# History
+
+Platform can keep two kinds of history for a document type. `documentsKeepHistory` keeps every version of each document in Drive, under the document itself, so an application can read what a document said at any earlier time. The three `keeps*History` flags record events instead: each transfer, purchase or price update of a document becomes a record in the document history system contract, where it can be queried by document, by contract, by identity and by time.
+
+## `documentsKeepHistory`
+
+Keeps every version of every document of the type, not only the latest.
+
+| | |
+|---|---|
+| **Where** | document type |
+| **Value** | boolean |
+| **Default** | the contract config's `documentsKeepHistoryContractDefault`, which is `false` unless the contract says otherwise |
+| **Since** | protocol version 1 |
+| **On update** | Fixed (`DocumentTypeUpdateError`, 40212). Adding or removing the key without changing its value is refused too, as a schema change (`IncompatibleDocumentTypeSchemaError`, 10246). |
+| **Errors** | `InvalidDocumentTransitionActionError` (10404) for a delete of a document of the type, from protocol version 14 |
+
+### Example
+
+```json
+"profile": {
+ "type": "object",
+ "documentsKeepHistory": true,
+ "canBeDeleted": false,
+ "properties": {
+ "displayName": { "type": "string", "maxLength": 25, "position": 0 },
+ "bio": { "type": "string", "maxLength": 140, "position": 1 }
+ },
+ "required": ["displayName"],
+ "additionalProperties": false
+}
+```
+
+Every edit of a profile adds a version, and every earlier version stays readable. `canBeDeleted: false` has to be written out, since a type that keeps history can never delete and the default is `true`.
+
+### How it works
+
+- Each document is stored as a small tree of its own: every version the document has had, keyed by the block time it was written at, and a pointer to the current one. A query by id or through an index sees the current version.
+- Every write that changes the document adds a version: a replace, and also a transfer, a price update or a purchase.
+- Nothing is ever removed. Drive refuses to delete a document whose type keeps history, so its owner's delete is refused (10404 from protocol version 14; before it, the delete failed inside Drive as an internal error), and the type can have neither moderator deletion nor a `ttl`.
+- The `getDocumentHistory` query returns a document's versions from a given time on, each with the block time it was written at, at most 10 per request, and with a proof when asked.
+- Every version stays stored, paid for by the write that added it.
+- A doctype-level sum or average (`documentsSummable`, `documentsAverageable`) counts only each document's current version. See [Counts, Sums and Averages](aggregates.md).
+
+### Rules at registration
+
+All refusals below are `InvalidContractStructure` (10231).
+
+- From protocol version 14, the type must set `canBeDeleted: false`. A contract registered earlier with both flags on stays readable, and its next update must turn `canBeDeleted` off on that type. See [Deletion](deletion.md).
+- Refused together with `ttl`, with `canBeDeletedByModerators` and with `indexOnly`.
+
+## `keepsTransferHistory`
+
+Records every transfer of a document of the type in the document history contract.
+
+| | |
+|---|---|
+| **Where** | document type |
+| **Value** | boolean |
+| **Default** | `false` |
+| **Since** | protocol version 13 |
+| **On update** | Fixed (`DocumentTypeUpdateError`, 40212) |
+| **Errors** | none of its own |
+
+## `keepsPurchaseHistory`
+
+Records every purchase of a document of the type in the document history contract.
+
+| | |
+|---|---|
+| **Where** | document type |
+| **Value** | boolean |
+| **Default** | `false` |
+| **Since** | protocol version 13 |
+| **On update** | Fixed (`DocumentTypeUpdateError`, 40212) |
+| **Errors** | none of its own |
+
+## `keepsPricingHistory`
+
+Records every price update of a document of the type in the document history contract.
+
+| | |
+|---|---|
+| **Where** | document type |
+| **Value** | boolean |
+| **Default** | `false` |
+| **Since** | protocol version 13 |
+| **On update** | Fixed (`DocumentTypeUpdateError`, 40212) |
+| **Errors** | none of its own |
+
+## The document history contract
+
+### Example
+
+```json
+"ticket": {
+ "type": "object",
+ "documentsMutable": false,
+ "canBeDeleted": false,
+ "transferable": 1,
+ "tradeMode": 1,
+ "keepsTransferHistory": true,
+ "keepsPurchaseHistory": true,
+ "keepsPricingHistory": true,
+ "properties": {
+ "event": { "type": "string", "maxLength": 63, "position": 0 },
+ "seat": { "type": "string", "maxLength": 10, "position": 1 }
+ },
+ "required": ["event", "seat"],
+ "additionalProperties": false
+}
+```
+
+Each time a ticket is given away, listed or sold, a record of it is written to the document history contract, so anyone can look up who held a ticket, what it was offered at and what it sold for. The DPNS `domain` type sets all three flags from protocol version 13.
+
+### How records work
+
+The document history contract is a system contract registered at protocol version 13, with the id `6voHRaoiPcfmMhbqCA9dixH98xcgPQ9UEcuaXjpVu3LD`. It has one document type per flag:
+
+| Flag | Record type | Written for | Properties | `$ownerId` of the record |
+|---|---|---|---|---|
+| `keepsTransferHistory` | `transfer` | each transfer | `dataContractId`, `documentTypeName`, `documentId`, `toIdentityId` | the sender |
+| `keepsPurchaseHistory` | `purchase` | each purchase | `dataContractId`, `documentTypeName`, `documentId`, `sellerId`, `price` | the buyer |
+| `keepsPricingHistory` | `priceUpdate` | each price update | `dataContractId`, `documentTypeName`, `documentId`, `price` | the owner who set the price |
+
+- The platform writes the record as part of the transition that made the change, so a record exists exactly when the transfer, purchase or price update succeeded. Each record also carries `$createdAt` and `$createdAtBlockHeight`, the block's time and height.
+- The identity that signed the action owns the record, and writing it is part of that transition's fees.
+- Records can never be changed or deleted, and nobody can create one directly: the record types set `documentsMutable: false`, `canBeDeleted: false` and `creationRestrictionMode: 2`.
+- The record types are indexed for lookups by document (`byDocument`: contract, document, time) and by contract (`byContract`: contract, time). Transfers are also indexed by sender (`from`) and recipient (`to`), purchases by buyer (`buyer`), seller (`seller`) and price (`byPrice`).
+- They also keep provable aggregates. `purchase` keeps a count, total and average of `price` over all its records, and per contract or per document over a time range. `priceUpdate` keeps a count of all its records, and a count, total and average of asking prices per contract or per document over a time range, where a document listed three times counts three times. `transfer` keeps a count of all its records, and per contract over a time range. See [Counts, Sums and Averages](aggregates.md).
+- A flag only records the action the type allows. `keepsTransferHistory` on a type that is not transferable records nothing; no rule ties the flags to `transferable` or `tradeMode`.
+- The flags are fixed on update, so an existing type cannot start or stop recording. A type added by an update may set them.
+
+### Rules at registration
+
+- An `indexOnly` type may keep no history of either kind. See [Index-Only Types](index-only.md).
+
+## See also
+
+- [Creation, Transfers and Trading](ownership-and-trading.md), for the actions the three flags record
+- [Deletion](deletion.md) and [Time To Live](ttl.md), for why a type that keeps history can never lose a document
+- [Counts, Sums and Averages](aggregates.md), for the aggregates of the history records
+- [Contract-Level Keys and config](contract-config.md), for `documentsKeepHistoryContractDefault` and the contract's own `keepsHistory`
diff --git a/book/src/contract-keywords/index-only.md b/book/src/contract-keywords/index-only.md
new file mode 100644
index 00000000000..0128ad4632c
--- /dev/null
+++ b/book/src/contract-keywords/index-only.md
@@ -0,0 +1,187 @@
+# Index-Only Types
+
+Some documents are nothing but a position: a like says which post, which hashtag and which identity, and nothing else. Stored as an ordinary document, a like pays for a serialized body, a row in the primary tree and a reference in every index, for a fact its index entries already hold. An **index-only** type stores no body and no row: its index entries are its documents. That cuts the storage of a small document by more than half, and makes each index a uniqueness rule. In exchange, its documents can only be created and deleted, every property must live in an index or in the entry's value, and a query returns documents rebuilt from index entries rather than fetched by `$id`.
+
+Five keywords shape an index-only type: `indexOnly` and `entryPayload` on the document type, and `terminal`, `preallocated` and `skipIfAbsent` on its indexes. All of them arrived at protocol version 14 and are fixed once the type exists.
+
+## Example
+
+A `like` of a social contract whose `post` type cannot be deleted:
+
+```json
+"like": {
+ "type": "object",
+ "indexOnly": true,
+ "documentsMutable": false,
+ "canBeDeleted": true,
+ "indices": [
+ {
+ "name": "byHashtagPost",
+ "properties": [{ "hashtag": "asc" }, { "postId": "asc" }],
+ "countable": "countable",
+ "rangeCountable": true,
+ "rankedCountable": true,
+ "skipIfAbsent": true
+ },
+ {
+ "name": "byPost",
+ "properties": [{ "postId": "asc" }],
+ "countable": "countable",
+ "rangeCountable": true,
+ "rankedCountable": true
+ },
+ { "name": "byLiker", "properties": [{ "$ownerId": "asc" }], "terminal": "postId" }
+ ],
+ "properties": {
+ "hashtag": { "type": "string", "minLength": 1, "maxLength": 63, "position": 0 },
+ "postId": {
+ "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "refersTo": {
+ "type": "permanentDocument",
+ "documentType": "post",
+ "propertyAgreement": { "hashtag": "hashtag" }
+ },
+ "position": 1
+ }
+ },
+ "required": ["postId"],
+ "additionalProperties": false
+}
+```
+
+`byPost` holds one entry per post and liker (its terminal defaults to `$ownerId`), so an identity can like a post once, and it counts and ranks posts by likes. `byHashtagPost` ranks the posts under each hashtag, and only likes that carry a hashtag enter it. `byLiker` lists the posts one identity liked. The reference makes sure the post exists and that a like's hashtag is its post's.
+
+## `indexOnly`
+
+| | |
+|---|---|
+| **Where** | document type |
+| **Value** | boolean |
+| **Default** | `false` |
+| **Since** | protocol version 14 |
+| **On update** | Fixed (`DocumentTypeUpdateError`, 40212) |
+| **Errors** | `DuplicateUniqueIndexError` (40105), `DocumentNotFoundError` (40101), `InvalidDocumentTransitionActionError` (10404) |
+
+`true` stores the type's documents only as index entries. Each entry sits under the index's values, is keyed by the terminal's values in place of the document id, and holds a 32-byte **row commitment**: a hash over all of the document's values that ties its entries in the different indexes together as one document.
+
+**Create.** A create writes one entry into each index. If any of those entries already exists the create is refused with `DuplicateUniqueIndexError` (40105), so every index is a uniqueness rule over its properties and its terminal. `refersTo` and the other property checks run as on any document.
+
+**Delete.** A document has no id to delete by. It is deleted with an `indexOnlyDelete` transition that carries all of its values (and `$createdAt` when the type requires it); every entry those values produce must exist and carry the matching row commitment, or the delete is refused with `DocumentNotFoundError` (40101). Deleting by id on an index-only type, or with `indexOnlyDelete` on an ordinary type, is refused with `InvalidDocumentTransitionActionError` (10404). The owner can only ever reach its own entries, since every index holds `$ownerId`. `canBeDeleted: false` forbids deletes as on any type.
+
+**No other action.** A document cannot be replaced, transferred, sold or repriced.
+
+**Queries.** A query goes through one index and returns documents rebuilt from its entries: the index's properties, the terminal, and `$ownerId` and `$createdAt` where the index holds them. A query through an index that holds only some of the properties yields only those. The rebuilt `$id` is a hash of the entry's position and addresses nothing, so there is no fetch by `$id` and no `startAt` cursor; a query pages by the terminal instead (`postId > `, with a limit). A query that sets the terminal with an equality can put an `in` on the index's last property, with a limit of at least its number of values; a range on an index property in such a query is refused, because its pages could hold fewer rows than exist. List the equality-bound properties first in an index, and page by a range on its terminal instead. The proof that a create or delete took effect is the presence or absence of its entry in the **proof index**, an index that involves no `$createdAt` and does not skip.
+
+Rules at registration:
+
+- `documentsMutable: false`; `transferable` `0`; `tradeMode` `0`; no `documentsKeepHistory` or `keeps*History`; no `transient` properties.
+- None of the document type aggregate keywords (`documentsCountable` and the others): the type has no primary tree. The index keywords of [Counts, Sums and Averages](aggregates.md) and [Ranked Indexes](ranked.md) are allowed.
+- At least one index. No index is `unique` or `contested`, and none sets `nullSearchable: false`.
+- Every index holds `$ownerId`, as a property or in its terminal.
+- The only system properties an index may list are `$ownerId` and `$createdAt`, and an indexed `$createdAt` must be in `required`.
+- At least one index involves no `$createdAt` and does not set `skipIfAbsent`: the proof index.
+- Every property is in `required`, except the first property of a `skipIfAbsent` index. An object holding an indexed property is required too.
+- Every property appears in at least one index that does not skip, as a property or a terminal component, except the `entryPayload` properties and a skip index's first property.
+- The type cannot also set [`ttl`](ttl.md) or `canBeDeletedByModerators`, and a `refersTo` lookup cannot target it.
+
+## `entryPayload`
+
+| | |
+|---|---|
+| **Where** | document type |
+| **Value** | array of 1 to 16 distinct property names, each 1 to 64 characters |
+| **Default** | absent |
+| **Since** | protocol version 14 |
+| **On update** | Fixed (40212). The list is read as a set, so reordering it is no change. |
+
+The properties stored in each entry's value, after the row commitment, instead of in a key. They are for data the application reads but never queries by, such as a public key or a ciphertext: they need not be indexed, and they come back with every query result.
+
+```json
+"indices": [{ "name": "byRequest", "terminal": ["appEphemeralPubKeyHash", "$ownerId"] }],
+"entryPayload": ["walletEphemeralPubKey", "encryptedPayload"]
+```
+
+With a flat index keyed by a request hash and the responder, this is a key-value table: a query on the hash returns every responder with its public key and ciphertext.
+
+Rules at registration:
+
+- Only on an `indexOnly` type.
+- Each entry names a top-level property that is `required`, a scalar (not an object or an array of values), and bounded: `maxLength` on a string, `maxItems` on a byte array.
+- A payload property appears in no index, neither as a property nor in a terminal.
+- The largest size each payload property can take, plus two bytes each, adds up to at most 5120 bytes. With several indexes, the payload is repeated in every entry.
+
+## `terminal`
+
+| | |
+|---|---|
+| **Where** | index of an `indexOnly` type |
+| **Value** | a property name, or an array of 1 to 10 distinct names |
+| **Default** | `"$ownerId"` |
+| **Since** | protocol version 14 |
+| **On update** | Fixed, like every index (`DataContractInvalidIndexDefinitionUpdateError`, 10217) |
+
+Where an ordinary index keys each entry by the document id, an index-only index keys it by the terminal's values: the **member key**. There is one entry per index values and member key, so the terminal decides what the index makes unique. `byPost` above, with the default terminal, allows one like per post and owner; `byLiker`, with `postId` as terminal under `$ownerId`, holds the same pairs the other way round.
+
+An array is a composite terminal whose values are joined in order. An index with no `properties` at all is a **flat** index, keyed by its terminal alone, as `byRequest` above is.
+
+A query that fixes every property of the index can test one member key ("did I like this post") or walk the member keys in order, a page at a time.
+
+Rules at registration:
+
+- Only on an `indexOnly` type.
+- Each component is `$ownerId` or a property of the type that could be indexed: not an object or an array of values, a string with `maxLength` of at most 63, a byte array with `maxItems` of at most 255. No other system property.
+- Every component but the last has a fixed width: a byte array with `minItems` equal to `maxItems`, an identifier, an integer or a boolean. A string can only be last.
+- The whole member key is at most 255 bytes. On a flat index, the level key, the component names each preceded by a zero byte, is at most 255 bytes as well.
+- A component is not one of the index's `properties`, and not an optional property.
+- A flat index takes no count, sum, ranking, `timeRange`, `skipIfAbsent` or `preallocated` keyword.
+
+## `preallocated`
+
+| | |
+|---|---|
+| **Where** | index of an `indexOnly` type |
+| **Value** | boolean |
+| **Default** | `false` |
+| **Since** | protocol version 14 |
+| **On update** | Fixed (10217) |
+
+The first entry under a new set of values pays for every tree on its path; later ones pay for one entry. When the whole path is decided by a referenced document, `preallocated: true` creates the trees when that document is created, paid by its creator, so every entry costs the same from the first one on. Deleting the last entry keeps the trees, so a post with no likes still shows in the rankings with a count of zero.
+
+In the example, `byHashtagPost` could be preallocated: `postId` is the reference and `hashtag` agrees with the post's. `byLiker` could not, since no post decides who likes it.
+
+Rules at registration:
+
+- Only on an `indexOnly` type.
+- Every index property is either a property with a `permanentDocument` reference to a type of the same contract, or a key of that reference's `propertyAgreement`. A `deletableDocument` reference does not qualify, since the trees would outlive a deleted target. `$ownerId` may only be the terminal.
+- The referenced property of each such agreement key holds at most 255 bytes, since creating a referenced document makes its value an index key (40126 when the contract is created or updated).
+- Not with `timeRange`.
+
+A referenced document whose agreed value takes more bytes than the referring property can hold preallocates nothing for that index, since no entry could agree with it.
+
+## `skipIfAbsent`
+
+| | |
+|---|---|
+| **Where** | index of an `indexOnly` type |
+| **Value** | boolean |
+| **Default** | `false` |
+| **Since** | protocol version 14 |
+| **On update** | Fixed (10217) |
+
+A document that leaves out the index's first property writes no entry into this index, and its delete looks for none. The index then holds only the documents that carry the property, and its counts and rankings are "among the documents that have it". It is the one way a property of an index-only type can be optional: in the example, a like without a hashtag is not in `byHashtagPost` and pays nothing for it. A present but empty value is not absent and is indexed.
+
+A query only uses a skip index when it constrains or orders by the index's first property, so a query cannot silently miss the documents that lack it.
+
+Rules at registration:
+
+- Only on an `indexOnly` type.
+- The first property is a top-level property of the type, not a system property, and not listed in `required`.
+- Every index that involves an optional property sets `skipIfAbsent` and lists that property first; an optional property is never a terminal.
+
+## See also
+
+- [Index-Only Document Types](../drive/index-only-document-types.md) for the entry layout, the row commitment, the full constraint list and the query surface.
+- [Indexes](indexes.md), [Counts, Sums and Averages](aggregates.md) and [Ranked Indexes](ranked.md) for the index keywords an index-only type uses.
+- [References (refersTo)](refers-to.md) for `permanentDocument` references and `propertyAgreement`, which `preallocated` relies on.
+- [Mutability](mutability.md), [Deletion](deletion.md) and [Creation, Transfers and Trading](ownership-and-trading.md) for the flags an index-only type must set.
diff --git a/book/src/contract-keywords/indexes.md b/book/src/contract-keywords/indexes.md
new file mode 100644
index 00000000000..918eb584d12
--- /dev/null
+++ b/book/src/contract-keywords/indexes.md
@@ -0,0 +1,176 @@
+# Indexes (indices)
+
+A document type's `indices` list says which queries its documents can answer and which values must be unique. Without an index, a query can only address a document by its `$id`. With one, Drive keeps the documents sorted by the index's properties, so a query that fixes those properties reaches the matching documents directly instead of reading the whole type. Every index costs storage and processing on each write, and an index can never be added, removed or changed once the document type exists, so a contract author decides them before the document type is registered.
+
+This chapter covers the four keywords every index uses: `name`, `properties`, `unique` and `nullSearchable`. The other index keywords, for contests, counts, sums, rankings, time windows and index-only types, have their own chapters (see [More index keywords](#more-index-keywords)).
+
+## Example
+
+```json
+"post": {
+ "type": "object",
+ "indices": [
+ { "name": "byOwnerTime", "properties": [{ "$ownerId": "asc" }, { "$createdAt": "asc" }] },
+ { "name": "bySlug", "properties": [{ "$ownerId": "asc" }, { "slug": "asc" }], "unique": true },
+ { "name": "byTopic", "properties": [{ "meta.topic": "asc" }], "nullSearchable": false }
+ ],
+ "properties": {
+ "slug": { "type": "string", "minLength": 1, "maxLength": 63, "position": 0 },
+ "text": { "type": "string", "maxLength": 280, "position": 1 },
+ "meta": {
+ "type": "object",
+ "properties": {
+ "topic": { "type": "string", "minLength": 1, "maxLength": 32, "position": 0 }
+ },
+ "additionalProperties": false,
+ "position": 2
+ }
+ },
+ "required": ["$createdAt", "slug", "text"],
+ "additionalProperties": false
+}
+```
+
+`byOwnerTime` lists one author's posts in the order they were written. `bySlug` lets each author use a slug once: a second post by the same owner with the same slug is refused. `byTopic` finds posts by a property nested inside `meta`, and leaves out posts that have no topic.
+
+## `indices`
+
+| | |
+|---|---|
+| **Where** | document type |
+| **Value** | array of 1 to 10 index objects |
+| **Default** | absent: the type has no index, and its documents can only be addressed by `$id` |
+| **Since** | protocol version 1 |
+| **On update** | Fixed: an index may not be added, removed or changed (`DataContractInvalidIndexDefinitionUpdateError`, 10217) |
+| **Errors** | `DuplicateUniqueIndexError` (40105) |
+
+Each entry of `indices` is one index. Drive builds a tree for it when the contract is registered and maintains it on every create, replace, transfer, purchase, price update and delete of a document of the type.
+
+A query uses an index when its `where` clauses fix the index's leading properties, in order, and it orders by the properties that follow. An index on `[a, b]` answers `a == x`, `a == x AND b == y`, and `a == x` ordered by `b`, but not `b == y` alone. The query picker and the tree layout are described in [Indexes](../drive/indexes.md#query-traversal).
+
+Rules at registration:
+
+- At most 10 indexes, and at least one when the key is present (an empty `indices` array is refused by the meta-schema).
+- No two indexes may have the same properties in the same order (`DuplicateIndexError`, 10201).
+- At most one index may be contested, and a type with a contested index may have no other unique index (`ContestedUniqueIndexWithUniqueIndexError`, 10249). See [Contested Indexes](contested.md).
+
+## `name`
+
+| | |
+|---|---|
+| **Where** | index |
+| **Value** | string, 1 to 32 characters |
+| **Default** | none: required |
+| **Since** | protocol version 1 |
+| **On update** | Fixed: indexes are compared by name, so renaming an index is a removal plus an addition (10217) |
+| **Errors** | `DuplicateIndexNameError` (10211) at registration |
+
+The name identifies the index in queries that name one, in error messages and in the contract update rule. Two indexes of one document type may not share a name (`DuplicateIndexNameError`, 10211).
+
+## `properties`
+
+| | |
+|---|---|
+| **Where** | index |
+| **Value** | array of 1 to 10 objects, each `{ "": "asc" }` with exactly one key |
+| **Since** | protocol version 1 |
+| **On update** | Fixed (10217) |
+| **Errors** | `UndefinedIndexPropertyError` (10209), `SystemPropertyIndexAlreadyPresentError` (10208), `InvalidIndexPropertyTypeError` (10206), `InvalidIndexedPropertyConstraintError` (10205), all at registration |
+
+The indexed properties, in order. The order matters: a query uses the index through a prefix of the list. The only sort order the meta-schema accepts is `"asc"`; a query may still walk an index in descending order.
+
+What may be indexed:
+
+- **A top-level property** of the type, by its name.
+- **A property inside an object**, by its dotted path. DPNS indexes `records.identity`, the `identity` property of a domain's `records` object.
+- **System properties**: `$ownerId`, `$createdAt`, `$updatedAt`, `$transferredAt`, their `*BlockHeight` and `*CoreBlockHeight` variants, and `$creatorId` on a type that records it (see [System Properties](system-properties.md)). A timestamp or block height is only recorded when the type lists it in `required`; an index on one that is not required holds every document under null.
+- **Not `$id`**, which the document type's primary tree already indexes (`SystemPropertyIndexAlreadyPresentError`, 10208).
+
+What each indexed property must be, because its value becomes a GroveDB key of at most 255 bytes:
+
+- A property the type defines (`UndefinedIndexPropertyError`, 10209).
+- Not an object, an array of values or a typed array (`InvalidIndexPropertyTypeError`, 10206). A byte array, including an identifier, is fine.
+- A string must declare `maxLength` of at most 63, since a character can take four bytes. A byte array must declare `maxItems` of at most 255. A missing or larger bound is refused (`InvalidIndexedPropertyConstraintError`, 10205). An index with a ranking has tighter bounds (see [Ranked Indexes](ranked.md)).
+- From protocol version 14, not a transient property, nor one inside a transient object: its value is never stored, so the index would never hold it. See [transient](transient.md).
+
+## `unique`
+
+| | |
+|---|---|
+| **Where** | index |
+| **Value** | boolean |
+| **Default** | `false` |
+| **Since** | protocol version 1 |
+| **On update** | Fixed (10217) |
+| **Errors** | `DuplicateUniqueIndexError` (40105) |
+
+On a unique index no two documents may hold the same values for all of the index's properties. A create, replace, transfer, purchase or price update that would make a second document with the same values is refused with `DuplicateUniqueIndexError` (40105); so is a moderator's restore of a deleted document whose values have been taken since. A replace that leaves the indexed values as they were is not held against the document itself.
+
+A document that leaves any of the indexed properties out is not held to the index: uniqueness cannot be decided on a missing value, so two such documents may coexist. Make the properties `required` when every document must be unique.
+
+Uniqueness is per index. `bySlug` above is unique over the pair (`$ownerId`, `slug`), so two authors may use the same slug; a unique index on `slug` alone would make each slug global.
+
+A unique index with a [`timeRange`](time-range.md) treats two documents in the same window as equal on the bucketed property. A unique index cannot carry a ranking.
+
+## `nullSearchable`
+
+| | |
+|---|---|
+| **Where** | index |
+| **Value** | boolean |
+| **Default** | `true` |
+| **Since** | protocol version 1 |
+| **On update** | Fixed (10217) |
+
+With the default, a document whose indexed properties are all missing is still entered in the index, under null, so a query for the null value finds it. `false` leaves such a document out of this index: it exists, and other indexes and `$id` still reach it, but this index does not. A document with only some of the properties missing is entered either way.
+
+DPNS sets `"nullSearchable": false` on its `records.identity` index, so domains that point at no identity take no room in it.
+
+`nullSearchable: false` is refused on an index with a ranking or a `timeRange`, and on an index of an index-only type.
+
+## Null handling
+
+A property is null in an index when the document leaves it out. Putting the rules above together:
+
+| The document's indexed values | Entered in the index? | Held to `unique`? |
+|---|---|---|
+| all present | yes | yes |
+| some missing | yes, under null for the missing ones | no |
+| all missing | only if `nullSearchable` is `true` | no |
+
+The storage shape of each case is in [Null Handling](../drive/indexes.md#null-handling).
+
+## Changing indexes
+
+Drive builds an index's trees when the document type is registered and never backfills them, so an index added later would miss every existing document, and a removed or changed one would leave orphaned trees. A contract update may therefore not add, remove or change any index of an existing document type. From protocol version 14 the update is refused with `DataContractInvalidIndexDefinitionUpdateError` (10217), naming the first index that differs. Indexes are compared by name: reordering the `indices` array is no change, and renaming an index is a removal plus an addition. Earlier protocol versions refused every such change as well, though not always with this error.
+
+A document type that the update adds may declare any indexes, as a new contract may.
+
+## Limits
+
+| Limit | Value |
+|---|---|
+| Indexes per document type | 10 |
+| Properties per index | 10 |
+| Characters in an index name | 32 |
+| Characters in an index property path | 256 |
+| `maxLength` of an indexed string | 63 (lower on a ranked index) |
+| `maxItems` of an indexed byte array | 255 (lower on a ranked index) |
+| Contested indexes per document type | 1 |
+
+## More index keywords
+
+An index entry may carry more keywords, each with its own chapter:
+
+- [Contested Indexes](contested.md): `contested` turns a unique index into a scarce resource that masternodes award by vote, the way DPNS gives out names.
+- [Counts, Sums and Averages](aggregates.md): `countable`, `summable`, `averageable` and their `range*` forms keep totals per indexed value, so counts, sums and averages are read without walking the documents.
+- [Ranked Indexes](ranked.md): `rankedCountable`, `rankedSummable` and `rankedAverageable` order the indexed values by those totals, for "top 10" queries with proofs.
+- [Time-Range Indexes](time-range.md): `timeRange` groups documents into time windows, for "trending this hour" queries.
+- [Index-Only Types](index-only.md): `terminal`, `preallocated` and `skipIfAbsent` shape the indexes of a type whose documents live only in their indexes.
+
+## See also
+
+- [Indexes](../drive/indexes.md) in the Drive part: the index trie, the GroveDB layout and the query picker.
+- [Contested Indexes](contested.md), [Counts, Sums and Averages](aggregates.md), [Ranked Indexes](ranked.md), [Time-Range Indexes](time-range.md), [Index-Only Types](index-only.md).
+- [System Properties](system-properties.md) for what `$ownerId`, `$createdAt` and the others hold.
+- [Contract Keywords](../contract-keywords.md) for the conventions of these chapters.
diff --git a/book/src/contract-keywords/max-bytes.md b/book/src/contract-keywords/max-bytes.md
new file mode 100644
index 00000000000..fcc6b54fcbc
--- /dev/null
+++ b/book/src/contract-keywords/max-bytes.md
@@ -0,0 +1,61 @@
+# maxBytes
+
+`maxBytes` caps how many bytes a string may take when it is encoded as UTF-8, which is how Platform stores it. JSON Schema's `maxLength` counts characters, and one character takes from one to four bytes, so `maxLength` alone does not bound the stored size. Reach for `maxBytes` when the size of a document matters: to keep storage fees predictable, or to stay under the 5120 bytes any one stored value may take.
+
+| | |
+|---|---|
+| **Where** | A string property, at the top level or inside an object; or the `items` of a typed array of strings, where it bounds every element |
+| **Value** | An integer from 1 to 65535, no lower than the property's `minLength` |
+| **Default** | Absent: only `maxLength` and the 5120-byte cap on every value apply |
+| **Since** | protocol version 14 |
+| **On update** | May be raised or removed; adding it or lowering it is refused (`IncompatibleDocumentTypeSchemaError`, 10246) |
+| **Errors** | `DocumentPropertyMaxBytesExceededError` (10421) on a document; at registration `JsonSchemaError` (10101) or `InvalidContractStructure` (10231) |
+
+## Example
+
+```json
+"description": {
+ "type": "string",
+ "minLength": 1,
+ "maxLength": 4096,
+ "maxBytes": 4096,
+ "position": 1
+}
+```
+
+This is the `description` of a proposal in the moderation charters system contract. `maxLength` admits 4096 characters, which could take up to 16384 bytes (more than the 5120-byte cap on a stored value); `maxBytes` holds the stored text to 4096 bytes, so a description in plain ASCII can use every character and one written only in four-byte characters, such as most emoji, a quarter of them.
+
+On a typed array the keyword goes on `items`:
+
+```json
+"tags": {
+ "type": "array",
+ "maxItems": 8,
+ "items": { "type": "string", "minLength": 1, "maxLength": 32, "maxBytes": 64 },
+ "position": 2
+}
+```
+
+Each of up to 8 tags is at most 32 characters and at most 64 bytes.
+
+## How it works
+
+- The check runs wherever a document's properties are validated: every create and replace in consensus, and every client that validates a document before sending it.
+- It runs after the JSON schema validation. A value the schema refuses (too many characters, the wrong type) is reported with the schema's error, not this one.
+- Every string the document holds for a property that declares `maxBytes` is measured in UTF-8 bytes. One that is longer refuses the transition with `DocumentPropertyMaxBytesExceededError` (10421), which names the property and both lengths. For a typed array the error names the element, as in `tags[2]`.
+- A property the document leaves out is not checked.
+
+The cap bounds the value only; `maxLength` and `minLength` still apply in characters. Setting both is normal: `maxLength` says what a user may type, `maxBytes` what the platform stores.
+
+## Rules at registration
+
+- The keyword is allowed only on a string property or on the string `items` of a typed array. On any other property, the typed array itself included, the meta-schema refuses it (`JsonSchemaError`, 10101).
+- The value is an integer from 1 to 65535 (10101).
+- It may not be lower than `minLength`: a string of `minLength` characters takes at least that many bytes, so a lower cap would refuse every value (`InvalidContractStructure`, 10231).
+
+## See also
+
+- [Byte Caps on Strings](../data-model/documents.md#byte-caps-on-strings-maxbytes), the deep dive
+- [Property Schemas](property-schemas.md), for `maxLength` and `minLength`
+- [Typed Arrays](typed-arrays.md), for keywords on `items`
+- [Contract Keywords overview](../contract-keywords.md), for the 5120-byte value limit and the conventions of these tables
diff --git a/book/src/contract-keywords/mutability.md b/book/src/contract-keywords/mutability.md
new file mode 100644
index 00000000000..ff8350f2aca
--- /dev/null
+++ b/book/src/contract-keywords/mutability.md
@@ -0,0 +1,155 @@
+# Mutability
+
+These three keywords decide what a replace may change once a document exists. A replace is the transition an owner sends to overwrite a document with a new version of it. `documentsMutable` turns replaces on or off for the whole document type. `immutable` freezes chosen properties while the rest of the document stays editable, and `immutableAllowSetting` lets some of those frozen properties be filled in once, later, when they were left empty at creation.
+
+None of the three governs deletion, transfers or trading: see [Deletion](deletion.md) and [Creation, Transfers and Trading](ownership-and-trading.md).
+
+## `documentsMutable`
+
+Whether the owner of a document may replace it. Set it to `false` for records that must never change after they are written: votes, receipts, name registrations.
+
+| | |
+|---|---|
+| **Where** | document type |
+| **Value** | boolean |
+| **Default** | the contract config's `documentsMutableContractDefault`, which is `true` unless the contract says otherwise |
+| **Since** | protocol version 1 |
+| **On update** | Fixed (`DocumentTypeUpdateError`, 40212). Adding or removing the key without changing its value is refused too, as a schema change (`IncompatibleDocumentTypeSchemaError`, 10246). |
+| **Errors** | `InvalidDocumentTransitionActionError` (10404) for a replace of a type set to `false` |
+
+### Example
+
+```json
+"vote": {
+ "type": "object",
+ "documentsMutable": false,
+ "canBeDeleted": false,
+ "properties": {
+ "proposalId": {
+ "type": "array",
+ "byteArray": true,
+ "minItems": 32,
+ "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "position": 0
+ },
+ "choice": { "type": "string", "enum": ["yes", "no", "abstain"], "position": 1 }
+ },
+ "required": ["proposalId", "choice", "$createdAt"],
+ "additionalProperties": false
+}
+```
+
+A vote is cast once and stays as cast: no replace is accepted, and with `canBeDeleted: false` its owner cannot take it back either.
+
+### How it works
+
+- With `true`, the document's owner may replace it. The replace carries the whole new document, which is validated against the schema as a create is, and a `$revision` one higher than the stored one (`InvalidDocumentRevisionError`, 40106). Anyone other than the owner is refused (`DocumentOwnerIdMismatchError`, 40102). When the type lists `$updatedAt` in `required`, the replace sets it to the block's time, and likewise `$updatedAtBlockHeight` and `$updatedAtCoreBlockHeight` to the block heights.
+- With `false`, every replace is refused (`InvalidDocumentTransitionActionError`, 10404).
+- A type whose documents cannot be replaced may still let them be transferred or sold (`transferable`, `tradeMode`). Such a document changes owner and price, but its owner can never edit its properties. The DPNS `domain` type works this way.
+- A document stores a `$revision` when its type allows a replace, a transfer or trading. A type that allows none of them stores none. See [System Properties](system-properties.md).
+
+### Rules at registration
+
+- A contested index needs a type whose documents cannot be replaced (`ContestedUniqueIndexOnMutableDocumentTypeError`, 10248). See [Contested Indexes](contested.md).
+- An `indexOnly` type must set `documentsMutable: false`. See [Index-Only Types](index-only.md).
+- `immutable` and `immutableAllowSetting` are only accepted when the type's documents are mutable (`InvalidContractStructure`, 10231).
+- `canBeDeletedByModeratorsFor` on a mutable type needs `$updatedAt` in `required`. See [Deletion](deletion.md).
+
+## `immutable`
+
+The top-level properties that are frozen when a document is created, on a type whose documents can otherwise be replaced. Reach for it when most of a document is editable but some of it is a commitment: the shop an order was placed with, the author of a post, the item that was ordered.
+
+| | |
+|---|---|
+| **Where** | document type, on a type with `documentsMutable: true` |
+| **Value** | array of top-level property names, no repeats |
+| **Default** | empty: every property may change |
+| **Since** | protocol version 14 |
+| **On update** | May gain entries, never lose one (`DocumentTypeUpdateError`, 40212) |
+| **Errors** | `DocumentImmutablePropertyChangedError` (40128) for a replace that changes, adds or removes a listed property |
+
+### Example
+
+```json
+"order": {
+ "type": "object",
+ "documentsMutable": true,
+ "properties": {
+ "shop": {
+ "type": "array",
+ "byteArray": true,
+ "minItems": 32,
+ "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "position": 0
+ },
+ "item": { "type": "string", "maxLength": 100, "position": 1 },
+ "status": { "type": "string", "enum": ["open", "paid", "shipped"], "position": 2 },
+ "trackingCode": { "type": "string", "maxLength": 40, "position": 3 }
+ },
+ "required": ["shop", "item", "status"],
+ "immutable": ["shop", "item", "trackingCode"],
+ "immutableAllowSetting": ["trackingCode"],
+ "additionalProperties": false
+}
+```
+
+The buyer can move `status` along as often as needed, but never change which shop or which item the order is for. `trackingCode` is left out when the order is placed, may be filled in by one later replace, and is frozen from then on.
+
+### How it works
+
+- On every replace, each listed property of the new document is compared with the stored one. A property that differs is refused with `DocumentImmutablePropertyChangedError` (40128). "Differs" covers a changed value, a value the stored document did not have (unless `immutableAllowSetting` allows it, below), and a value the replace leaves out.
+- Values are compared by their data, not their bytes: the order of an object's members and the width an integer is stored in do not count as changes.
+- Freezing an object freezes everything inside it.
+- One change is always allowed: a replace may clear a listed `deletableDocument` reference by id once the document it points to has been deleted. Every replace checks such a reference again, so without this the document could never be replaced again. See [References](refers-to.md).
+- Transfers, price updates and purchases carry no property values, so the list does not affect them.
+
+### Rules at registration
+
+All refusals below are `InvalidContractStructure` (10231).
+
+- Only on a type whose documents are mutable. On a type with `documentsMutable: false` every property is already frozen.
+- Every entry names a declared top-level property. System properties (`$ownerId`, `$createdAt` and the rest) are refused, since the platform manages them. Nested paths such as `meta.author` are refused: list the object that contains them.
+- No entry may be a `transient` property: it is never stored, so every replace that supplies it would count as a change.
+- An immutable property may not hold a `deletableDocument` reference that a replace could not clear once its target is gone: a typed array of them, one inside an object, or one found through a `lookup`. A single reference by id held directly by the property is allowed.
+- On a type whose documents can be transferred or traded, an immutable property may not hold a `contract` reference whose `contractRequirements` has an `owner` requirement: after a change of owner the new owner could neither meet it nor repoint it.
+
+### On update
+
+The list may grow: an update may freeze a property that was editable, and documents already stored keep the values they have. It may never shrink, since documents were written on the promise that those properties would not change. The comparison is of the parsed lists, so reordering them is no change.
+
+## `immutableAllowSetting`
+
+The `immutable` properties that a replace may still set while the stored document has no value for them. It is for optional values that are not known when the document is created, like a tracking code or a closing date, and must not change once they are known.
+
+| | |
+|---|---|
+| **Where** | document type, next to `immutable` |
+| **Value** | array of property names, each also listed in `immutable`, no repeats |
+| **Default** | empty |
+| **Since** | protocol version 14 |
+| **On update** | May lose entries at any time. May gain an entry only for a property that becomes immutable in the same update (`DocumentTypeUpdateError`, 40212). |
+| **Errors** | `DocumentImmutablePropertyChangedError` (40128) for a replace that changes or removes the property once it holds a value |
+
+### How it works
+
+- While the stored document has no value for the property, a replace may set it. That first value is then frozen like the rest of the `immutable` list: it can neither change nor be removed.
+- It only means something for an optional property. A required one always has a value from creation.
+
+### Rules at registration
+
+- Every entry must also be in `immutable` (`InvalidContractStructure`, 10231).
+- An entry may not be a `deletableDocument` reference by id (`InvalidContractStructure`, 10231). Such a reference may be cleared once its target is deleted, and the next replace could then set it again to a different document.
+
+### On update
+
+Dropping an entry tightens the rule and is always allowed. Adding one to a property that was already immutable would let documents change what they were promised to keep, so it is only allowed together with making the property immutable in the same update.
+
+## See also
+
+- [Immutable Properties on Mutable Document Types](../data-model/documents.md#immutable-properties-on-mutable-document-types), for how the replace compares values
+- [Deletion](deletion.md), [Creation, Transfers and Trading](ownership-and-trading.md) and [History](history.md), the other keywords on what may happen to a document
+- [System Properties](system-properties.md), for `$revision` and `$updatedAt`
+- [transient](transient.md) and [References](refers-to.md), for the properties `immutable` refuses
+- [Contract Keywords](../contract-keywords.md#reading-the-chapters), for how the summary tables read
diff --git a/book/src/contract-keywords/owner-refers-to.md b/book/src/contract-keywords/owner-refers-to.md
new file mode 100644
index 00000000000..4cd3b39b94a
--- /dev/null
+++ b/book/src/contract-keywords/owner-refers-to.md
@@ -0,0 +1,123 @@
+# Writer and Creator References
+
+A property's [`refersTo`](refers-to.md) judges a value the writer chose. `ownerRefersTo` and `creatorRefersTo` judge an identity of the document instead: its writer (`$ownerId`) or its creator (`$creatorId`). Each takes the same declaration a property's `refersTo` takes, limited to the targets an identity's id can be. Reach for them to say who may write a document of a type: "only a member of this team", "only someone with a profile". A type declares at most one of the two: `ownerRefersTo` when its documents stay with their owner, `creatorRefersTo` when they can be transferred or traded.
+
+Both are checked the way a property's reference is: the same targets, keys, errors and replace rules, with the identity as the value. They count as one reference each (times the leaves of an expression) against the [reference budget](refers-to.md#the-reference-budget), and the check runs before the properties' references.
+
+## `ownerRefersTo`
+
+| | |
+|---|---|
+| **Where** | The document type, at the top level of its schema. Only on a type whose documents can be neither transferred nor traded. |
+| **Value** | A `refersTo` declaration whose value is the writer: `identity`, a `permanentDocument` or `deletableDocument` with a [`lookup`](refers-to-lookup.md), a [`listElement`](refers-to-list-element.md), or an [`anyOf` or `allOf`](refers-to-expressions.md) whose leaves are all of these. |
+| **Since** | protocol version 14 |
+| **On update** | Fixed: adding, removing or changing it is refused (`IncompatibleDocumentTypeSchemaError`, 10246). |
+| **Errors** | The error the target reports for a property, at the path `$ownerId`: `ReferencedEntityNotFoundError` (40120) when a lookup finds nothing or the writer is not in the list, `ReferencedDocumentPropertyMismatchError` (40127) for an agreement pair. |
+
+### Example
+
+```json
+"post": {
+ "type": "object",
+ "ownerRefersTo": {
+ "type": "deletableDocument",
+ "contractId": "Bwr4WHCPz5rFVAD87RqTs3izo4zpzwsEdKPWUT1NS1C7",
+ "documentType": "profile",
+ "lookup": { "index": "ownerId", "keys": { "$ownerId": "." } }
+ },
+ "properties": {
+ "text": { "type": "string", "minLength": 1, "maxLength": 280, "position": 0 }
+ },
+ "required": ["text"],
+ "additionalProperties": false
+}
+```
+
+Only an identity with a DashPay profile may create a post. In the lookup, `"."` is the writer: DashPay's unique `ownerId` index must find a `profile` owned by them. Profiles can be deleted, so the target is a `deletableDocument` one and every replace asks again: a writer whose profile is gone can no longer edit their posts, though they can still delete them.
+
+The moderation charters contract's `resignationRequest` combines two targets: its writer must be one of the elected `members` of the charter the request names, or have an `addedModerator` document for it that exists now. The declaration is shown under [`anyOf`](refers-to-expressions.md#anyof).
+
+### How it works
+
+- **On create**, the writer is checked against the target exactly as a property's value would be. An `identity` target always holds and reads nothing: the transition has already proved that the writer exists.
+- **In a lookup**, `"."` is the writer, and so is a `"$ownerId"` key part. In a `propertyAgreement`, a pair keyed by `$ownerId` names the same writer.
+- **On replace**, the declaration is checked again when a property its lookup or an agreement pair reads changed, and on every replace when it has a pair keyed by `$ownerId` or a `deletableDocument` lookup (alone or as a leaf). Otherwise nothing is read: the writer is always the owner, a permanent target is never deleted and its key never moves.
+- **Transfers and purchases** cannot happen on such a type, so the owner of every document is a writer that was checked.
+- A refusal names `$ownerId` as its path. Registration errors name the declaration `.$ownerId`.
+
+### Rules at registration
+
+- The document type's documents can be neither transferred nor traded (`transferable` and `tradeMode` absent or `0`). Otherwise a transfer or a purchase, which is not a write, would hand a document to an owner the declaration never checked; declare `creatorRefersTo` instead.
+- The target is one the writer's id can be. `contract`, `token` and a document by id are refused, since an identity's id is never one of those ids, and so is `identityPublicKey`, which pairs the value with a key id the writer does not carry. The same holds for every leaf of an expression.
+- Every other rule is a property reference's: those of the [lookup](refers-to-lookup.md#rules-at-registration), the [list](refers-to-list-element.md#rules-at-registration) and each [`propertyAgreement`](refers-to.md#propertyagreement), checked against another contract's stored type where the declaration names one.
+
+A malformed declaration is refused by the meta-schema (`JsonSchemaError`, 10101) or the parser (`InvalidContractStructure`, 10231); one that cannot hold, with the reference errors of its target (see [Errors](refers-to.md#errors)).
+
+## `creatorRefersTo`
+
+| | |
+|---|---|
+| **Where** | The document type, at the top level of its schema. Only on a type that records creator ids: a transferable or tradeable type of a format-1 contract (see [System Properties](system-properties.md)). |
+| **Value** | A `refersTo` declaration whose value is the creator: `identity`, a `permanentDocument` with a [`lookup`](refers-to-lookup.md), a [`listElement`](refers-to-list-element.md), or an [`anyOf` or `allOf`](refers-to-expressions.md) whose leaves are all of these. |
+| **Since** | protocol version 14 |
+| **On update** | Fixed: adding, removing or changing it is refused (`IncompatibleDocumentTypeSchemaError`, 10246). |
+| **Errors** | The error the target reports for a property, at the path `$creatorId`: `ReferencedEntityNotFoundError` (40120), `ReferencedDocumentPropertyMismatchError` (40127). |
+
+### Example
+
+```json
+"moderatorBadge": {
+ "type": "object",
+ "transferable": 1,
+ "creatorRefersTo": {
+ "type": "listElement",
+ "contractId": "EG7RGfV8fDTayC2FyVr8HwdpJh3fXDbVztcfE94UmN88",
+ "documentType": "electedCharter",
+ "propertyAgreement": { "electedCharterId": "$id" },
+ "inList": "members"
+ },
+ "properties": {
+ "electedCharterId": {
+ "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "position": 0
+ }
+ },
+ "required": ["electedCharterId"],
+ "additionalProperties": false
+}
+```
+
+Only an elected member of the charter `electedCharterId` names, in the moderation charters contract, may mint a badge. Once minted, the badge can be transferred to anyone (`transferable: 1`), and it keeps its creator.
+
+### How it works
+
+- **On create**, the creator is the writer, and is checked as `ownerRefersTo` checks the writer. An `identity` target reads nothing.
+- **On replace**, the value is the stored creator, whoever writes. The declaration is checked again when a property its lookup or an agreement pair reads changed, and on every replace when it has a pair keyed by `$ownerId`. Such a pair still names the writer, not the creator.
+- **Transfers and purchases** need no check: they do not change the creator.
+- **In a lookup**, `"."` is the creator.
+- A refusal names `$creatorId` as its path. Registration errors name the declaration `.$creatorId`.
+
+### Rules at registration
+
+- The document type records creator ids. Such a type's documents can be transferred or traded, so `ownerRefersTo` is refused on it, and a type declares at most one of the two.
+- The targets are those of `ownerRefersTo` except `deletableDocument`: the creator never changes, and a document a transfer handed on could not be replaced by its new owner once the document the lookup found was deleted.
+- A lookup may not read `"$ownerId"`: on a type whose documents can change owner, a transfer or a purchase would move that key part without a write.
+- It can only be declared on a type when the type is created, since an update may not add it. So every document of the type records its creator.
+- Every other rule is as for `ownerRefersTo`.
+
+## Choosing between them
+
+| The type's documents | Declare | Judges |
+|---|---|---|
+| stay with their owner (`transferable` and `tradeMode` absent or `0`) | `ownerRefersTo` | whoever writes the document, who is always its owner |
+| can be transferred or traded | `creatorRefersTo` | the identity that created the document, whoever writes it later |
+
+For a rule about the writer and a document one of its own properties already names, a [`propertyAgreement`](refers-to.md#propertyagreement) pair keyed by `$ownerId` on that property's reference is enough: `{ "$ownerId": "$ownerId" }` requires the writer to own the referenced document. The type-level keywords are for rules no property carries, such as "the writer has a profile" or "the writer is on this list".
+
+## See also
+
+- [References (refersTo)](refers-to.md) for the targets, keys and errors.
+- [Lookups](refers-to-lookup.md), [List Elements](refers-to-list-element.md) and [Expressions](refers-to-expressions.md), the targets a writer or creator reference usually takes.
+- [On the writer or the creator](../data-model/documents.md#on-the-writer-or-the-creator-ownerrefersto-creatorrefersto) in the Documents chapter, with the internals.
+- [System Properties](system-properties.md) for `$ownerId` and `$creatorId`, and [Creation, Transfers and Trading](ownership-and-trading.md) for `transferable` and `tradeMode`.
diff --git a/book/src/contract-keywords/ownership-and-trading.md b/book/src/contract-keywords/ownership-and-trading.md
new file mode 100644
index 00000000000..b14a3e9a572
--- /dev/null
+++ b/book/src/contract-keywords/ownership-and-trading.md
@@ -0,0 +1,119 @@
+# Creation, Transfers and Trading
+
+A document belongs to the identity in its `$ownerId`: at first the one that created it. These three keywords decide who may create documents of a type, and whether a document may later change hands. `creationRestrictionMode` limits who creates. `transferable` lets an owner give a document away. `tradeMode` lets an owner put a price on a document and anyone else buy it at that price.
+
+The example below is used throughout the chapter:
+
+```json
+"ticket": {
+ "type": "object",
+ "documentsMutable": false,
+ "canBeDeleted": false,
+ "creationRestrictionMode": 1,
+ "transferable": 1,
+ "tradeMode": 1,
+ "properties": {
+ "event": { "type": "string", "maxLength": 63, "position": 0 },
+ "seat": { "type": "string", "maxLength": 10, "position": 1 }
+ },
+ "required": ["event", "seat", "$createdAt"],
+ "additionalProperties": false
+}
+```
+
+Only the organiser, the identity that owns the contract, can issue tickets. A ticket's holder may give it to a friend, or list it for sale; anyone may then buy it at the listed price, with no approval from the seller. Nobody can edit a ticket or delete it.
+
+## `creationRestrictionMode`
+
+Who may create documents of the type.
+
+| | |
+|---|---|
+| **Where** | document type |
+| **Value** | `0` anyone, `1` the contract owner only, `2` nobody |
+| **Default** | `0` |
+| **Since** | protocol version 1 |
+| **On update** | Fixed (`DocumentTypeUpdateError`, 40212). Adding or removing the key without changing its value is refused too, as a schema change (`IncompatibleDocumentTypeSchemaError`, 10246). |
+| **Errors** | `DocumentCreationNotAllowedError` (10416) |
+
+### How it works
+
+- `0`: any identity may create documents of the type.
+- `1`: only the identity that owns the contract may. A create signed by anyone else is refused (`DocumentCreationNotAllowedError`, 10416). Once created, a document may still pass to other identities if the type is transferable or tradeable.
+- `2`: no create transition is ever accepted. This is for system contracts whose documents only the platform writes, such as the document history contract (see [History](history.md)) and the keyword search contract. On a user's contract the type would stay empty for good.
+- The mode rules creation only. Replaces, deletes, transfers and sales are decided by each document's own owner and by the other keywords.
+
+### Rules at registration
+
+- A type with mode `1` or `2` may not carry `canBeDeletedByModerators` (`InvalidContractStructure`, 10231): its documents belong to the contract owner or the platform, and no moderator may delete those. See [Deletion](deletion.md).
+
+## `transferable`
+
+Whether an owner may give a document to another identity.
+
+| | |
+|---|---|
+| **Where** | document type |
+| **Value** | `0` never, `1` always |
+| **Default** | `0` |
+| **Since** | protocol version 1 |
+| **On update** | Fixed (`DocumentTypeUpdateError`, 40212). Adding or removing the key without changing its value is refused too, as a schema change (10246). |
+| **Errors** | `InvalidDocumentTransitionActionError` (10404) for a transfer of a type set to `0` |
+
+### How it works
+
+- The owner sends a transfer transition naming the document, the recipient and a `$revision` one higher than the stored one (`InvalidDocumentRevisionError`, 40106). Only the owner may transfer (`DocumentOwnerIdMismatchError`, 40102). The recipient does not have to agree.
+- The document gets the recipient as its `$ownerId` and a new `$revision`. A price it was listed at is removed, so a transferred document is no longer for sale. When the type lists `$transferredAt` in `required`, it is set to the block's time, and likewise `$transferredAtBlockHeight` and `$transferredAtCoreBlockHeight` to the block heights.
+- A transfer carries no property values, and `$creatorId` keeps naming the identity that created the document. The one change the platform makes itself: from protocol version 13, a DPNS `domain` that is transferred or sold has its `records.identity` pointed at the new owner.
+- The document is checked as it will be stored, with its new owner: against the type's unique indexes (`DuplicateUniqueIndexError`, 40105), against a `distinctFrom: "$ownerId"` property (`DocumentPropertyNotDistinctError`, 10419), and against `propertyConstraints` rules that read `$ownerId` (`DocumentPropertyConstraintViolatedError`, 10422).
+- On a moderated contract, a transfer to a banned or suspended identity is refused (`ContractModerationCounterpartyBarredError`, 41114).
+- A transfer of a document past its `ttl` expiry is refused (`DocumentExpiredError`, 40140).
+- With `keepsTransferHistory: true`, each transfer is also recorded in the document history contract. See [History](history.md).
+
+## `tradeMode`
+
+Whether documents of the type can be sold through the platform's built-in marketplace. With `1`, direct purchase, an owner sets a price and any other identity may buy the document at that price, with no approval.
+
+| | |
+|---|---|
+| **Where** | document type |
+| **Value** | `0` none, `1` direct purchase |
+| **Default** | `0` |
+| **Since** | protocol version 1 |
+| **On update** | Fixed (`DocumentTypeUpdateError`, 40212). Adding or removing the key without changing its value is refused too, as a schema change (10246). |
+| **Errors** | `InvalidDocumentTransitionActionError` (10404) for a price update or a purchase on a type set to `0`, and for a purchase by the document's own owner; `DocumentNotForSaleError` (40108); `DocumentIncorrectPurchasePriceError` (40109) |
+
+### How it works
+
+- **Listing.** The owner sends a price update transition with a price in credits and the next `$revision`. Only the owner may set the price (40102). The price is stored with the document, the revision goes up, and `$updatedAt` is set when the type lists it in `required`.
+- **Buying.** Any identity other than the owner sends a purchase transition naming the document, the next `$revision` and the price. A document with no price set is not for sale (`DocumentNotForSaleError`, 40108), and the price in the transition must equal the listed price exactly (`DocumentIncorrectPurchasePriceError`, 40109). An owner cannot buy its own document (10404).
+- **What a purchase does.** The price moves from the buyer's credit balance to the seller's. The document gets the buyer as its `$ownerId` and a new `$revision`, the listing is removed, and `$transferredAt` is set when the type lists it in `required`. The buyer's credit balance must cover the price.
+- The new owner is checked exactly as for a transfer: unique indexes (40105), `distinctFrom: "$ownerId"` (10419) and `propertyConstraints` rules that read `$ownerId` (10422).
+- On a moderated contract, a banned or suspended buyer is refused like any barred writer, and so is a purchase from a banned or suspended seller (`ContractModerationCounterpartyBarredError`, 41114).
+- A price update or a purchase of a document past its `ttl` expiry is refused (`DocumentExpiredError`, 40140).
+- With `keepsPricingHistory` and `keepsPurchaseHistory`, each price update and each purchase is also recorded in the document history contract. See [History](history.md).
+
+## How they combine
+
+- `transferable` and `tradeMode` are independent. The DPNS `domain` type sets both, so a name can be given away or sold. A type may allow sales without gifts, or gifts without sales.
+- Neither needs `documentsMutable`. A document that cannot be replaced can still change owner and price, as the ticket above does.
+- A type that is transferable or tradeable stores a `$revision` on each document even when its documents cannot be replaced, and, from protocol version 10 on a format-1 contract whose config is version 1 or later, records each document's creator in `$creatorId`. See [System Properties](system-properties.md).
+- Every action has its own optional token cost and action fee: `transfer`, `update_price` and `purchase`, besides `create`. See [Token Costs](token-cost.md) and [Action Fees](action-fees.md).
+- `signatureSecurityLevelRequirement` applies to all of these actions, so a buyer signs a purchase with a key at the level the type requires. See [Signing and Keys](signing-keys.md).
+
+## Rules at registration
+
+All refusals below are `InvalidContractStructure` (10231).
+
+- `ownerRefersTo` is refused on a type whose documents can be transferred or traded: a document could end up with an owner the declaration never checked. Such a type uses `creatorRefersTo`, which checks the creator, who never changes. `creatorRefersTo` is only accepted on such a type. See [Writer and Creator References](owner-refers-to.md).
+- An `indexOnly` type can be neither transferable nor tradeable. See [Index-Only Types](index-only.md).
+- On a transferable or tradeable type, an `immutable` property may not hold a `contract` reference with an `owner` requirement. See [Mutability](mutability.md).
+- `creationRestrictionMode` `1` or `2` is refused together with `canBeDeletedByModerators`.
+
+## See also
+
+- [History](history.md), for recording transfers, purchases and price updates
+- [Mutability](mutability.md) and [Deletion](deletion.md), the other keywords on what may happen to a document
+- [System Properties](system-properties.md), for `$ownerId`, `$creatorId`, `$revision` and `$transferredAt`
+- [distinctFrom](distinct-from.md) and [propertyConstraints](property-constraints.md), which also judge a new owner
+- [Contract Moderation](../data-model/contract-moderation.md), for barred counterparties
diff --git a/book/src/contract-keywords/property-constraints.md b/book/src/contract-keywords/property-constraints.md
new file mode 100644
index 00000000000..e12417b9893
--- /dev/null
+++ b/book/src/contract-keywords/property-constraints.md
@@ -0,0 +1,383 @@
+# propertyConstraints
+
+`propertyConstraints` holds named rules that every created or replaced document of a type must meet. JSON Schema bounds one property at a time; these rules relate properties to each other: a deposit that covers price times quantity, percentages that add up to 100, a closed order that carries its closing time, a second party who is not the owner. Each rule is a small tree of comparisons, arithmetic and logic that consensus evaluates against the document. A rule can also read a total of other documents, how many there are or what an integer property adds up to, from the count and sum trees their indexes keep (see [Totals of other documents](#totals-of-other-documents)).
+
+| | |
+|---|---|
+| **Where** | Document type |
+| **Value** | An object of rules, at least one. Each key is the rule's name (1 to 64 letters, digits or underscores); each value is a condition (see [Conditions](#conditions)) |
+| **Default** | Absent: no rules |
+| **Since** | protocol version 14 |
+| **On update** | Fixed: adding, removing or changing a rule is refused (`IncompatibleDocumentTypeSchemaError`, 10246). Stored documents were judged against the rules as they were |
+| **Errors** | `DocumentPropertyConstraintViolatedError` (10422) on a document; at registration `JsonSchemaError` (10101) or `InvalidContractStructure` (10231) |
+
+## Example
+
+```json
+"order": {
+ "type": "object",
+ "properties": {
+ "price": { "type": "integer", "minimum": 0, "maximum": 1000000000, "position": 0 },
+ "fee": { "type": "integer", "minimum": 0, "maximum": 1000000, "position": 1 },
+ "quantity": { "type": "integer", "minimum": 1, "maximum": 10000, "position": 2 },
+ "deposit": { "type": "integer", "minimum": 0, "position": 3 },
+ "status": { "type": "string", "enum": ["open", "pending", "closed"], "position": 4 },
+ "closedAt": { "type": "integer", "minimum": 0, "position": 5 }
+ },
+ "required": ["price", "quantity", "deposit", "status"],
+ "propertyConstraints": {
+ "depositCoversOrder": {
+ "lessThanOrEqual": [
+ { "multiply": [{ "add": ["price", "fee"] }, "quantity"] },
+ "deposit"
+ ]
+ },
+ "feeWaivedOrAtLeastTen": {
+ "anyOf": [{ "equal": ["fee", 0] }, { "greaterThanOrEqual": ["fee", 10] }]
+ },
+ "closedNeedsClosedAt": {
+ "anyOf": [{ "notEqual": ["status", { "const": "closed" }] }, { "present": "closedAt" }]
+ }
+ },
+ "additionalProperties": false
+}
+```
+
+`depositCoversOrder` reads `(price + fee) * quantity <= deposit`. `feeWaivedOrAtLeastTen` reads `fee == 0 || fee >= 10`; an order that leaves `fee` out passes, since a missing integer reads as 0. `closedNeedsClosedAt` says an order whose status is `closed` carries a `closedAt`.
+
+## How it works
+
+- **Create and replace.** The rules run after the JSON schema validation of the document's properties (and after [maxBytes](max-bytes.md)), so every value a rule reads has passed its property's schema. A replace is judged on the whole new document, not only on what changed.
+- **Name order, first failure.** Rules are checked in the order of their names, and the first rule the document breaks refuses the transition with `DocumentPropertyConstraintViolatedError` (10422). The error names the document type, the rule, and why it failed (below).
+- **Transfer and purchase.** These change only the owner and the transfer's time and heights. Rules that read `$ownerId`, `$transferredAt…` or a total that depends on the owner are judged again, against the stored document with its new owner and transfer values; other rules are not, since nothing they read changed. A transfer or purchase that would break such a rule is refused with 10422.
+- **Price updates** change only the update's time and heights, so the rules that read `$updatedAt…` are judged again the same way; other rules are not.
+- **Deletes** are not judged, with one exception: a delete of an [index-only](index-only.md) document carries the row's values, which are validated like a create's, rules included. The delete carries neither the owner nor any time or height, which is why an index-only type may not have a rule reading `$ownerId` or a system time or height.
+- **State and fees.** A rule reads the document, its owner and its times and heights, and a `countOf` or `sumOf` reads a total from state. Each such total is a state read billed with the write; nothing else a rule does adds a fee, and it changes nothing stored. The limits below bound its cost. SDKs that validate a document before sending it apply the same rules, except those reading a total, which they cannot read.
+
+Why a rule fails, as the error reports it:
+
+| Reason | When |
+|---|---|
+| does not hold | The rule evaluates without a fault and comes out false |
+| overflow | A value it reads, or a result it computes on the way, does not fit a 128-bit signed integer |
+| division by zero | A `divide` or `modulo` whose divisor evaluates to 0 |
+| negative exponent | A `power` whose exponent evaluates to a negative number |
+| not an integer | A value it reads for an integer property is a float with no fractional part, such as `5.0`, which the schema's `integer` type admits but an integer property cannot store |
+
+## Conditions
+
+A rule is a condition: a JSON object with exactly one key.
+
+| Condition | Form | Holds when |
+|---|---|---|
+| `equal`, `notEqual` | `[left, right]` | The two sides are equal, or differ. The sides are two integer expressions, or a string property and a string constant or another string property, or an identifier property and an identifier constant, another identifier property or `$ownerId` |
+| `lessThan`, `lessThanOrEqual`, `greaterThan`, `greaterThanOrEqual` | `[left, right]` | The left integer expression compares with the right one this way. Integers only |
+| `in` | `[expression, [v1, v2, ...]]` | The expression takes one of the listed values: two or more, no two alike, all integers or all strings. With strings, the expression is a string property, or an identifier property or `$ownerId` with the strings as base58 identifiers |
+| `notIn` | `[expression, [v1, v2, ...]]` | The expression takes none of the listed values: an `in` negated, listed the same way, in as many nodes. A string or identifier property the document leaves out takes none |
+| `startsWith`, `endsWith` | `[text, affix]` | The first string starts, or ends, with the second, byte for byte with no case folding. Each side is a string constant, a string property or an `ifAbsent` string default, at least one a property and never the same one twice. A string property left out without a default takes no string, and the condition does not hold for it |
+| `contains` | `["path", value]` | The typed array property at the path holds an element equal to the value: an integer expression among integers; a string constant, a string property or an `ifAbsent` string default among strings; an identifier constant, an identifier property or `$ownerId` among identifiers. An array the document leaves out holds nothing, and a string or identifier property it leaves out is among no elements |
+| `present` | `"path"` | The document holds the property, with a value other than null and, for an object, with at least one member present |
+| `absent` | `"path"` | The document leaves the property out, sets it to null, or gives an object no member that is present |
+| `anyOf` | `[c1, c2, ...]` | At least one of two or more conditions holds |
+| `allOf` | `[c1, c2, ...]` | Every one of two or more conditions holds |
+| `not` | `condition` | Its one condition does not hold |
+| `ifThen` | `[if, then]` | If the first condition holds, the second must. The second is evaluated only when the first holds, and a fault in either breaks the rule. The two may not be alike |
+| `ifThenElse` | `[if, then, else]` | If the first condition holds, the second must; if not, the third must. Only the branch the first selects is evaluated. No two of the three may be alike |
+
+Conditions nest: `{ "not": { "allOf": [{ "equal": ["price", 0] }, { "greaterThan": ["quantity", 10] }] } }` refuses a free order of more than 10. An `anyOf` or `allOf` may not list the same condition twice, nor hold one of its own kind directly (it says what one flat list says), and a `not` may not hold a `not` or a `notIn` directly.
+
+An `in` says what an `anyOf` of `equal` comparisons says, in far fewer nodes: `{ "in": ["fee", [0, 10, 25, 50]] }` is 6 nodes where the `anyOf` is 13.
+
+`startsWith` and `endsWith` test a string's ends: `{ "startsWith": ["url", { "const": "https://" }] }` holds a link to https, `{ "endsWith": ["url", { "const": ".dash" }] }` to a domain, and `{ "startsWith": ["path", "parentPath"] }` holds a reply's path under its parent's. A constant tested against a property that declares an `enum` must start or end one of its values.
+
+A `contains` looks the other way round, for one value among an array's elements:
+
+- `{ "not": { "contains": ["labels", { "const": "used" }] } }` refuses a `"used"` label;
+- `{ "contains": ["participants", "$ownerId"] }` holds the owner to the participants, and since it reads `$ownerId`, a transfer or purchase to someone else is refused;
+- `{ "contains": ["tiers", "quantity"] }` holds the quantity to one of the tiers the document lists.
+
+The kind of the array's elements decides what the value is: a `{ "const": "sale" }` is a string among strings and a base58 identifier among identifiers.
+
+## Expressions
+
+An integer expression is one of:
+
+| Expression | Form | Value |
+|---|---|---|
+| integer | `100` | Itself. A number written `100.0` reads as 100 |
+| path | `"price"`, `"meta.total"` | The value of an integer or boolean property of the document type, 1 for true and 0 for false. A property the document leaves out, or sets to null, reads as 0 |
+| `ifAbsent` | `{ "ifAbsent": ["quantity", 1] }` | The property's value, or the given integer when the document leaves it out |
+| `add`, `multiply` | `{ "add": [a, b, ...] }` | The sum or product of two or more operands |
+| `subtract` | `{ "subtract": [a, b] }` | `a - b` |
+| `divide` | `{ "divide": [a, b] }` | The Euclidean quotient of `a` by `b` |
+| `modulo` | `{ "modulo": [a, b] }` | The Euclidean remainder of `a` by `b`, never negative |
+| `power` | `{ "power": [a, b] }` | `a` to the power `b` |
+| `min`, `max` | `{ "max": [a, b, ...] }` | The least or greatest of two or more operands, every one evaluated |
+| `abs` | `{ "abs": a }` | The absolute value of its one operand |
+| `length`, `byteLength` | `{ "length": "title" }` | The characters (as `maxLength` counts them) or UTF-8 bytes (as `maxBytes` counts them) of a string property, 0 when the document leaves it out |
+| `count` | `{ "count": "tags" }` | The items of an array property, or the bytes of a byte array property, 0 when the document leaves it out |
+| system time or height | `"$createdAt"`, `"$updatedAtBlockHeight"` | A time or height the document records (see [Times and heights](#times-and-heights)) |
+| `countOf`, `sumOf` | `{ "countOf": ["listing", { "$ownerId": "$ownerId" }] }` | A total of documents of a type of the same contract, read from state (see [Totals of other documents](#totals-of-other-documents)) |
+
+Where `maxLength`, `maxBytes` and `maxItems` bound one property by a fixed number, a size can be compared with another property or bounded only under a condition: `{ "lessThanOrEqual": [{ "count": "tags" }, "maxTags"] }` holds a list to its own limit. A size never breaks a rule by itself: a property left out or null has size 0, and so would a value of another type, which the schema validation refuses first.
+
+Two more forms appear only in string and identifier comparisons, never inside arithmetic:
+
+| Form | Meaning |
+|---|---|
+| `{ "const": "closed" }` | A string constant, or, compared with an identifier property or `$ownerId`, a base58 identifier |
+| `{ "ifAbsent": ["status", "open"] }` | A string property, read as the given string when the document leaves it out |
+
+A bare JSON string is always a path and a bare JSON number always a value, so a constant string needs `{ "const": ... }`. The values an `in` lists are literals and need no wrapper. A path is a property name, or names joined by dots for a nested property (`"rewardSplit.leader"`); the only `$` names a rule accepts are `$ownerId` and the times and heights below.
+
+A `number` property (a float) cannot be read by a rule, which keeps every result exact.
+
+## Strings
+
+A string property is compared for equality only, never ordered and never used in arithmetic:
+
+- `{ "equal": ["status", { "const": "closed" }] }` or `notEqual`, with the constant on either side;
+- `{ "notEqual": ["fromCurrency", "toCurrency"] }`, two bare paths that both name string properties, which compares their strings;
+- `{ "in": ["status", ["open", "pending"]] }`, whose values are two or more distinct strings.
+
+A string property the document leaves out equals no constant and no other string property, not even one also left out. So `notEqual` holds for it, and `equal` and `in` do not. `{ "ifAbsent": ["status", "open"] }` gives it a default instead: it may stand wherever the bare path stands, and the property then reads as that string when it is left out. `present` and `absent` test it directly.
+
+When the property declares an `enum`, every constant compared with it, and every `ifAbsent` default given to it, must be one of the enum's values. A misspelled constant is refused at registration instead of making the rule quietly never hold.
+
+## Identifiers and `$ownerId`
+
+An identifier property compares in the same three ways: `{ "equal": ["paymentToken", { "const": "" }] }` or `notEqual`, `{ "notEqual": ["buyerId", "sellerId"] }`, and `{ "in": ["paymentToken", ["", ""]] }`. Constants are base58 identifiers of 32 bytes, checked at registration and compared by their bytes, whatever form the document gives the identifier in. An identifier property the document leaves out equals no identifier, not even another one left out. Identifiers take no `ifAbsent` default and are never ordered.
+
+`$ownerId`, the document's owner, is an identifier operand too:
+
+- `{ "equal": ["authorId", "$ownerId"] }` holds the `authorId` property to the owner.
+- `{ "in": ["$ownerId", ["", ""]] }` lets only the listed identities own a document of the type.
+
+It is not a property: `present`, `absent` and integer expressions refuse it, and comparing it with itself is refused. On create and replace it is the writer. A transfer or purchase is judged with the new owner, as described in [How it works](#how-it-works). An [index-only](index-only.md) type may not declare a rule that reads it.
+
+## Times and heights
+
+A rule can read when the document was created, last updated and last transferred, as an integer:
+
+| | block time (ms) | Platform block height | Core block height |
+|---|---|---|---|
+| creation | `$createdAt` | `$createdAtBlockHeight` | `$createdAtCoreBlockHeight` |
+| last update: a create, a replace or a price update | `$updatedAt` | `$updatedAtBlockHeight` | `$updatedAtCoreBlockHeight` |
+| last transfer: a create, a transfer or a purchase | `$transferredAt` | `$transferredAtBlockHeight` | `$transferredAtCoreBlockHeight` |
+
+- `{ "lessThanOrEqual": [{ "subtract": ["endsAt", "$createdAt"] }, 604800000] }` keeps a listing to a week from its creation.
+- `{ "lessThanOrEqual": ["$updatedAt", "endsAt"] }` refuses a replace or a price update after the listing ends.
+- `{ "lessThanOrEqual": ["$transferredAt", "endsAt"] }` refuses a transfer or a purchase after it ends.
+
+A rule may read one only when the document type records it by listing it in `required`, so every stored document holds it. None takes an `ifAbsent` default, `present` and `absent` refuse them, and an [index-only](index-only.md) type reads none. Each write is judged with the values the stored document ends up with: a create with its block's time and heights for all three events; a replace with the stored creation and transfer values and its block's as the update; a price update with its block's as the update; a transfer or a purchase with its block's as the transfer.
+
+SDK pre-checks run before the block exists: they use the device clock for the times a write records, and do not judge a rule reading a block height, which is unknown until the block.
+
+## Totals of other documents
+
+`countOf` and `sumOf` read a total from state: how many documents of a type of the same contract match a filter, or what one of their integer properties adds up to. The total is the one a count or sum tree keeps ([Count Trees](../drive/document-count-trees.md), [Sum Trees](../drive/document-sum-trees.md)), so reading it costs about the same however many documents match.
+
+| Form | Value | The counted type needs |
+|---|---|---|
+| `{ "countOf": ["listing"] }` | How many `listing` documents there are | `documentsCountable` |
+| `{ "countOf": ["listing", { "$ownerId": "$ownerId" }] }` | How many of them match the filter | A countable index whose properties are exactly the filter's keys |
+| `{ "sumOf": ["pledge", "amount"] }` | The total `amount` over every `pledge` | `documentsSummable: "amount"` |
+| `{ "sumOf": ["pledge", "amount", { "campaignId": "campaignRef" }] }` | The total over those matching the filter | An index with `summable: "amount"` whose properties are exactly the filter's keys |
+
+A filter maps each key, a property of the counted type or `$ownerId`, to the value it must take, read from the document being written: one of its properties (`"campaignRef"`), `$ownerId`, an integer, or a `{ "const": ... }` string or base58 identifier. The counted type may be the rule's own.
+
+```json
+"propertyConstraints": {
+ "atMostTenListings": {
+ "lessThanOrEqual": [{ "countOf": ["listing", { "$ownerId": "$ownerId" }] }, 10]
+ },
+ "pledgesWithinGoal": {
+ "lessThanOrEqual": [{ "sumOf": ["pledge", "amount", { "campaignId": "campaignRef" }] }, "goal"]
+ }
+}
+```
+
+- **As it will be after the write.** The total is the stored one with the write applied. When the counted type is the rule's own, a create adds the document, a replace swaps its stored version for the new one, and a transfer or purchase moves it to its new owner. So `atMostTenListings`, declared on `listing`, keeps every owner at ten or fewer, and a replace of one of ten is allowed.
+- **Judged when the rule's own type is written.** A rule is never judged on writes of the type it counts. On its own type it holds for good, since every write that could raise the total is judged; a type with a contested index cannot total its own documents, since a document a contest awards is stored without any rule judged. On another type it is only checked when its own type is written, and can go stale later: deleting a `profile` does not undo a `post` that needed one. Deletes are not judged, so a lower bound can be broken by deleting documents.
+- **Transfers, purchases and price updates.** A total that depends on the owner (a filter value of `$ownerId`, or a `$ownerId` key on the rule's own type) is read again for a transfer or purchase, the document counted toward its new owner. A rule a price update judges, one reading `$updatedAt…`, reads its totals too.
+- **Billed.** Each total is a state read billed with the write. A total two rules read alike is read once.
+- **Every earlier write counts.** A document batch carries one transition, and each state transition of a block is applied before the next is validated, so a total includes every write before it.
+- **SDK pre-checks** cannot read state, so they do not judge a rule reading a total.
+
+## Evaluation order and short-circuiting
+
+Conditions are checked in declared order and no further than the outcome needs. A comparison evaluates its left side, then its right. `anyOf` stops at the first condition that holds, `allOf` at the first that fails. Operands are evaluated left to right.
+
+A fault (an overflow, a division by zero, a negative exponent, a value that is not an integer) in a condition that is evaluated breaks the rule, whatever the other conditions would say, and `not` does not turn a fault into a pass. String comparisons, `present` and `absent` never fault. So an earlier condition can guard a later one:
+
+```json
+{ "anyOf": [{ "equal": ["b", 0] }, { "equal": [{ "divide": ["a", "b"] }, 2] }] }
+```
+
+holds for a `b` of 0 without dividing by it. The same two conditions the other way round divide by zero and break the rule.
+
+## Arithmetic
+
+- Integers are exact over 128-bit signed integers. Every intermediate result must fit, and one that does not breaks the rule instead of wrapping. `add` and `multiply` fold their operands from the left, so an overflow on the way is a fault even when a later operand would bring the total back in range.
+- `divide` and `modulo` are Euclidean: the remainder is never negative, and the quotient is the one that goes with it. `-7` divided by `2` is `-4`, remainder `1`. For operands that are not negative this is ordinary integer division.
+- A divisor that evaluates to 0 breaks the rule. A negative exponent breaks it too, since it has no integer result. `0` to the power `0` is `1`.
+- There are no floats.
+
+## Rules at registration
+
+The meta-schema checks the shape (`JsonSchemaError`, 10101):
+
+- the keyword is an object of one or more rules, named with 1 to 64 letters, digits or underscores;
+- every condition and every operator object has exactly one key;
+- a comparison, `subtract`, `divide`, `modulo` and `power` take exactly two operands; `add` and `multiply` two or more; `anyOf` and `allOf` two or more conditions, no two alike; an `in` two or more distinct values, all integers or all strings;
+- no `anyOf` or `allOf` holds its own kind directly, and no `not` holds a `not` or a `notIn`;
+- a path matches `$ownerId`, one of the nine [times and heights](#times-and-heights), or dotted names of 1 to 64 letters, digits or underscores, so `$revision` and other system properties are refused;
+- a `countOf` lists a type name and optionally a filter, and a `sumOf` a type name, a property and optionally a filter; a filter has one or more keys, each `$ownerId` or a dotted path, and each value is a path, `$ownerId`, an integer or a `{ "const": ... }` string.
+
+The parser then checks the rules against the document type (`InvalidContractStructure`, 10231):
+
+- every path an integer expression reads names an integer or boolean property; every path `length` or `byteLength` measures names a string property, and every path `count` counts an array or byte array property; every path a `contains` looks in names a typed array property whose elements are integers, strings or identifiers, of the kind of the value looked for (a string constant among them in the elements' `enum` when they declare one); every path compared with a string, or tested by `startsWith` or `endsWith`, names a string property, and a constant tested against one with an `enum` starts or ends one of its values; every path compared with an identifier names an identifier property; every path `present` or `absent` tests names a property of any type, an object included;
+- no rule reads a property that is `transient` or inside a transient object, since a stored document could never be held to it;
+- every comparison and `in` reads at least one property: a comparison of constants would hold for every document or for none;
+- strings and identifiers are compared only with `equal`, `notEqual` and `in`; a string is never compared with an identifier; a property is never compared with itself;
+- string constants and `ifAbsent` defaults are in the property's `enum` when it has one; identifier constants are base58 identifiers of 32 bytes;
+- no literal divisor is 0 and no literal exponent is negative;
+- every time or height a rule reads is one the type lists in `required`, and takes no `ifAbsent` default;
+- `present` and `absent` do not name `$ownerId` or a time or height, and an index-only type has no rule reading any of them;
+- no `anyOf` or `allOf` lists two conditions that parse alike, such as `1` and `1.0`, or two `in` conditions listing the same values in another order, and no `ifThen` or `ifThenElse` holds two alike conditions;
+- no condition or operand nests more than 64 levels deep;
+- once every document type of the contract is parsed, every `countOf` and `sumOf` counts a type of the contract that is not index-only, and not its own type when that has a contested index, with a tree that keeps the total as set out in [Totals of other documents](#totals-of-other-documents). A unique, contested, ranked, time-range or index-only-terminal index keeps no such total, nor does one with more properties than the filter has keys;
+- every key of a filter is `$ownerId` or an integer, string or identifier property of the counted type, and its value is of the same kind; a string constant is in the key's `enum` when it has one, and an identifier constant is base58;
+- every property a filter value reads is listed in `required`, with every object around it, so a write always has the value; an index-only type has no rule reading a total.
+
+Three limits come from the protocol version 14 `SystemLimits`, and a rule over one is refused the same way:
+
+- at most 16 rules per document type (`max_property_constraints`);
+- at most 32 nodes per rule (`max_property_constraint_nodes`);
+- at most 4 distinct `countOf` and `sumOf` totals read by one document type's rules, a total read twice counting once (`max_property_constraint_aggregates`).
+
+A rule within 32 nodes is never deep enough to reach the 64-level bound. Nodes are counted like this:
+
+| Part of a rule | Nodes |
+|---|---|
+| A comparison of integers | 1, plus its two sides |
+| An `equal` or `notEqual` of strings or identifiers, a `startsWith` or an `endsWith` | 3: the condition and its two sides |
+| An `in` over integers | 1, plus its expression, plus 1 per value |
+| An `in` over strings or identifiers | 2, plus 1 per value |
+| `contains` | 2, plus the value it looks for |
+| `present`, `absent` | 1 |
+| `anyOf`, `allOf` | 1, plus their conditions |
+| `not` | 1, plus its condition |
+| `ifThen`, `ifThenElse` | 1, plus their conditions |
+| `notIn` | as the `in` it negates |
+| An integer, a path, an `ifAbsent`, a size (`length`, `byteLength`, `count`) or a time or height | 1 |
+| `add`, `multiply`, `subtract`, `divide`, `modulo`, `power`, `min`, `max`, `abs` | 1, plus their operands |
+| `countOf`, `sumOf` | 1, plus 1 per filter key |
+
+`depositCoversOrder` above is 7 nodes (the comparison, `multiply`, `add` and four paths), and `closedNeedsClosedAt` is 5. An `in` fits up to 30 values in 32 nodes.
+
+## Worked examples
+
+**Percentages that add up.** The moderation charters system contract requires a proposal's reward split to be whole. The paths name members of the `rewardSplit` object, each an integer from 0 to 100:
+
+```json
+"propertyConstraints": {
+ "rewardSplitIsWhole": {
+ "equal": [
+ { "add": ["rewardSplit.leader", "rewardSplit.equal", "rewardSplit.actions"] },
+ 100
+ ]
+ }
+}
+```
+
+Six nodes: the comparison, `add`, three paths and `100`.
+
+**One of two, or both or neither.** A contact card must give an email or a phone; a shipping block gives a street and a city together or not at all:
+
+```json
+"propertyConstraints": {
+ "reachable": { "anyOf": [{ "present": "email" }, { "present": "phone" }] },
+ "addressComplete": {
+ "anyOf": [
+ { "allOf": [{ "present": "street" }, { "present": "city" }] },
+ { "allOf": [{ "absent": "street" }, { "absent": "city" }] }
+ ]
+ }
+}
+```
+
+`present` and `absent` work on properties of any type, strings and objects included. On an integer they are also the only way to tell "not given" from "given as 0", since a missing integer reads as 0 in an expression. An object with no member present, `{}` or `{ "inner": {} }`, counts as absent: a stored document does not keep it, and a transfer, purchase or price update is judged on the stored document, so a create or replace is judged the same way.
+
+**A time window.** An event ends after it starts, and lasts at most a week (`startsAt` and `endsAt` are required integer timestamps in milliseconds):
+
+```json
+"propertyConstraints": {
+ "endsAfterStart": { "lessThan": ["startsAt", "endsAt"] },
+ "atMostAWeek": { "lessThanOrEqual": [{ "subtract": ["endsAt", "startsAt"] }, 604800000] }
+}
+```
+
+**A status workflow.** On a `ticket` type, `status` is optional and means `open` when it is left out. A closed ticket names who closed it; an open or pending one does not:
+
+```json
+"propertyConstraints": {
+ "closedNamesCloser": {
+ "anyOf": [{ "notEqual": ["status", { "const": "closed" }] }, { "present": "closedBy" }]
+ },
+ "activeHasNoCloser": {
+ "anyOf": [
+ { "not": { "in": [{ "ifAbsent": ["status", "open"] }, ["open", "pending"]] } },
+ { "absent": "closedBy" }
+ ]
+ }
+}
+```
+
+Without the `ifAbsent`, a ticket with no status would equal none of the listed strings, the `in` would not hold, and `activeHasNoCloser` would let it carry a `closedBy`. If `status` declares an `enum`, `"closed"`, `"open"` and `"pending"` must all be in it.
+
+**Who may own a badge.** On a transferable `badge` type, only two identities may ever hold one:
+
+```json
+"propertyConstraints": {
+ "knownHolder": {
+ "in": [
+ "$ownerId",
+ ["HJtU46rVEkKJgevQhiVt2YhdHtDS3xtGzzJqit8en5mb", "GfRNCXeyuB3th33a6nkJKQJecKMdRzuBiPsgvSCyUTvK"]
+ ]
+ }
+}
+```
+
+A create by anyone else is refused, and so is a transfer or sale of a badge to anyone else: the rule reads `$ownerId`, so it is judged again with the new owner.
+
+**A guarded division.** The average unit price of a batch is at most 100, and a batch may be empty:
+
+```json
+"propertyConstraints": {
+ "unitPriceCapped": {
+ "anyOf": [
+ { "equal": ["quantity", 0] },
+ { "lessThanOrEqual": [{ "divide": ["total", "quantity"] }, 100] }
+ ]
+ }
+}
+```
+
+The `equal` comes first, so an empty batch never reaches the division. Written the other way round, an empty batch breaks the rule with a division by zero.
+
+**A flag in arithmetic.** A boolean reads as 1 or 0, so a waived fee must be 0:
+
+```json
+"propertyConstraints": {
+ "waivedMeansFree": { "equal": [{ "multiply": ["waiveFee", "fee"] }, 0] }
+}
+```
+
+## See also
+
+- [Property Constraints](../data-model/documents.md#property-constraints-propertyconstraints), the deep dive
+- [distinctFrom](distinct-from.md), a single-keyword way to keep two identifiers apart
+- [Property Schemas](property-schemas.md), for the one-property bounds JSON Schema gives
+- [transient](transient.md), [Index-Only Types](index-only.md)
+- [Contract Keywords overview](../contract-keywords.md), for the limits and the conventions of these tables
diff --git a/book/src/contract-keywords/property-schemas.md b/book/src/contract-keywords/property-schemas.md
new file mode 100644
index 00000000000..fb6a57ff49c
--- /dev/null
+++ b/book/src/contract-keywords/property-schemas.md
@@ -0,0 +1,300 @@
+# Property Schemas
+
+Each entry of a document type's `properties` is a property schema: JSON Schema (draft 2020-12), limited to the keywords in this chapter, plus three Platform keywords that say how a value is stored (`position`, `byteArray` and `contentMediaType`). The schema is checked when the contract is registered, and every created or replaced document is validated against it. Platform keywords with more to them, such as [`maxBytes`](max-bytes.md), [`refersTo`](refers-to.md), [`distinctFrom`](distinct-from.md), [`encryptedFor`](encrypted-for.md), [`generatedFrom`](generated-from.md), [`requiredSince`](required-since.md) and a typed array's [`items`](typed-arrays.md), have chapters of their own.
+
+| Keyword | Applies to | On update |
+|---|---|---|
+| [`type`](#type) | every property | Fixed |
+| [`position`](#position) | every property | Fixed |
+| [`minLength`, `maxLength`, `pattern`, `format`](#strings) | strings | Loosened or removed only |
+| [`minimum`, `maximum`, `exclusiveMinimum`, `exclusiveMaximum`, `multipleOf`](#numbers) | integers and numbers | Loosened or removed only, keeping an integer's width; `multipleOf` fixed |
+| [`enum`, `const`](#enum-and-const) | any | `enum` may gain values; `const` may be removed |
+| [`byteArray`, `contentMediaType`](#byte-arrays-and-identifiers) | byte arrays | Fixed |
+| [`minItems`, `maxItems`, `uniqueItems`, `contains`](#arrays) | arrays | Loosened or removed only; `contains` fixed |
+| [`properties`, `required`, `additionalProperties`, `minProperties`, `maxProperties`, `dependentRequired`](#objects) | objects | Members may be added; the rest fixed, except `dependentRequired` may lose entries |
+| [`$ref`](#ref) | any | Fixed |
+| [`$id`, `$comment`, `description`, `examples`](#annotations) | any | Free (`$id` may only be added) |
+
+Each section gives the error a contract update gets for breaking its rules. Most are refused with `IncompatibleDocumentTypeSchemaError` (10246), from the comparison of the old and new schemas; a change to how a value is stored is refused with `DocumentTypeUpdateError` (40212).
+
+## How property schemas are checked
+
+When a contract is registered or updated:
+
+- The schema is checked against the document meta-schema. A keyword the meta-schema does not allow where it is written, or a value of the wrong shape, is refused with `JsonSchemaError` (10101).
+- The parser reads each property's type and bounds, and refuses what the meta-schema cannot express (`InvalidContractStructure`, 10231).
+- The schema is compiled for validating documents. A `pattern` that is not a valid regular expression, or a `format` the validator does not know, is refused here (`JsonSchemaError`, 10101).
+
+When a document is created or replaced:
+
+1. Every string and byte array value is at most 5120 bytes, whatever the schema allows (`DocumentFieldMaxSizeExceededError`, 10417). From protocol version 14 a value nested more than 256 levels deep is refused as well (`ValueError`, 10103).
+2. The document's properties are validated against the schema. Each failure is reported as a `JsonSchemaError` (10101) naming the keyword and the property.
+3. Platform's own checks, which JSON Schema cannot express, come next: [`maxBytes`](max-bytes.md) and [`propertyConstraints`](property-constraints.md) among them.
+
+A transfer, a price update and a purchase carry no property values, so they are not validated against the schema again.
+
+The schema also decides how each value is stored, since a stored document holds no property names or type tags:
+
+| Property | Stored as |
+|---|---|
+| `integer` | 1, 2, 4 or 8 bytes, chosen by its bounds (see [Numbers](#numbers)) |
+| `number` | 8 bytes, a 64-bit floating point number |
+| `boolean` | 1 byte |
+| `string` | a length prefix, then the UTF-8 bytes |
+| byte array with `minItems` equal to `maxItems` | the bytes, with no prefix |
+| any other byte array | a length prefix, then the bytes |
+| identifier | 32 bytes |
+| `object` | a length prefix, then its members |
+| typed array | an element count, then the elements (see [Typed Arrays](typed-arrays.md)) |
+
+An optional property adds one byte in front that says whether it is present. See [Document Serialization](../serialization/document-serialization.md#value-encoding-by-type) for the exact encoding.
+
+## `type`
+
+| | |
+|---|---|
+| **Where** | Every property; also the elements of a typed array |
+| **Value** | One of `"string"`, `"integer"`, `"number"`, `"boolean"`, `"object"`, `"array"` |
+| **Since** | protocol version 1 |
+| **On update** | Fixed (`IncompatibleDocumentTypeSchemaError`, 10246) |
+| **Errors** | `JsonSchemaError` (10101): a document value of another type |
+
+`type` is a single name. A list of types, and `"null"`, are refused at registration: every stored value needs one known encoding.
+
+An `array` is one of two things:
+
+- a **byte array**, with `byteArray: true`: a string of bytes, such as a hash or an [identifier](#byte-arrays-and-identifiers);
+- from protocol version 14, a **typed array**, with an `items` schema: a list of values of one scalar type. See [Typed Arrays](typed-arrays.md).
+
+An array that is neither is refused.
+
+## `position`
+
+| | |
+|---|---|
+| **Where** | Every property, at every level. Not on the elements of a typed array. |
+| **Value** | An integer, 0 or more |
+| **Since** | protocol version 1 |
+| **On update** | Fixed (10246) |
+| **Errors** | `MissingPositionsInDocumentTypePropertiesError` (10411) at registration |
+
+A document is stored with its values one after another and no property names. `position` is the property's place in that sequence.
+
+- The top-level properties of a document type must use the positions 0, 1, 2 and so on, with no gap and no number used twice (10411). A top-level property without a `position` is refused.
+- The members of an object need a `position` too. Number them from 0 within the object; only the top level is checked for gaps.
+- A property that takes its schema from a [`$ref`](#ref) writes its `position` next to the `$ref`.
+- A property added by a contract update takes the next free position. An existing position can never change, or stored documents would be read in the wrong order.
+
+## Strings
+
+| | |
+|---|---|
+| **Keywords** | `minLength`, `maxLength`, `pattern`, `format` |
+| **Where** | Properties of type `string`, and the string elements of a typed array |
+| **Value** | `minLength`, `maxLength`: an integer, 0 or more, counting characters. `pattern`: a regular expression. `format`: the name of a JSON Schema format, such as `"uri"` |
+| **Since** | protocol version 1 |
+| **On update** | `maxLength` may be raised or removed, and `minLength` lowered or removed. `pattern` and `format` may be removed. None of them may be added, and no other change is allowed (10246). |
+| **Errors** | `JsonSchemaError` (10101) |
+
+```json
+"username": {
+ "type": "string",
+ "minLength": 3,
+ "maxLength": 63,
+ "pattern": "^[a-zA-Z0-9_]+$",
+ "position": 0
+}
+```
+
+A `username` is 3 to 63 letters, digits or underscores.
+
+- `minLength` and `maxLength` count characters, and a character takes 1 to 4 bytes in UTF-8. To cap the stored size, add [`maxBytes`](max-bytes.md). Whatever `maxLength` says, no single string may exceed 5120 bytes (10417).
+- `pattern` is written in the syntax of Rust's `regex` crate, which has no lookaround and no backreferences. A pattern that does not compile is refused at registration (10101).
+- `format` is checked on every document. The validator knows `date-time`, `date`, `time`, `email`, `idn-email`, `hostname`, `ipv4`, `ipv6`, `uri` and `regex`. Any other format, `uuid` and `uri-reference` included, is refused at registration (10101).
+- A string with `pattern` or `format` must declare a `maxLength` of at most 50000, so that matching stays cheap.
+- A string used in an index needs a `maxLength` of at most 63 (`InvalidIndexedPropertyConstraintError`, 10205). See [Indexes](indexes.md).
+
+## Numbers
+
+| | |
+|---|---|
+| **Keywords** | `minimum`, `maximum`, `exclusiveMinimum`, `exclusiveMaximum`, `multipleOf` |
+| **Where** | Properties of type `integer` or `number`, and such elements of a typed array |
+| **Value** | A number. On an integer element of a typed array, `minimum` and `maximum` are integers. |
+| **Since** | protocol version 1 |
+| **On update** | `maximum` and `exclusiveMaximum` may be raised or removed, and `minimum` and `exclusiveMinimum` lowered or removed, unless that changes how an integer is stored (`DocumentTypeUpdateError`, 40212). None may be added, and `multipleOf` is fixed (10246). |
+| **Errors** | `JsonSchemaError` (10101) |
+
+```json
+"rating": { "type": "integer", "minimum": 1, "maximum": 5, "position": 2 }
+```
+
+A `rating` is a whole number from 1 to 5, and is stored in one byte.
+
+A `number` is always stored in 8 bytes. An `integer` is stored in the smallest width its `minimum` and `maximum` allow, when the contract's config has `sizedIntegerTypes` on (the default; see [Contract-Level Keys and config](contract-config.md)):
+
+| `minimum` | `maximum` | Stored as |
+|---|---|---|
+| 0 or more | up to 255 | 1 byte, unsigned |
+| 0 or more | up to 65535 | 2 bytes, unsigned |
+| 0 or more | up to 4294967295 | 4 bytes, unsigned |
+| 0 or more | higher | 8 bytes, unsigned |
+| below 0 | both bounds within -128 to 127 | 1 byte, signed |
+| below 0 | both bounds within -32768 to 32767 | 2 bytes, signed |
+| below 0 | both bounds within -2147483648 to 2147483647 | 4 bytes, signed |
+| below 0 | otherwise | 8 bytes, signed |
+
+With only a `minimum`, the integer takes 8 bytes, unsigned when the minimum is 0 or more. With only a `maximum`, it takes the unsigned width the maximum gives, so an integer that may be negative needs a `minimum` too. With neither, an `enum` of integers picks the width from its smallest and largest members; without one, the integer takes 8 bytes, signed. `exclusiveMinimum` and `exclusiveMaximum` do not affect the width. With `sizedIntegerTypes` off, every integer takes 8 bytes, signed.
+
+Stored documents and index entries hold each integer at its width, so a contract update may not change the width or the sign. Raising `maximum` past the width, lowering `minimum` below 0, removing a bound, or adding an `enum` value outside the width is refused with `DocumentTypeUpdateError` (40212); so is turning `sizedIntegerTypes` on when it would change an existing integer's width. A change that keeps the width is accepted.
+
+## `enum` and `const`
+
+| | |
+|---|---|
+| **Where** | Any property. `enum` also on the string, integer, number and boolean elements of a typed array; `const` never on elements. |
+| **Value** | `enum`: an array of one or more values, none repeated. `const`: one value. |
+| **Since** | protocol version 1 |
+| **On update** | `enum` may gain values but not lose one; the keyword may be removed, not added. `const` may be removed, not added or changed (10246). An `enum` value that changes an integer's width is refused (`DocumentTypeUpdateError`, 40212). |
+| **Errors** | `JsonSchemaError` (10101) |
+
+```json
+"status": { "type": "string", "enum": ["open", "closed", "archived"], "position": 3 }
+```
+
+`enum` lists the values a property may take and `const` the one value it must take. Both only narrow what documents may hold, so an update may widen them (more `enum` values, or no keyword at all) but never narrow them, which could leave stored documents invalid. On an integer without bounds, `enum` also sets the stored width (see [Numbers](#numbers)).
+
+## Byte arrays and identifiers
+
+| | |
+|---|---|
+| **Keywords** | `byteArray`, `contentMediaType` |
+| **Where** | Properties of type `array`, and the array elements of a typed array |
+| **Value** | `byteArray`: `true`, the only value. `contentMediaType`: `"application/x.dash.dpp.identifier"` makes the byte array an identifier. |
+| **Since** | protocol version 1 |
+| **On update** | Fixed (10246). The length bounds follow the rules of [Arrays](#arrays), but a byte array may not switch between a fixed and a variable length, or change its fixed length (`DocumentTypeUpdateError`, 40212). |
+| **Errors** | `JsonSchemaError` (10101) |
+
+```json
+"authorId": {
+ "type": "array",
+ "byteArray": true,
+ "minItems": 32,
+ "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "position": 1
+}
+```
+
+`authorId` is an identifier: the 32-byte id of an identity, a document, a contract or a token.
+
+- `byteArray: true` makes an array a string of bytes. Its `minItems` and `maxItems` count bytes. When the two are equal the bytes are stored as they are; otherwise they carry a length prefix.
+- `contentMediaType: "application/x.dash.dpp.identifier"` makes a byte array an identifier. It must come with `byteArray: true`, `minItems: 32` and `maxItems: 32`, and it may not carry `uniqueItems`. An identifier is shown in base58, and it is the kind of property [`distinctFrom`](distinct-from.md) and most [`refersTo`](refers-to.md) targets are declared on.
+- A byte array used in an index needs a `maxItems` of at most 255 (`InvalidIndexedPropertyConstraintError`, 10205).
+- On a typed array, `contentMediaType` belongs on the `items`, not on the array.
+
+## Arrays
+
+| | |
+|---|---|
+| **Keywords** | `minItems`, `maxItems`, `uniqueItems`, `contains` |
+| **Where** | Properties of type `array`. `minItems` and `maxItems` also on byte array elements of a typed array. |
+| **Value** | `minItems`, `maxItems`: an integer, 0 or more. `uniqueItems`: a boolean. `contains`: a schema. |
+| **Since** | protocol version 1 |
+| **On update** | `maxItems` may be raised or removed and `minItems` lowered or removed (10246), within the byte array rule above (40212); a typed array keeps its `maxItems`. `uniqueItems` may be removed or set to `false`, not added (10246). `contains` is fixed (10246). |
+| **Errors** | `JsonSchemaError` (10101) |
+
+- `minItems` and `maxItems` count bytes on a byte array and elements on a typed array. A typed array must declare `maxItems`, at most 1024.
+- `uniqueItems: true` on a typed array refuses a document that repeats an element. On a plain byte array it refuses a repeated byte. It is refused on an identifier, and on the elements of a typed array.
+- `contains` is the JSON Schema keyword: at least one element must match the schema it holds.
+
+## Objects
+
+| | |
+|---|---|
+| **Keywords** | `properties`, `required`, `additionalProperties`, `minProperties`, `maxProperties`, `dependentRequired` |
+| **Where** | Properties of type `object` |
+| **Value** | The same as at the top of a document type: see [Document Shape](document-shape.md) |
+| **Since** | protocol version 1 |
+| **On update** | Members may be added, never removed; `required` and `additionalProperties` are fixed; `dependentRequired` may lose entries, not gain them (10246). `minProperties` and `maxProperties` are fixed (10246). |
+| **Errors** | `JsonSchemaError` (10101) |
+
+```json
+"records": {
+ "type": "object",
+ "properties": {
+ "identity": {
+ "type": "array",
+ "byteArray": true,
+ "minItems": 32,
+ "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "position": 0
+ }
+ },
+ "minProperties": 1,
+ "additionalProperties": false,
+ "position": 5
+}
+```
+
+An object groups members under one property, as DPNS groups a name's records.
+
+- An object declares `properties`, 1 to 100 members each with a `position`, and `additionalProperties: false`, unless it takes its schema from a [`$ref`](#ref).
+- Its `required` names the members every value of the object must hold. It cannot change after the type exists, so a member added by an update is optional.
+- An object is stored with its members inline. An object cannot be indexed, but a member can: an index names it by its dotted path, such as `records.identity`.
+
+## `$ref`
+
+| | |
+|---|---|
+| **Where** | Any property, except the elements of a typed array |
+| **Value** | `"#/$defs/"`: a definition in the contract's `schemaDefs` |
+| **Since** | protocol version 1 |
+| **On update** | Fixed (10246) |
+| **Errors** | `InvalidJsonSchemaRefError` (10207) at registration |
+
+`$ref` lets several document types share one schema, written once in the contract's `schemaDefs`:
+
+```json
+"schemaDefs": {
+ "address": {
+ "type": "object",
+ "properties": {
+ "street": { "type": "string", "maxLength": 100, "position": 0 },
+ "city": { "type": "string", "maxLength": 50, "position": 1 }
+ },
+ "required": ["street", "city"],
+ "additionalProperties": false
+ }
+}
+```
+
+A property of any document type of the contract then reads `"shippingAddress": { "$ref": "#/$defs/address", "position": 2 }`.
+
+- Only local references, starting with `#`, are allowed. The platform places `schemaDefs` under `$defs` in every document type (see [`$schema` and `$defs`](document-shape.md#schema-and-defs)).
+- A reference that does not resolve, or that leads back to itself, is refused (10207).
+- The property keeps its own `position`; the definition supplies everything else.
+- The `$ref` itself cannot change on update. The definition it points at can, under the rules of the keywords it holds (`IncompatibleDataContractSchemaError`, 10213).
+
+## Annotations
+
+| | |
+|---|---|
+| **Keywords** | `$id`, `$comment`, `description`, `examples` |
+| **Where** | Any property. `$comment` and `description` also on the elements of a typed array. |
+| **Value** | `$id`: a string starting with `#`. `$comment`, `description`: a string. `examples`: an array of values. |
+| **Since** | protocol version 1 |
+| **On update** | `$comment`, `description` and `examples`: free. `$id`: may be added, not removed or changed (10246). |
+| **Errors** | none |
+
+Notes for people and tools reading the contract. They do not change what a document may hold.
+
+## See also
+
+- [Document Shape](document-shape.md), for the keywords at the top of a document type
+- [Typed Arrays](typed-arrays.md), for arrays of values
+- [maxBytes](max-bytes.md), for capping a string's size in bytes
+- [Document Serialization](../serialization/document-serialization.md), for the stored form of every type
+- [Indexes](indexes.md), for the limits on indexed properties
+- [Error Codes](../error-handling/error-codes.md)
diff --git a/book/src/contract-keywords/ranked.md b/book/src/contract-keywords/ranked.md
new file mode 100644
index 00000000000..b79ddf392e2
--- /dev/null
+++ b/book/src/contract-keywords/ranked.md
@@ -0,0 +1,129 @@
+# Ranked Indexes
+
+A ranked index answers "which values score highest": the five restaurants with the best average grade, the ten hashtags with the most posts, the three sellers with the largest sales. The range aggregates of [Counts, Sums and Averages](aggregates.md) keep a count or a sum for every value of an index, but in the order of the values, so finding the top five means reading every value. A ranking adds a second, ordered view of the same totals, and a "top K" query reads K entries from one end of it, with a proof that grows with K rather than with the number of values. Each ranking costs one more tree that every write under the index updates, so a contract declares only the rankings it will query.
+
+The index's **groups** are the distinct values of its last property. The three keywords rank the groups by document count, by the sum of the summed property, or by its average.
+
+## Example
+
+```json
+"review": {
+ "type": "object",
+ "indices": [
+ {
+ "name": "byRestaurant",
+ "properties": [{ "restaurantId": "asc" }],
+ "averageable": "grade",
+ "rangeAverageable": true,
+ "rankedAverageable": true,
+ "rankedCountable": true
+ }
+ ],
+ "properties": {
+ "restaurantId": { "type": "string", "minLength": 1, "maxLength": 32, "position": 0 },
+ "grade": { "type": "integer", "minimum": 0, "maximum": 100, "position": 1 }
+ },
+ "required": ["restaurantId", "grade"],
+ "additionalProperties": false
+}
+```
+
+`averageable` and `rangeAverageable` keep the count and the sum of `grade` per restaurant; `rankedAverageable` orders the restaurants by average grade and `rankedCountable` by number of reviews. The query
+
+```sql
+SELECT avg(grade) FROM review GROUP BY restaurantId ORDER BY avg(grade) DESC LIMIT 3
+```
+
+returns the three best-rated restaurants with their counts and sums, proved.
+
+## `rankedCountable`
+
+| | |
+|---|---|
+| **Where** | index |
+| **Value** | boolean, or `{ "at": }` |
+| **Default** | `false` |
+| **Since** | protocol version 14 |
+| **On update** | Fixed, like every index (`DataContractInvalidIndexDefinitionUpdateError`, 10217) |
+
+Ranks groups by how many documents they hold. Needs `rangeCountable: true` on the index, or `rangeAverageable: true`, which implies it.
+
+`true` ranks the values of the index's last property. On a compound index, the ranking is kept separately for each value of the properties before it: on `[city, restaurantId]` each city has its own ranking of restaurants, and a query names the city.
+
+The object form `{ "at": ... }` places the ranking at another level of the index. Naming an earlier property ranks that property's values by the number of documents beneath them, whatever the later properties hold:
+
+```json
+{
+ "name": "byHashtagPost",
+ "properties": [{ "hashtag": "asc" }, { "postId": "asc" }],
+ "countable": "countable",
+ "rangeCountable": true,
+ "rankedCountable": { "at": ["hashtag", "postId"] }
+}
+```
+
+`"hashtag"` in `at` ranks hashtags by their total posts across all post ids; `"postId"`, the last property, is the same as `true` and ranks the posts under one hashtag. An array declares several rankings on one index; each level named costs one more ordered tree to maintain on every write beneath it. A query addresses a ranking by the property it groups by, with every property before that one fixed.
+
+`at` names only the index's own properties, each once. The object form cannot be combined with `rankedSummable` or `rankedAverageable`: a ranking at an earlier level is fed by a chain of counts that cannot also carry a sum.
+
+## `rankedSummable`
+
+| | |
+|---|---|
+| **Where** | index |
+| **Value** | boolean |
+| **Default** | `false` |
+| **Since** | protocol version 14 |
+| **On update** | Fixed (10217) |
+
+Ranks the groups of the index's last property by the sum of the index's `summable` property: the recipients who received the most, the products that sold the most units. Needs `rangeSummable: true`, or `rangeAverageable: true`.
+
+## `rankedAverageable`
+
+| | |
+|---|---|
+| **Where** | index |
+| **Value** | boolean |
+| **Default** | `false` |
+| **Since** | protocol version 14 |
+| **On update** | Fixed (10217) |
+
+Ranks the groups of the index's last property by the average of the `averageable` property. Needs both range totals: `rangeAverageable: true`, or `rangeCountable: true` with `rangeSummable: true`.
+
+The three keywords are independent. `rankedAverageable` does not imply `rankedCountable` or `rankedSummable`, unlike `averageable`, which is shorthand for a count and a sum. Declare each ranking the application will query, and no other.
+
+## Queries
+
+A ranked query names one aggregate, groups by the ranked property, orders by the aggregate and takes a limit, with an optional offset. On a compound index such as `[city, restaurantId]`:
+
+```sql
+SELECT count(*) FROM review WHERE city == "London" GROUP BY restaurantId ORDER BY count(*) DESC LIMIT 5
+```
+
+Every property before the grouped one must be fixed with an equality; at most one of them may instead be an `in` of 2 to 10 values, whose rankings are merged. There is no ranking across different values of those properties. A ranked index still answers every range query the same `range*` flags answer: ranking never changes what they return.
+
+The request shape, ties, offsets and proofs are described in [Ranked Index Examples](../drive/ranked-index-examples.md).
+
+## Rules at registration
+
+- **Its range totals.** Each ranking needs the range flags listed above, checked both by the meta-schema and by the parser. A written-out `false` needs nothing.
+- **A non-unique index.** Every group of a unique index holds at most one document, so a ranking on one is refused, and a contested index, which is unique, cannot have one either.
+- **`nullSearchable` left at `true`.** With `false`, documents missing the property would still leave an empty group in the ranking.
+- **A shorter key.** The ranked property's value becomes part of the ordered tree's key, behind an 8-byte sort key (16 bytes when `rankedAverageable` is declared), so its worst case must fit in what is left of the 255-byte limit. A ranked string takes `maxLength` of at most 61, or 59 when the index declares `rankedAverageable`; a ranked byte array takes `maxItems` of at most 247, or 239. A larger bound is refused (`InvalidIndexedPropertyConstraintError`, 10205). Identifiers, integers and other fixed-width values always fit. The bound applies to each ranked property: the last one when a ranking is declared with `true`, and each property named in `at`.
+- **Time windows.** On a [`timeRange`](time-range.md) index, a ranking must sit below the bucketed timestamp, which gives one ranking per window. A single-property time-range index cannot be ranked, and `at` cannot name the bucketed property.
+- **Other indexes of the type.** A compound ranked index `[p1, ..., pn]` is refused when another countable or summable index ends at exactly `[p1, ..., pn-1]`. A ranking at an earlier level (`at`) also restricts the other indexes that reach that level; the full table is in [Shape Restrictions](../drive/document-ranked-trees.md#shape-restrictions).
+- **One index per property list.** Two indexes with the same properties are a `DuplicateIndexError` (10201), so the rankings of one property list go on one index.
+- **Protocol version 14.** Earlier versions do not know the keywords and refuse them.
+
+A broken rule other than the key length is refused as `InvalidContractStructure` (10231), or by the meta-schema as `JsonSchemaError` (10101).
+
+## Costs
+
+Each ranking is one ordered tree, rewritten whenever a document under it is created, changed or deleted, on top of the range totals it is built from. Two rankings on one index cost two rewrites per write; a fully ranked `at` array costs one per ranked level. A ranking at the index's first property gets its tree when the contract is registered; a ranking at a deeper level gets one tree per value above it, as documents arrive.
+
+## See also
+
+- [Document Ranked Trees](../drive/document-ranked-trees.md) for the tree variants, prefix-level rankings and how the rankings are maintained.
+- [Ranked Index Examples](../drive/ranked-index-examples.md) for worked queries and proofs.
+- [Counts, Sums and Averages](aggregates.md) for the range totals a ranking builds on.
+- [Time-Range Indexes](time-range.md) for rankings per time window.
diff --git a/book/src/contract-keywords/refers-to-expressions.md b/book/src/contract-keywords/refers-to-expressions.md
new file mode 100644
index 00000000000..9250094abcf
--- /dev/null
+++ b/book/src/contract-keywords/refers-to-expressions.md
@@ -0,0 +1,130 @@
+# Expressions
+
+A [`refersTo`](refers-to.md) declaration may combine several targets instead of naming one. `anyOf` holds when at least one of its operands holds, and `allOf` when every operand holds for the same value. Reach for an expression when a value may point at one of several kinds of thing ("an elected member or an added one"), or must satisfy several references at once ("a member of the team who also has a profile"). Both combinators take the same operands, follow the same limits and are checked the same way; they differ only in when they stop.
+
+## `anyOf`
+
+| | |
+|---|---|
+| **Where** | In place of a single target: in `refersTo` on an identifier property or on the `items` of a typed array of identifiers, in `ownerRefersTo` and `creatorRefersTo`, and as an operand of an `allOf`. Not on a key id property. |
+| **Value** | `{ "anyOf": [ ... ] }`, the declaration's only key: 2 to 4 distinct [operands](#operands). |
+| **Since** | protocol version 14 |
+| **On update** | Fixed (`IncompatibleDocumentTypeSchemaError`, 10246), including a change of operand order. |
+| **Errors** | None of its own: when no operand holds, the write is refused with the last operand's error, for example `ReferencedEntityNotFoundError` (40120). |
+
+The operands are checked in the order they are listed, and the first that holds decides: the rest are not read. When none holds, the write is refused with the error of the last one.
+
+The moderation charters contract lets a member of a seated team resign with a `resignationRequest`. Its writer must be on the team, either elected or added later:
+
+```json
+"ownerRefersTo": {
+ "anyOf": [
+ {
+ "type": "listElement",
+ "documentType": "electedCharter",
+ "propertyAgreement": { "electedCharterId": "$id" },
+ "inList": "members"
+ },
+ {
+ "type": "deletableDocument",
+ "documentType": "addedModerator",
+ "lookup": {
+ "index": "byElectedCharterMember",
+ "keys": { "electedCharterId": "electedCharterId", "memberId": "." }
+ }
+ }
+ ]
+}
+```
+
+The writer must be one of the `members` of the `electedCharter` this document's `electedCharterId` names, or have an `addedModerator` document for that charter that exists now. Here the expression is the writer's reference (see [Writer and Creator References](owner-refers-to.md)); the same declaration works on an identifier property, where it judges the property's value.
+
+## `allOf`
+
+| | |
+|---|---|
+| **Where** | In place of a single target: in `refersTo` on an identifier property or on the `items` of a typed array of identifiers, in `ownerRefersTo` and `creatorRefersTo`, and as an operand of an `anyOf`. Not on a key id property. |
+| **Value** | `{ "allOf": [ ... ] }`, the declaration's only key: 2 to 4 distinct [operands](#operands). |
+| **Since** | protocol version 14 |
+| **On update** | Fixed (`IncompatibleDocumentTypeSchemaError`, 10246), including a change of operand order. |
+| **Errors** | None of its own: the write is refused with the first failing operand's error, for example `ReferencedEntityNotFoundError` (40120). |
+
+The operands are checked in the order they are listed, and the first that fails decides: the rest are not read, and the write is refused with that operand's error.
+
+```json
+"memberId": {
+ "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "refersTo": {
+ "allOf": [
+ {
+ "type": "listElement",
+ "contractId": "EG7RGfV8fDTayC2FyVr8HwdpJh3fXDbVztcfE94UmN88",
+ "documentType": "electedCharter",
+ "propertyAgreement": { "electedCharterId": "$id" },
+ "inList": "members"
+ },
+ {
+ "type": "deletableDocument",
+ "contractId": "Bwr4WHCPz5rFVAD87RqTs3izo4zpzwsEdKPWUT1NS1C7",
+ "documentType": "profile",
+ "lookup": { "index": "ownerId", "keys": { "$ownerId": "." } }
+ }
+ ]
+ },
+ "position": 1
+}
+```
+
+This property of a hypothetical moderator directory, next to an `electedCharterId` identifier property, must be one of the elected members of that charter in the moderation charters contract, and must have a DashPay profile when the document is written. The list check comes first, so a value that is not a member is refused without the profile lookup being read.
+
+## Operands
+
+An operand is a leaf or a nested expression.
+
+**Leaves.** A leaf is an ordinary target declaration with its own keys, one of:
+
+- `identity`;
+- `permanentDocument`, by id or with a [`lookup`](refers-to-lookup.md);
+- [`listElement`](refers-to-list-element.md);
+- `deletableDocument` with a `lookup`.
+
+The first three are existence checks against things that are never deleted, so an expression made only of them holds for good once it holds. A deletable lookup may find nothing later, which is why an expression holding one is checked on every replace.
+
+The other targets are refused as operands, since they do not compose with other operands:
+
+- `deletableDocument` by id: once its document is deleted a replace may clear the property, which assumes the property refers to that one target.
+- `identityPublicKey`, in either form: it pairs the value with a key id that no other operand reads.
+- `contract`: its requirements are gates judged against the block time and the writer, not an existence check, and a contract id is never also an identity or document id.
+- `token`: a token id is never also an identity or document id.
+
+**Nesting.** An operand may be an expression of the other combinator: an `allOf` inside an `anyOf`, or an `anyOf` inside an `allOf`. An `anyOf` directly inside an `anyOf`, or an `allOf` inside an `allOf`, is refused, since it says what one flat list says.
+
+A [`propertyAgreement`](refers-to.md#propertyagreement) or a `lookup` belongs to its leaf, inside it: the expression itself holds nothing but its combinator.
+
+## How it works
+
+- **Order decides the error.** A refusal is always the error a leaf declared alone would give, naming the property (or the element) as a single reference would. So the author's order decides which error a writer sees: put the most general operand of an `anyOf` last, and the cheapest or most telling operand of an `allOf` first.
+- **Every read is billed.** A value the second operand of an `anyOf` holds for also pays for the first operand's read. An `allOf` whose first operand fails reads nothing more.
+- **Agreements are per leaf.** A `propertyAgreement` is checked only against its own leaf's document. A value whose first leaf fails its agreement can still be accepted through a second leaf that has none.
+- **Typed arrays.** On the `items` of a typed array, each element meets the expression on its own.
+- **Replace.** An expression is checked again when any of its leaves would be checked again alone (see [On replace](refers-to.md#on-replace)), and then it is evaluated whole, since which operands hold may have changed. An expression holding a `deletableDocument` lookup is therefore checked on every replace.
+- **Budget.** Every leaf counts against the [reference budget](refers-to.md#the-reference-budget), since each may be read for each value. An `anyOf` of two leaves on a typed array of `maxItems` 15 counts 30.
+- **Queries.** An expression cannot be the join property of a chained query or of a composite join by id, and a `preallocated` index is never bound through one.
+
+## Rules at registration
+
+- The combinator is the declaration's only key, and lists at least two operands: a single one is declared on its own. No two operands of one list may be alike; a leaf naming the declaring contract's id in `contractId` is the same as one leaving it out.
+- A list holds at most **4** operands, and any path from the declaration to a leaf passes through at most **4** combinators. The `anyOf` holding an `allOf` is 2 deep.
+- Every leaf is one of the four admitted targets, and is checked exactly as the same target declared alone: its document type, the type's deletability, its `propertyAgreement` and its `lookup` or list. Every leaf must pass, since each has to be a declaration that could hold. The errors name the leaf by where it sits: `refersTo anyOf[1].allOf[1] lookup: ...` from the parser, `resignation.memberId.anyOf[1].allOf[1]` from the registration check.
+- An `immutable` property may not hold an expression with a `deletableDocument` lookup leaf (`InvalidContractStructure`, 10231).
+- Every leaf counts against the reference budget of 256.
+
+A malformed expression is refused by the meta-schema (`JsonSchemaError`, 10101) or the parser (`InvalidContractStructure`, 10231); a leaf that cannot hold, with the reference error it would get alone (see [Errors](refers-to.md#errors)). On an update, any change is refused (10246): an operand added, removed, changed or moved, `anyOf` swapped for `allOf`, or a single target turned into an expression or back.
+
+## See also
+
+- [References (refersTo)](refers-to.md) for the targets and the replace rules.
+- [Lookups](refers-to-lookup.md), [List Elements](refers-to-list-element.md) and [Writer and Creator References](owner-refers-to.md), whose declarations are the usual leaves and holders of an expression.
+- [Reference expressions](../data-model/documents.md#reference-expressions-anyof-allof) in the Documents chapter, with the internals.
+- [propertyConstraints](property-constraints.md), whose rules also combine with `anyOf` and `allOf` but compare the document's own values instead of reading other state.
diff --git a/book/src/contract-keywords/refers-to-list-element.md b/book/src/contract-keywords/refers-to-list-element.md
new file mode 100644
index 00000000000..df4cad192bc
--- /dev/null
+++ b/book/src/contract-keywords/refers-to-list-element.md
@@ -0,0 +1,87 @@
+# List Elements
+
+A `listElement` reference says the value must be one of the identifiers a list on another document holds. `inList` names the list, and the `$id` pair of `propertyAgreement` names the document holding it. Reach for it when membership is written down once as a list on a document that never changes, such as the members elected with a charter, and other documents must name one of them.
+
+| | |
+|---|---|
+| **Where** | `refersTo` on an identifier property, or on the `items` of a typed array of identifiers, where every element must be listed; an operand of an [expression](refers-to-expressions.md); or [`ownerRefersTo` and `creatorRefersTo`](owner-refers-to.md), where the writer or the creator must be listed. |
+| **Value** | `"type": "listElement"` with `documentType` (the type holding the list), `propertyAgreement` (exactly one pair with `$id` on its referenced side, and up to nine other pairs), `inList` (the path of a typed array of identifiers on that type), and optionally `contractId`. |
+| **Since** | protocol version 14 |
+| **On update** | Fixed, like the rest of `refersTo` (`IncompatibleDocumentTypeSchemaError`, 10246). |
+| **Errors** | `ReferencedEntityNotFoundError` (40120) when the value is not in the list or the list's document is not found; `ReferencedDocumentPropertyMismatchError` (40127) for another pair; at registration `ReferencedDocumentListInvalidError` (40138) for a list of another contract. |
+
+## Example
+
+The moderation charters contract lets the leader of a seated team take an elected member off it with a `removedModerator` document:
+
+```json
+"removedModerator": {
+ "type": "object",
+ "documentsMutable": false,
+ "canBeDeleted": true,
+ "properties": {
+ "electedCharterId": {
+ "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "refersTo": {
+ "type": "permanentDocument",
+ "documentType": "electedCharter",
+ "propertyAgreement": { "$ownerId": "$ownerId" }
+ },
+ "position": 0
+ },
+ "memberId": {
+ "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "distinctFrom": "$ownerId",
+ "refersTo": {
+ "type": "listElement",
+ "documentType": "electedCharter",
+ "propertyAgreement": { "electedCharterId": "$id" },
+ "inList": "members"
+ },
+ "position": 1
+ }
+ },
+ "required": ["$createdAt", "electedCharterId", "memberId"],
+ "additionalProperties": false
+}
+```
+
+`electedCharterId` must name an elected charter the writer owns, so only the team's leader can write one. `memberId` must be one of the `members` of the `electedCharter` document whose `$id` this document's `electedCharterId` holds, so only an elected member can be removed. `electedCharter` is immutable and can never be deleted, so its `members` never change.
+
+## How it works
+
+When the referring document is created or replaced:
+
+1. The document holding the list is the one whose `$id` equals the value of the `$id` pair's referring property. It is fetched by id, once per write: in the example the `electedCharterId` reference and the list reference share one fetch. The list is collected once, so checking many values against it costs one read.
+2. The other `propertyAgreement` pairs are checked against that document, as for any document reference (`ReferencedDocumentPropertyMismatchError`, 40127).
+3. The value must be in the list. On a typed array every element must be; on the writer's or creator's reference, the writer or creator must be.
+
+A value not in the list, a list document that does not exist, or a value set while the `$id` property is not, refuses the write with `ReferencedEntityNotFoundError` (40120), naming the property, or an element by its list path (`memberIds[1]` for the second):
+
+```text
+electedCharter 7kX...: members [Alice, Bob]
+removedModerator { electedCharterId: 7kX..., memberId: Alice } -> accepted
+removedModerator { electedCharterId: 7kX..., memberId: Carol } -> refused, 40120:
+ referenced list element (members of the electedCharter document electedCharterId names)
+ not found for path memberId
+```
+
+A replace checks the reference again when its value changed, or when the referring side of any of its pairs changed: the `$id` property among them, since it may now name another document, whose list is then checked against every value. A pair keyed by `$ownerId` is checked on every replace. When only a typed array of values changed, the elements the stored list already held are not checked again. Nothing else can make a validated value unlisted: the list's document is never deleted and its list never changes.
+
+## Rules at registration
+
+- `propertyAgreement` holds exactly one pair with `$id` on the referenced side. Its referring side is an identifier property of the referring document type: not `$ownerId` (no document has the writer's id) and not a typed array (one document holds the list). It must be stored, so it and every object around it are not `transient`, and a reader can tell from the stored document which list the value was checked against. It may be optional. It needs no `refersTo` of its own; if it has one, that must be a reference by id to `documentType` in the list's contract, or the pair could never hold. Refused with `InvalidContractStructure` (10231) otherwise.
+- The other pairs follow the rules of every [`propertyAgreement`](refers-to.md#propertyagreement) (`ReferencedDocumentPropertyAgreementInvalidError`, 40126).
+- `documentType` must exist (`ReferencedDocumentTypeNotFoundError`, 40121), and its documents must never disappear: `canBeDeleted: false`, no `canBeDeletedByModerators` and no `ttl`.
+- `inList` is a property path (a dotted one for a nested list, never a `$` name) of a stored typed array of identifiers on `documentType`, and the list must be fixed once a document is written: the type is immutable (`documentsMutable: false`), or the list's top-level property is listed under `immutable`. A list that is also under `immutableAllowSetting` qualifies, since it can only be set once on a document that had none, and an absent list accepted no value.
+- The checks on `documentType` and `inList` are refused with `InvalidContractStructure` (10231) for a document type of the declaring contract. For one of another contract, a deletable type is refused with `ReferencedDocumentTypeDeletableError` (40122) and a list that does not qualify with `ReferencedDocumentListInvalidError` (40138).
+- Each value counts one against the [reference budget](refers-to.md#the-reference-budget), `maxItems` for a typed array.
+
+## See also
+
+- [References (refersTo)](refers-to.md) for `documentType`, `contractId`, `propertyAgreement` and the replace rules.
+- [Expressions](refers-to-expressions.md) and [Writer and Creator References](owner-refers-to.md), where a list element is a common operand or target.
+- [An element of a list](../data-model/documents.md#an-element-of-a-list-listelement) in the Documents chapter, with the internals.
+- [Typed Arrays](typed-arrays.md) and [Mutability](mutability.md) for the list and the rules that keep it fixed.
diff --git a/book/src/contract-keywords/refers-to-lookup.md b/book/src/contract-keywords/refers-to-lookup.md
new file mode 100644
index 00000000000..3223a77dd7f
--- /dev/null
+++ b/book/src/contract-keywords/refers-to-lookup.md
@@ -0,0 +1,124 @@
+# Lookups
+
+A `lookup` lets a document reference find its target through a unique index of the referenced document type instead of by id. The property's value is then one part of the index key, the rest comes from the referring document or its writer, and the reference holds if the index finds a document. Reach for it when the natural value is an identity (a member, an author, a recipient) and the rule is "this identity has a document of that kind": with an id reference the writer would have to find the document's id first, and the id would say nothing about who it belongs to.
+
+| | |
+|---|---|
+| **Where** | Inside a `permanentDocument` or `deletableDocument` [`refersTo`](refers-to.md) declaration: on an identifier property, on the `items` of a typed array of identifiers, as an operand of an [expression](refers-to-expressions.md), or in [`ownerRefersTo` and `creatorRefersTo`](owner-refers-to.md). |
+| **Value** | `{ "index": ..., "keys": { ... } }`. `index` is the name of a unique index of `documentType`, 1 to 32 characters. `keys` maps every property of that index (1 to 10) to where its value comes from: `"."` (the reference's own value), `"$ownerId"` (the writer) or a property path of the referring document type. |
+| **Since** | protocol version 14 |
+| **On update** | Fixed, like the rest of `refersTo`: adding, removing or changing a lookup is refused (`IncompatibleDocumentTypeSchemaError`, 10246). |
+| **Errors** | `ReferencedEntityNotFoundError` (40120) when the index finds no document; at registration `ReferencedDocumentLookupInvalidError` (40137) for an index of another contract, `InvalidContractStructure` (10231) for one of the same contract. |
+
+## Example
+
+The moderation charters contract's `joinRequest` type, which is immutable and can never be deleted, has this unique index: one join request per proposal and owner.
+
+```json
+"indices": [
+ {
+ "name": "bySubmittedCharter",
+ "properties": [{ "submittedCharterId": "asc" }, { "$ownerId": "asc" }],
+ "unique": true
+ }
+]
+```
+
+The same contract's `electedCharter` lists its team in `members`, each of whom must have asked to join:
+
+```json
+"members": {
+ "type": "array", "minItems": 0, "maxItems": 15, "uniqueItems": true,
+ "items": {
+ "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "distinctFrom": "$ownerId",
+ "refersTo": {
+ "type": "permanentDocument",
+ "documentType": "joinRequest",
+ "lookup": {
+ "index": "bySubmittedCharter",
+ "keys": { "submittedCharterId": "submittedCharterId", "$ownerId": "." }
+ }
+ }
+ },
+ "position": 2
+}
+```
+
+Every member must be the owner of a `joinRequest` whose `submittedCharterId` equals this elected charter's `submittedCharterId`. For each element the key is `submittedCharterId` read from the elected charter and `$ownerId` filled with the element itself (`"."`). The same form works on a single identifier property, where `"."` is the property's value.
+
+## How it works
+
+### Assembling the key
+
+When the referring document is created or replaced, a key is assembled for each value (each element of a typed array), one part per entry of `keys`:
+
+- `"."`: the value being checked, the property's value or the element.
+- `"$ownerId"`: the referring document's owner, the writer.
+- a property path (`"submittedCharterId"`, `"meta.charterId"`): the referring document's value at that path.
+
+The index is queried for at most one document, billed as a document fetch. If it finds one, the reference holds, and any [`propertyAgreement`](refers-to.md#propertyagreement) pairs beside the `lookup` are checked against that document (`ReferencedDocumentPropertyMismatchError`, 40127). If it finds none, the write is refused with `ReferencedEntityNotFoundError` (40120) naming the property, or the element by its list path (`members[1]`); the error's target reads "found through unique index" and the index name.
+
+### Permanent and deletable lookups
+
+A `permanentDocument` lookup never dangles. Its referenced documents are never deleted, and registration makes sure the key of every one of them is fixed once written (see [below](#rules-at-registration)), so the document a key found stays there. A replace checks it again only:
+
+- when the property itself changed (for a typed array, only the elements the stored list did not hold);
+- when a property a key reads changed, and then every value, every element included;
+- when an agreement's referring property changed, or on every replace for a pair keyed by `$ownerId`.
+
+A key part read from `"$ownerId"` never triggers a check: only a type whose documents keep their writer may read it.
+
+A `deletableDocument` lookup promises less. Once the document a key found is deleted, a new document filed later under the same key makes the reference hold again, with different content, where an id reference to a deleted document stays dead. So a deletable lookup means "a document with this key exists now", which is what a membership gate needs. Every replace checks it again, whether or not the replace touched it, and no `immutable` property may hold one. The moderation charters contract's `resignationRequest` uses one to accept a writer who has an `addedModerator` document for the charter at the time of writing, as the alternative to being an elected member (see [Writer and Creator References](owner-refers-to.md)).
+
+A lookup can also reach into another contract. This `recipientId` must be an identity that has a DashPay profile when the document is written:
+
+```json
+"recipientId": {
+ "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "refersTo": {
+ "type": "deletableDocument",
+ "contractId": "Bwr4WHCPz5rFVAD87RqTs3izo4zpzwsEdKPWUT1NS1C7",
+ "documentType": "profile",
+ "lookup": { "index": "ownerId", "keys": { "$ownerId": "." } }
+ },
+ "position": 0
+}
+```
+
+DashPay's `profile` type has a unique index `ownerId` on `$ownerId`, and profiles can be deleted, so the reference is a `deletableDocument` one.
+
+### What a lookup cannot do
+
+The value of a lookup reference is a key part, not a document id. So a lookup reference cannot be the join property of a chained query or of a composite join by id, and a `preallocated` index is never bound through one (see [Index-Only Types](index-only.md)). A lookup counts as one reference per value against the [reference budget](refers-to.md#the-reference-budget), like any other.
+
+## Rules at registration
+
+**On the referring side**, checked for every contract by the parser (`InvalidContractStructure`, 10231):
+
+- `lookup` is only allowed on `permanentDocument` and `deletableDocument` references.
+- `"."` fills exactly one key part. Without it every value would find the same document.
+- Every other source is `"$ownerId"` or a property path. No other `$` name is accepted, and a path may not name the reference property itself: write `"."` for that.
+- A property a key reads must exist on the referring type, be required (and so must every object around it), not be `transient` or inside a transient object, and hold a single value, not an object or an array. A lookup never runs with a missing key part, and a reader can assemble the same key from the stored document.
+- A `"$ownerId"` key part needs a referring type whose documents can be neither transferred nor traded: a transfer or a purchase would move the writer part of the key without a write.
+- A `deletableDocument` lookup may not sit under an `immutable` property, alone or as an operand of an expression: every replace checks it again, so once its document is deleted the property would have to change.
+
+**On the referenced side**, refused with `InvalidContractStructure` (10231) for a document type of the declaring contract and with `ReferencedDocumentLookupInvalidError` (40137) for one of another contract:
+
+- The index exists and is `unique`, so a key finds at most one document. It does not bucket its first property by a `timeRange`, the referenced type is not `indexOnly`, and no index property is `transient`.
+- `keys` maps every property of the index exactly once, by its name on the referenced side (system properties such as `$ownerId` included), in any order, and nothing else.
+- Each source holds the same type of value as the index property it fills. `"."` and `"$ownerId"` are identifiers.
+- The key stays with the document it found. Every schema property of the index must be fixed once written: the referenced type is immutable (`documentsMutable: false`), or the property's top-level property is listed under `immutable`. `$ownerId` may be a key part only where the referenced documents can be neither transferred nor traded. `$updatedAt` and its block height forms may be one only where they can be neither replaced, transferred nor traded, and `$transferredAt` and its forms only where they can be neither transferred nor traded. `$id`, `$creatorId`, `$createdAt` and its forms never change.
+
+**The referenced type** must exist (`ReferencedDocumentTypeNotFoundError`, 40121). A `permanentDocument` lookup into a type whose documents can disappear is refused with `ReferencedDocumentTypeDeletableError` (40122), and a `deletableDocument` lookup into one whose documents cannot with `ReferencedDocumentTypeNotDeletableError` (40131).
+
+Index definitions and the flags these rules read cannot change on a contract update, so a lookup that registered keeps resolving.
+
+## See also
+
+- [References (refersTo)](refers-to.md) for the targets, `propertyAgreement` and the replace rules.
+- [Expressions](refers-to-expressions.md), where a lookup may be an operand, and [Writer and Creator References](owner-refers-to.md), where `"."` is the writer or the creator.
+- [Resolved through a unique index](../data-model/documents.md#resolved-through-a-unique-index-lookup) in the Documents chapter, with the internals.
+- [Indexes (indices)](indexes.md), [Time-Range Indexes](time-range.md), [Mutability](mutability.md) and [transient](transient.md) for the keywords the rules read.
diff --git a/book/src/contract-keywords/refers-to.md b/book/src/contract-keywords/refers-to.md
new file mode 100644
index 00000000000..95a3a456f86
--- /dev/null
+++ b/book/src/contract-keywords/refers-to.md
@@ -0,0 +1,325 @@
+# References (refersTo)
+
+An identifier (a 32-byte id) can hold any value. `refersTo` says what it points at, and Platform then checks, whenever a document is created or replaced, that the thing it names exists: an identity, a data contract, a token, a document, or one key of an identity. Reach for it when a document only makes sense next to something else: a reply needs its post, a join request needs the proposal it joins, an encrypted message needs the key it was encrypted to. The check runs when the document is written. Nothing checks the reference again when its target changes later, and nothing resolves it for a reader.
+
+| | |
+|---|---|
+| **Where** | An identifier property, at the top level or inside an object; the `items` of a typed array of identifiers, where every element is checked; and, for one form of `identityPublicKey`, an integer key id property. `ownerRefersTo` and `creatorRefersTo` carry the same declaration at the document type level. |
+| **Value** | An object: `type`, naming one [target](#targets), with the [keys](#keys) that target takes; or an object holding only `anyOf` or only `allOf` (see [Expressions](refers-to-expressions.md)). |
+| **Default** | Absent: the identifier is not checked against anything. |
+| **Since** | protocol version 14 |
+| **On update** | Fixed: adding, removing or changing any part of a declaration is refused (`IncompatibleDocumentTypeSchemaError`, 10246). |
+| **Errors** | `ReferencedEntityNotFoundError` (40120) when the target does not exist; the full list is under [Errors](#errors). |
+
+## Example
+
+```json
+"reply": {
+ "type": "object",
+ "documentsMutable": true,
+ "canBeDeleted": true,
+ "properties": {
+ "postId": {
+ "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "refersTo": { "type": "deletableDocument", "documentType": "post" },
+ "position": 0
+ },
+ "mentions": {
+ "type": "array", "minItems": 0, "maxItems": 5, "uniqueItems": true,
+ "items": {
+ "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "refersTo": { "type": "identity" }
+ },
+ "position": 1
+ },
+ "text": { "type": "string", "minLength": 1, "maxLength": 280, "position": 2 }
+ },
+ "required": ["postId", "text"],
+ "additionalProperties": false
+}
+```
+
+`postId` must be the id of a `post` document of this contract that exists when the reply is written. Posts can be deleted, so the reference is a `deletableDocument` one. Each of the up to five `mentions` must be the id of an existing identity. The type carries six references at most: one for `postId` and five for `mentions`.
+
+## Targets
+
+`type` names what the value points at.
+
+| `type` | The value must be | Keys it takes |
+|---|---|---|
+| `identity` | the id of an existing identity | none |
+| `contract` | the id of an existing data contract | `contractRequirements` |
+| `token` | the id of an existing token | none |
+| `permanentDocument` | the id of an existing document of a type whose documents can never disappear | `documentType` (required), `contractId`, `propertyAgreement`, `lookup` |
+| `deletableDocument` | the id of an existing document of a type whose documents can disappear | `documentType` (required), `contractId`, `propertyAgreement`, `lookup` |
+| `identityPublicKey` | an identity key that exists and is not disabled | `keyIdProperty` or `identityProperty` (one of them, required), `keyRequirements` |
+| `listElement` | one of the identifiers a list on another document holds | `documentType`, `propertyAgreement` and `inList` (all required), `contractId` |
+
+A key that belongs to another target is refused when the contract is registered.
+
+### `identity`
+
+The value is the id of an identity that exists. The check reads the identity's revision. Identities are never removed, so a validated reference never dangles.
+
+### `contract`
+
+The value is the id of a data contract that exists. [`contractRequirements`](#contractrequirements) can ask more of it: that it declares elected moderation, has reached a certain age, belongs to the writer, and so on. Contracts are never deleted.
+
+### `token`
+
+The value is the id of a token. The check reads the token's record, which is written when its contract is registered and never removed.
+
+### `permanentDocument`
+
+The value is the id of a document of `documentType`, in this contract or in the one `contractId` names. The referenced type must be one whose documents can never disappear: `canBeDeleted: false`, no `canBeDeletedByModerators` and no `ttl`. None of those can change on a contract update and document types are never removed, so a validated permanent reference never dangles.
+
+```json
+"reasons": {
+ "type": "array", "minItems": 0, "maxItems": 64, "uniqueItems": true,
+ "items": {
+ "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "refersTo": { "type": "permanentDocument", "documentType": "reason" }
+ },
+ "position": 2
+}
+```
+
+This is the moderation charters contract's `submittedCharter.reasons`: every element must be the id of a `reason` document, a type that is immutable and can never be deleted. With a [`lookup`](refers-to-lookup.md) the value is instead one part of a unique index key that finds the document.
+
+### `deletableDocument`
+
+The same for a type whose documents can disappear: deleted by their owner (`canBeDeleted`), removed by the contract's moderators (`canBeDeletedByModerators`), or removed by the platform when their `ttl` passes. Any one of the three makes a type deletable for references. The document must exist when the referring document is written, and may be deleted afterwards.
+
+Because the target may be gone, every replace of the referring document checks the reference again, whether or not the replace touched it. Once the target is deleted, the replace has to point the property at a document that exists or remove it. A required property cannot be removed, so a document whose required reference has lost its target can be replaced only after it is repointed; it can still be deleted. A single `deletableDocument` reference held by an `immutable` top-level property may be removed by a replace once its target is gone, an exception to the immutability rule.
+
+### `identityPublicKey`
+
+One key of one identity, which must exist and not be disabled. Identity keys can be disabled but never removed, so a validated key reference never dangles, while a disabled key refuses new writes. The declaration comes in two forms, which differ in which property carries it:
+
+- **On the identity property.** The identifier holds the identity's id and [`keyIdProperty`](#keyidproperty-and-identityproperty) names the sibling integer property holding the key id. The moderation charters contract's `joinRequest.recipientId`:
+
+ ```json
+ "recipientId": {
+ "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "refersTo": {
+ "type": "identityPublicKey",
+ "keyIdProperty": "recipientKeyId",
+ "keyRequirements": { "purpose": "decryption", "boundTo": "submittedCharter" }
+ },
+ "position": 1
+ }
+ ```
+
+- **On the key id property.** The integer holds the key id and [`identityProperty`](#keyidproperty-and-identityproperty) names whose key it is. The property must declare exactly the range of a key id, `"minimum": 0` and `"maximum": 4294967295`. The same contract's `joinRequest.senderKeyId`, a key of the writer:
+
+ ```json
+ "senderKeyId": {
+ "type": "integer", "minimum": 0, "maximum": 4294967295,
+ "refersTo": {
+ "type": "identityPublicKey",
+ "identityProperty": "$ownerId",
+ "keyRequirements": { "purpose": "encryption", "boundTo": "joinRequest" }
+ },
+ "position": 3
+ }
+ ```
+
+A key reference pairs the value with one key id, so it is refused on the elements of a typed array, as an operand of an expression and in `ownerRefersTo` or `creatorRefersTo`.
+
+### `listElement`
+
+The value must be one of the identifiers a typed array on another document holds. See [List Elements](refers-to-list-element.md).
+
+## Keys
+
+### `documentType`
+
+The name of the referenced document type, 1 to 64 letters, digits or underscores. Required on `permanentDocument`, `deletableDocument` and `listElement`, and refused on the other targets. For `permanentDocument` and `listElement` the type's documents must never disappear; for `deletableDocument` they must be able to.
+
+### `contractId`
+
+The contract holding `documentType`, as a base58 string or an array of 32 bytes. Absent means the declaring contract; naming the declaring contract's own id means the same. Only on the three document targets. A reference into another contract costs a billed fetch of that contract when a document is written, once per declaration: the elements of a typed array and the operands of an expression share it.
+
+### `propertyAgreement`
+
+Binds the referring document to the document it references: each pair `{ "": "" }` must hold as an equality when the referring document is written. It takes 1 to 10 pairs, on `permanentDocument`, `deletableDocument` and `listElement` references.
+
+- **The referring side** (the key) is a property of the declaring document type, a dotted path for a nested one, or `$ownerId`, the writer. A pair keyed by `$ownerId` is a write gate: only an identity whose id equals the referenced side may create or replace the document.
+- **The referenced side** (the value) is a property of the referenced document type, or one of the referenced document's own identifiers: `$ownerId` (its current owner, which follows it through transfers), `$creatorId` (its creator, which never changes, on types that record it, see [System Properties](system-properties.md)) or `$id` (its id). A system name needs an identifier on the referring side.
+- **Absence counts.** Both sides absent agree; one side absent is a mismatch, as a different value is.
+- **Values, not keys.** The two sides compare as values of their type: strings as text (so `""` is not `"\0"`), byte arrays and identifiers as bytes (an identifier equals the same 32 bytes, however it is carried), integers as numbers whatever width they are carried at, floats exactly, bit for bit (so `-0.0` is not `0.0`). A value of any length compares, past the 255 bytes of an index key too.
+- The comparison reads the document already fetched for the existence check, so it costs nothing more. A pair that does not hold refuses the write with `ReferencedDocumentPropertyMismatchError` (40127).
+
+```json
+"submittedCharterId": {
+ "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "refersTo": {
+ "type": "permanentDocument",
+ "documentType": "submittedCharter",
+ "propertyAgreement": {
+ "$ownerId": "$ownerId",
+ "targetContractId": "targetContractId"
+ }
+ },
+ "position": 1
+}
+```
+
+This is the moderation charters contract's `electedCharter.submittedCharterId`: the proposal it names must be owned by the writer, and must be for the same `targetContractId` as the elected charter. Other patterns: `{ "authorId": "$ownerId" }` makes a like carry its post's owner, and `{ "$ownerId": "$creatorId" }` lets only the referenced document's creator write.
+
+At registration both sides must exist and hold the same type of value (for integers, the same stored integer type). Neither side may be an object or a typed array, the referring side may not be the reference property itself, and the referenced side may not be `transient` (no stored document carries it; the referring side may be). A pair through which a [`preallocated`](index-only.md#preallocated) index is keyed needs a referenced property whose every value fits an index key, at most 255 bytes: a string of at most 63 characters or with `maxBytes` at most 255, or a byte array of at most 255 bytes. A pair breaking one of these is refused with `ReferencedDocumentPropertyAgreementInvalidError` (40126).
+
+### `contractRequirements`
+
+What a `contract` reference requires of the referenced contract beyond existing. It holds at least one of these keys:
+
+| Key | Value | Met when |
+|---|---|---|
+| `moderation` | `"elected"` | the contract declares an elected moderation team |
+| | `"electionOpen"` | the contract declares an elected team, and its own `electionDelay`, counted from the contract's creation, has passed at the block time of the write (or it declares no delay) |
+| `minimumAgeSeconds` | integer, 1 to 4294967295 | the contract's recorded creation time is at least that many seconds before the block time of the write |
+| `minimumSecondsSinceUpdate` | integer, 1 to 4294967295 | the same, counted from the later of its creation and its last update |
+| `owner` | `"self"` | the contract is owned by the writer of the referring document |
+| | `"other"` | the contract is owned by anyone else |
+| `readonly` | `true` | the contract's config is `readonly`: it can never be updated again |
+| `keepsHistory` | `true` | the contract's config keeps history |
+| `ownerProtected` | `true` or `false` | the contract's elected moderation protects, or does not protect, its owner from the team; a contract without elected moderation meets neither |
+
+A contract with no recorded creation time never meets `minimumAgeSeconds` or `minimumSecondsSinceUpdate`. The requirements are judged against the contract already fetched for the existence check, the writer and the block time, so they cost no further read. The first unmet one refuses the write with `ReferencedContractRequirementNotMetError` (40135); a contract that does not exist is still `ReferencedEntityNotFoundError` (40120).
+
+The moderation charters contract uses both moderation values: a proposal (`submittedCharter.targetContractId`) needs a target that is `elected`, so teams can form while the target's election delay runs, and the charter that opens the election (`electedCharter.targetContractId`) needs it `electionOpen`:
+
+```json
+"refersTo": { "type": "contract", "contractRequirements": { "moderation": "electionOpen" } }
+```
+
+`owner` is the one requirement judged against the writer, and a transfer or a purchase changes the owner without a write. On a type whose documents can be transferred or traded, a reference carrying `owner` is therefore checked again on every replace, so a new owner has to repoint it at a contract that meets the requirement for them, or remove it where it is optional. Such a reference may not sit under an `immutable` property of such a type, which could never be repointed. See [Elected Moderation](../data-model/contract-moderation.md#elected-moderation) for the moderation values.
+
+### `keyIdProperty` and `identityProperty`
+
+The two forms of `identityPublicKey`. A declaration takes exactly one of them.
+
+- **`keyIdProperty`**, on the identity property: the path of the integer property of the same document type holding the key id. It must exist, be an integer, and not carry a key reference of its own. If the identity is set and the key id is not, the write is refused with `ReferencedKeyIdPropertyInvalidError` (40125).
+- **`identityProperty`**, on the key id property: whose key the value is.
+ - `"$ownerId"`: the writer. The writer's existence is already proven, so the key fetch is the only read.
+ - `"$creatorId"`: the document's creator, only on a type that records creator ids. A document written before its type recorded them has none, and setting the key id on it is refused (40125).
+ - The path of an identifier property of the same type, which must exist, be an identifier and not carry an `identityPublicKey` reference of its own. A key id set while that property is not set is refused (40125).
+
+A stored key id may not be paired with a `transient` identity, since the key id alone names no key. Each rule is checked at registration and refused with `ReferencedKeyIdPropertyInvalidError` (40125).
+
+### `keyRequirements`
+
+What an `identityPublicKey` reference requires of the key beyond existing and not being disabled. It holds at least one of:
+
+- `purpose`: the key's purpose, one of `authentication`, `encryption`, `decryption`, `transfer`, `voting` or `owner`.
+- `boundTo`: a document type of the declaring contract. The key must be bound to exactly this contract and that document type; a key bound to the whole contract or to a contract group does not meet it.
+
+At registration `boundTo` must name a document type of the contract, and one a key of the required purpose can be bound to: only authentication, encryption and decryption keys carry a document type bound, an encryption key only where the type declares `requiresIdentityEncryptionBoundedKey`, and a decryption key only where it declares `requiresIdentityDecryptionBoundedKey` (see [Signing and Keys](signing-keys.md)). The requirements are judged against the key already fetched, and the first unmet one refuses the write with `ReferencedIdentityKeyRequirementNotMetError` (40136). A missing key is still 40123 and a disabled one 40124.
+
+### `lookup`, `inList`, `anyOf` and `allOf`
+
+- [`lookup`](refers-to-lookup.md) finds a `permanentDocument` or `deletableDocument` through a unique index of its type, with the value as one part of the key.
+- [`inList`](refers-to-list-element.md) names the list a `listElement` value must be in.
+- [`anyOf` and `allOf`](refers-to-expressions.md) combine several targets in one declaration.
+
+## How it works
+
+### On create
+
+When a document is created, every reference is checked against the current state: the `ownerRefersTo` or `creatorRefersTo` declaration first, then each property's. The first check that fails refuses the write with that check's error, naming the property by its path (`postId`, `meta.charterId`), an element by its list path (`mentions[2]` for the third) and the writer or creator reference as `$ownerId` or `$creatorId`. A refused write is still charged for the reads it made.
+
+- A reference property the document leaves out is not checked. Whether it may be left out is up to `required`.
+- Each check is a billed read: the identity, the contract (none for the declaring contract, which is already loaded), the token's record, the document by id or through its lookup, or the key. One write fetches a given document by id once, however many references name it.
+- On a typed array each element is checked in list order as a single reference would be. An element repeating an earlier one is not checked twice.
+
+### On replace
+
+A replace checks a reference again only when its outcome could have changed:
+
+| Declaration | Checked again on a replace when |
+|---|---|
+| `identity`, `token`, `contract` | the value changed |
+| `contract` with an `owner` requirement, on a type whose documents can be transferred or traded | every replace |
+| `permanentDocument`, `listElement` | the value changed, or the referring property of an agreement pair changed |
+| any document reference with a pair keyed by `$ownerId` | every replace |
+| `permanentDocument` with a `lookup` | also when a property a key reads changed |
+| `deletableDocument`, by id or with a `lookup` | every replace |
+| `identityPublicKey` with `keyIdProperty` | the identity or the key id changed |
+| key id with `identityProperty: "$ownerId"` | every replace |
+| key id with `identityProperty: "$creatorId"` | the key id changed |
+| key id with an identity property path | the key id or that property changed |
+| `anyOf` or `allOf` | one of its operands would be checked again; the whole expression is then checked |
+
+A value changed when the replace set it differently, added it or removed it. Changes are tracked per top-level property, so a change anywhere in an object checks again every reference inside that object. When a typed array changed, only the elements the stored list did not hold are checked, unless a rule above that applies to every element does (an agreement's referring property changed, a `$ownerId` pair, an `owner` requirement on such a type, or a `deletableDocument` target); then every element is checked. The rules for `ownerRefersTo` and `creatorRefersTo` are in [Writer and Creator References](owner-refers-to.md).
+
+The "every replace" rows exist because something the reference depends on can change without a write to the referring document: the writer after a transfer or purchase, or the target's existence for a deletable one.
+
+### Transfers, purchases, deletes and restores
+
+- A transfer or a purchase checks no reference. A reference governs writing, not holding: a new owner meets the writer gates (a `$ownerId` pair, `identityProperty: "$ownerId"`, an `owner` requirement) on their first replace.
+- Deleting a referring document checks nothing. Deleting a referenced document does not look for documents referring to it: a `permanentDocument` target cannot be deleted at all, and a `deletableDocument` reference meets its missing target on the referring document's next replace.
+- A document a moderator removed and later restores comes back as it was, without its references being checked again (see [Restoring Documents](../data-model/contract-moderation.md#restoring-documents)).
+
+## The reference budget
+
+Every reference is a billed read when a document is written, so a document type may carry at most **256** references per document. Registration counts:
+
+- one for each property declaring `refersTo`, whether an identifier or a key id;
+- `maxItems` for each typed array whose elements declare it;
+- one for the type's `ownerRefersTo` or `creatorRefersTo`;
+- each of these multiplied by the number of leaves when the declaration is an expression.
+
+The `reply` above counts 6. A typed array of `maxItems` 15 whose elements declare an `anyOf` of two targets counts 30. A type over the budget is refused at registration with `InvalidContractStructure` (10231).
+
+## Rules at registration
+
+A contract's declarations are checked when it is registered, and again for the whole contract on every update. A declaration that is malformed is refused by the meta-schema (`JsonSchemaError`, 10101) or by the parser (`InvalidContractStructure`, 10231). A declaration that is well formed but cannot hold is refused with the reference errors below: those are judged against the contract itself for its own document types, and against the stored contract for another contract's.
+
+- `refersTo` sits on an identifier property or on the `items` of a typed array of identifiers. On the array itself it is refused: the declaration belongs on its `items`. The one exception is the key id form of `identityPublicKey`, on an integer property with exactly `"minimum": 0` and `"maximum": 4294967295`.
+- A declaration holds `type` and the keys its target takes, or a single `anyOf` or `allOf`.
+- A referenced `documentType` must exist (`ReferencedDocumentTypeNotFoundError`, 40121). Its documents must never disappear for `permanentDocument` and `listElement` (`ReferencedDocumentTypeDeletableError`, 40122) and must be able to for `deletableDocument` (`ReferencedDocumentTypeNotDeletableError`, 40131). A `listElement` whose list is in the declaring contract is the exception: the parser checks its type and reports a deletable one as `InvalidContractStructure` (10231).
+- Every `propertyAgreement` pair must be one that can hold (40126), every key reference must fit the document type (40125), every `boundTo` must name a type a key can be bound to (10231), and every [lookup](refers-to-lookup.md#rules-at-registration) and [list](refers-to-list-element.md#rules-at-registration) must resolve.
+- An `immutable` property may not hold a `deletableDocument` reference that a replace could not remove: one inside an object, a typed array of them, or any `deletableDocument` found through a lookup. A single `deletableDocument` reference by id that is itself an immutable top-level property is allowed, but not also under `immutableAllowSetting`, which would let a replace set it to another document once it was cleared. A `contract` reference with an `owner` requirement may not sit under an immutable property of a type whose documents can be transferred or traded. All refused with `InvalidContractStructure` (10231); see [Mutability](mutability.md).
+- The type stays within the [reference budget](#the-reference-budget).
+- On an update, every existing declaration must be unchanged (10246). A document type the update adds may declare any reference.
+
+## Errors
+
+| Error | Code | When |
+|---|---|---|
+| `ReferencedEntityNotFoundError` | 40120 | Write: the identity, contract, token or document does not exist, a lookup finds no document, or a value is not in its list. |
+| `ReferencedDocumentTypeNotFoundError` | 40121 | Registration: `documentType` does not exist, or `contractId` names no contract. |
+| `ReferencedDocumentTypeDeletableError` | 40122 | Registration: a `permanentDocument` or `listElement` reference names a type whose documents can disappear. |
+| `ReferencedIdentityKeyNotFoundError` | 40123 | Write: the identity has no key with that id, or the identity does not exist. |
+| `ReferencedIdentityKeyDisabledError` | 40124 | Write: the key is disabled. |
+| `ReferencedKeyIdPropertyInvalidError` | 40125 | Registration: `keyIdProperty` or `identityProperty` names a property that does not fit, `$creatorId` on a type that records no creator ids, or a stored key id paired with a transient identity. Write: a key id without its identity, or an identity without its key id. |
+| `ReferencedDocumentPropertyAgreementInvalidError` | 40126 | Registration: a `propertyAgreement` pair names a missing, transient, object or typed array property, or two properties of different types, or `$creatorId` on a type that does not record it, or keys a preallocated index by a property that can hold more than 255 bytes. |
+| `ReferencedDocumentPropertyMismatchError` | 40127 | Write: a `propertyAgreement` pair does not hold. |
+| `ReferencedDocumentTypeNotDeletableError` | 40131 | Registration: a `deletableDocument` reference names a type whose documents can never disappear. |
+| `ReferencedContractRequirementNotMetError` | 40135 | Write: the referenced contract exists but does not meet a `contractRequirements` entry. |
+| `ReferencedIdentityKeyRequirementNotMetError` | 40136 | Write: the key exists and is enabled but does not meet a `keyRequirements` entry. |
+| `ReferencedDocumentLookupInvalidError` | 40137 | Registration: a `lookup` into another contract's document type cannot resolve. |
+| `ReferencedDocumentListInvalidError` | 40138 | Registration: an `inList` list on another contract's document type does not qualify. |
+
+The registration errors name the declaration as `.`, `.[]` for typed array elements, `.$ownerId` or `.$creatorId` for the writer and creator references, and add the operand for a leaf of an expression (`resignation.memberId.anyOf[1]`). When a document is written, the referenced document type is looked up and its deletability checked again as a safeguard (40121, 40122, 40131), but a registered contract cannot fail those checks later: contracts and document types are never removed, and the deletion flags cannot change. The other codes in the range (40128 to 40130, 40132 to 40134) belong to other keywords. See [Error Codes](../error-handling/error-codes.md).
+
+## More forms of reference
+
+- [Lookups](refers-to-lookup.md): `lookup`, a `permanentDocument` or `deletableDocument` found through a unique index of its type, with the value as one part of the key. The value can be an identity (a member, an author) instead of a document id.
+- [Expressions](refers-to-expressions.md): `anyOf` and `allOf`, several targets combined in one declaration.
+- [List Elements](refers-to-list-element.md): `listElement` and `inList`, a value that must be one of the identifiers a list on another document holds.
+- [Writer and Creator References](owner-refers-to.md): `ownerRefersTo` and `creatorRefersTo`, the same declaration applied to the document's writer or creator instead of a property.
+- [References on the elements](../data-model/documents.md#references-on-the-elements) of a typed array: a declaration on `items`, checked for every element.
+
+## See also
+
+- [Document References](../data-model/documents.md#document-references-refersto) in the Documents chapter, with the internals.
+- [Elected Moderation](../data-model/contract-moderation.md#elected-moderation) for the moderation charters contract, the first user of most reference forms.
+- [Typed Arrays](typed-arrays.md), [System Properties](system-properties.md), [Mutability](mutability.md), [Deletion](deletion.md), [Time To Live (ttl)](ttl.md) and [Creation, Transfers and Trading](ownership-and-trading.md) for the keywords references read.
+- [distinctFrom](distinct-from.md), which requires an identifier to differ from another, and [encryptedFor](encrypted-for.md), which names the key references an encrypted property was made with.
+- [Contract Keywords](../contract-keywords.md) for the conventions these pages use.
diff --git a/book/src/contract-keywords/required-since.md b/book/src/contract-keywords/required-since.md
new file mode 100644
index 00000000000..883717d75a6
--- /dev/null
+++ b/book/src/contract-keywords/required-since.md
@@ -0,0 +1,61 @@
+# requiredSince
+
+`requiredSince` lets a contract update add a property that every new document must hold. Documents already stored were written without it and stay valid: the property is required only of documents written under the contract version the annotation names, or a later one. Reach for it when an app needs a new mandatory field and the contract is already in use.
+
+| | |
+|---|---|
+| **Where** | A top-level property listed in the document type's `required` |
+| **Value** | A contract version: an integer from 1 to 4294967295 |
+| **Default** | Absent: a property listed in `required` is required of every document |
+| **Since** | protocol version 14 |
+| **On update** | May only appear on a property the update adds, equal to the contract version the update creates (`DataContractInvalidRequiredFieldsUpdateError`, 10276). An existing annotation may not be added, changed or removed (`IncompatibleDocumentTypeSchemaError`, 10246). |
+| **Errors** | `DataContractInvalidRequiredFieldsUpdateError` (10276), `InvalidContractStructure` (10231), `IncompatibleDocumentTypeSchemaError` (10246), all at registration; `JsonSchemaError` (10101) for a new document without the property |
+
+## Example
+
+Version 2 of a contract has a `post` type with one property, `text`. The update to version 3 adds a required `language`:
+
+```json
+"post": {
+ "type": "object",
+ "properties": {
+ "text": { "type": "string", "maxLength": 280, "position": 0 },
+ "language": { "type": "string", "minLength": 2, "maxLength": 8, "requiredSince": 3, "position": 1 }
+ },
+ "required": ["text", "language"],
+ "additionalProperties": false
+}
+```
+
+Every post created or replaced under version 3 or later must have a `language`. Posts written under versions 1 and 2 have none, and remain valid as they are.
+
+## How it works
+
+From protocol version 14, every create and replace stamps the document with the version of the contract it was written under. A transfer or a purchase keeps the stamp the document had, since it does not rewrite the document's properties. See [the contract version stamp](../serialization/document-serialization.md#the-contract-version-stamp-v3).
+
+- **New writes are held to the new schema.** A create must include the property. A replace sends the whole document again, so replacing a document written before the update must add the property too; the document is then stamped with the current version. Documents catch up one at a time, as they are replaced.
+- **Older documents are left alone.** A document stamped below the property's `requiredSince` may lack it. It can still be read, transferred, sold and deleted. A document last written before protocol version 14 has no stamp, and counts as older than every annotation.
+- **Storage follows the stamp.** A required property is stored without the presence byte an optional one carries. A property with `requiredSince` is stored as required in documents stamped at or above its version, and as optional in older ones. The latest contract alone therefore tells how to read every stored document.
+- **Readers must expect gaps.** An app that reads the type should handle documents without the property: every document stamped below the annotation may lack it.
+- **No index on it.** A contract update cannot add an index to an existing document type (see [Indexes](indexes.md)), so a property added this way cannot be indexed on that type.
+
+## Rules at registration
+
+- `requiredSince` sits only on a top-level property, and only on one listed in `required`. A nested property, or one that is not required, is refused (`InvalidContractStructure`, 10231). It cannot go on the elements of a typed array.
+- The value is never higher than the contract's own version (10276): a property cannot be scheduled to become required later.
+- On a new contract, which is version 1, the value may only be 1 (10276).
+
+On an update to an existing document type, which always creates the version one above the current one:
+
+- A property the update adds may be required only if it carries `requiredSince` equal to that new version. Without the annotation, or with any other value, the update is refused (10276).
+- An existing property may not become required, with or without the annotation (10276).
+- No property may leave `required` (10276).
+- A property's existing `requiredSince` may not be changed or removed, and one may not be added to an existing property (`IncompatibleDocumentTypeSchemaError`, 10246).
+
+A document type that the update adds has no older documents. Every `requiredSince` in it must still equal the version the update creates (10276).
+
+## See also
+
+- [Evolving a Contract: Adding Required Fields](../data-model/data-contracts.md#evolving-a-contract-adding-required-fields)
+- [The contract version stamp](../serialization/document-serialization.md#the-contract-version-stamp-v3), for how the stamp decides each property's layout
+- [Document Shape](document-shape.md#required), for the other rules of `required`
diff --git a/book/src/contract-keywords/signing-keys.md b/book/src/contract-keywords/signing-keys.md
new file mode 100644
index 00000000000..7c39c1257d6
--- /dev/null
+++ b/book/src/contract-keywords/signing-keys.md
@@ -0,0 +1,147 @@
+# Signing and Keys
+
+These keywords tie a document type to identity keys. `signatureSecurityLevelRequirement` sets how strong a key must be to sign transitions on documents of the type. `requiresIdentityEncryptionBoundedKey` and `requiresIdentityDecryptionBoundedKey` let identities register encryption and decryption keys bound to the type, for applications that encrypt data between users, and say how many such keys an identity may hold.
+
+Every identity key has a purpose (authentication, encryption, decryption and others) and a security level. The levels are, from strongest to weakest, `MASTER` (0), `CRITICAL` (1), `HIGH` (2) and `MEDIUM` (3): a lower number is a stronger key. See [Identity Keys](../sdk/identity-keys.md#security-level).
+
+## `signatureSecurityLevelRequirement`
+
+The weakest key security level that may sign a transition on documents of the type. Raise it to `1` for documents whose forgery would be costly, so that a weaker everyday key cannot write them.
+
+| | |
+|---|---|
+| **Where** | document type |
+| **Value** | `1` critical, `2` high, `3` medium |
+| **Default** | `2` (high) |
+| **Since** | protocol version 1 |
+| **On update** | Fixed (`DocumentTypeUpdateError`, 40212). Adding or removing the key without changing its value is refused too, as a schema change (`IncompatibleDocumentTypeSchemaError`, 10246). |
+| **Errors** | `InvalidSignaturePublicKeySecurityLevelError` (20004) |
+
+### Example
+
+```json
+"payout": {
+ "type": "object",
+ "documentsMutable": false,
+ "canBeDeleted": false,
+ "signatureSecurityLevelRequirement": 1,
+ "properties": {
+ "recipient": {
+ "type": "array",
+ "byteArray": true,
+ "minItems": 32,
+ "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "position": 0
+ },
+ "amount": { "type": "integer", "minimum": 1, "position": 1 }
+ },
+ "required": ["recipient", "amount"],
+ "additionalProperties": false
+}
+```
+
+Only a critical key may sign a transition that creates a payout. A high or medium key of the same identity is refused.
+
+### How it works
+
+The requirement admits the level it names and every stronger level except `MASTER`:
+
+| Value | Keys that may sign |
+|---|---|
+| `1` critical | critical |
+| `2` high (the default) | critical, high |
+| `3` medium | critical, high, medium |
+
+- It applies to every document transition on the type: create, replace, delete, transfer, price update and purchase. A purchase is signed by the buyer, so the buyer needs a key at that level.
+- A batch transition signs all its transitions with one key. When it holds transitions on several document types, the key must satisfy the strictest of their requirements. A batch that also holds a token transition needs a critical key.
+- The key must be an authentication key. A master key never signs a document batch, whatever the requirement: the batch is refused at the signature check (20004).
+- A key whose level the requirement does not admit is refused with `InvalidSignaturePublicKeySecurityLevelError` (20004) after the signature has been verified. The failure is paid: the identity's nonce for the contract is bumped and the fees are charged.
+
+### Rules at registration
+
+- The meta-schema admits only `1`, `2` and `3` (`JsonSchemaError`, 10101 otherwise). `0`, master, cannot be required.
+
+## `requiresIdentityEncryptionBoundedKey`
+
+Lets identities add encryption keys bound to this document type, and says how they are kept. Use it when documents of the type carry data encrypted to their readers, and each identity publishes the key others encrypt to.
+
+| | |
+|---|---|
+| **Where** | document type |
+| **Value** | `0` unique, `1` multiple, `2` multiple with a pointer to the latest |
+| **Default** | absent: no encryption key may be bound to the type |
+| **Since** | protocol version 1 |
+| **On update** | Fixed (`DocumentTypeUpdateError`, 40212) |
+| **Errors** | `DataContractBoundsNotPresentError` (10515) for an encryption key bound to a type that does not declare it; `IdentityPublicKeyAlreadyExistsForUniqueContractBoundsError` (40211) for a second key under `0` |
+
+## `requiresIdentityDecryptionBoundedKey`
+
+The same for decryption keys.
+
+| | |
+|---|---|
+| **Where** | document type |
+| **Value** | `0` unique, `1` multiple, `2` multiple with a pointer to the latest |
+| **Default** | absent: no decryption key may be bound to the type |
+| **Since** | protocol version 1 |
+| **On update** | Fixed (`DocumentTypeUpdateError`, 40212) |
+| **Errors** | `DataContractBoundsNotPresentError` (10515) for a decryption key bound to a type that does not declare it; `IdentityPublicKeyAlreadyExistsForUniqueContractBoundsError` (40211) for a second key under `0` |
+
+### Example
+
+Trimmed from the DashPay contract's `contactRequest`:
+
+```json
+"contactRequest": {
+ "type": "object",
+ "documentsMutable": false,
+ "canBeDeleted": false,
+ "requiresIdentityEncryptionBoundedKey": 2,
+ "requiresIdentityDecryptionBoundedKey": 2,
+ "properties": {
+ "toUserId": {
+ "type": "array",
+ "byteArray": true,
+ "minItems": 32,
+ "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "position": 0
+ },
+ "encryptedPublicKey": {
+ "type": "array",
+ "byteArray": true,
+ "minItems": 96,
+ "maxItems": 96,
+ "position": 1
+ },
+ "senderKeyIndex": { "type": "integer", "minimum": 0, "position": 2 },
+ "recipientKeyIndex": { "type": "integer", "minimum": 0, "position": 3 }
+ },
+ "required": ["toUserId", "encryptedPublicKey", "senderKeyIndex", "recipientKeyIndex"],
+ "additionalProperties": false
+}
+```
+
+An identity may bind any number of encryption and decryption keys to `contactRequest`, and the platform keeps a pointer to the newest of each. The request records the ids of the sender's and the recipient's keys in `senderKeyIndex` and `recipientKeyIndex`.
+
+### How it works
+
+- An identity key may carry contract bounds: a contract, or one document type of a contract. An encryption or decryption key bound to a document type is only accepted when the type declares the matching keyword; otherwise it is refused with `DataContractBoundsNotPresentError` (10515). The check runs when an identity is created with such a key, or updated to add one. The bound contract and document type must exist (`DataContractNotPresentError`, 10400; `InvalidDocumentTypeError`, 10406).
+- The value says how the identity's keys of that purpose, bound to the type, are kept:
+ - `0`, unique: the identity holds at most one, and it can never be replaced. A second is refused (`IdentityPublicKeyAlreadyExistsForUniqueContractBoundsError`, 40211).
+ - `1`, multiple: the identity may hold any number.
+ - `2`, multiple with a pointer to the latest: any number, and the platform keeps a pointer to the most recently added one, so a client reads the current key in one step.
+- Clients read keys bound to a contract or a document type with the `getIdentitiesContractKeys` query, which takes the identities, the contract, an optional document type name and the purposes.
+- Consensus reads these keywords only when keys are added. Nothing requires the writer of a document to hold such a key, and nothing checks which key encrypted a property. See [encryptedFor](encrypted-for.md) for what consensus does check about encrypted values.
+- Encryption and decryption keys are `MEDIUM` keys. See [Identity Keys](../sdk/identity-keys.md#what-security-level-controls).
+- Keys bound to the whole contract, rather than one document type, are governed by the contract config keys of the same names. See [Contract-Level Keys and config](contract-config.md).
+- Before protocol version 12, a decryption key bound to a document type was checked against `requiresIdentityEncryptionBoundedKey` by mistake. From 12 each purpose reads its own keyword.
+- From protocol version 14 an authentication key may also be bound to a contract or a document type, to limit what it may sign. That needs neither keyword. See [Contract Bounds](../sdk/identity-keys.md#contract-bounds).
+
+## See also
+
+- [Identity Keys Deep Dive](../sdk/identity-keys.md), for purposes, security levels and contract bounds
+- [encryptedFor](encrypted-for.md), for declaring how an encrypted property was made
+- [Creation, Transfers and Trading](ownership-and-trading.md), for the actions a key signs
+- [Contract-Level Keys and config](contract-config.md), for the contract-wide key requirements
diff --git a/book/src/contract-keywords/system-properties.md b/book/src/contract-keywords/system-properties.md
new file mode 100644
index 00000000000..731ff368cbf
--- /dev/null
+++ b/book/src/contract-keywords/system-properties.md
@@ -0,0 +1,142 @@
+# System Properties
+
+Every document carries a few values the platform manages rather than the writer: its id, its owner, and, on some types, its revision, its creator and the times it was created, updated and transferred. Their names start with `$`. A document type does not declare them in `properties`. It names them where it wants to use them: in `required`, to have a timestamp recorded, in `indices`, to query by them, and in the keywords that accept one, such as a reference's `propertyAgreement`.
+
+| Property | Holds | Recorded |
+|---|---|---|
+| [`$id`](#id) | the document's id | always |
+| [`$ownerId`](#ownerid) | the identity that owns the document now | always |
+| [`$revision`](#revision) | how many times the document has changed, plus one | on types whose documents can be replaced, transferred or sold |
+| [`$createdAt`, `$updatedAt`, `$transferredAt`](#timestamps) | block times of the creation, last update and last transfer | when listed in `required` |
+| [`$createdAtBlockHeight` and the other heights](#block-heights) | Platform and Core block heights of the same events | when listed in `required` |
+| [`$creatorId`](#creatorid) | the identity that created the document | on types whose documents can be transferred or sold |
+
+## Example
+
+```json
+"listing": {
+ "type": "object",
+ "documentsMutable": true,
+ "transferable": 1,
+ "tradeMode": 1,
+ "indices": [
+ { "name": "byCreator", "properties": [{ "$creatorId": "asc" }, { "$createdAt": "asc" }] },
+ { "name": "byOwner", "properties": [{ "$ownerId": "asc" }, { "$updatedAt": "asc" }] }
+ ],
+ "properties": {
+ "title": { "type": "string", "minLength": 1, "maxLength": 100, "position": 0 }
+ },
+ "required": ["$createdAt", "$updatedAt", "$transferredAt", "title"],
+ "additionalProperties": false
+}
+```
+
+Every listing records when it was created, last updated and last transferred, because `required` lists the three times. Listings can be replaced, transferred and sold, so each one also carries a revision and the id of its creator, and the two indexes find them by who made them and by who holds them now.
+
+## `$id`
+
+| | |
+|---|---|
+| **Where** | Every document |
+| **Value** | An identifier: 32 bytes |
+| **Recorded** | Always |
+| **Since** | protocol version 1 |
+| **Errors** | `InvalidDocumentTransitionIdError` (10405): a create whose id is not the one derived for it. `SystemPropertyIndexAlreadyPresentError` (10208): an index that names `$id`. |
+
+A document's id is derived from the contract id, the owner's id, the document type's name, entropy chosen by the client and, from protocol version 14, the identity contract nonce of the create transition. Consensus derives it again for every create and refuses a transition that carries another. The id never changes. See [Document ID Generation](../data-model/documents.md#document-id-generation).
+
+Documents are already stored by id, so an index may not name `$id`. A reference's `propertyAgreement` may name it on the referenced side (see [References](refers-to.md)).
+
+## `$ownerId`
+
+| | |
+|---|---|
+| **Where** | Every document |
+| **Value** | An identifier: the id of an identity |
+| **Recorded** | Always |
+| **Since** | protocol version 1 |
+| **Errors** | `DocumentOwnerIdMismatchError` (40102): a replace, transfer, price update or delete signed by an identity that does not own the document |
+
+The owner is the identity that created the document, until a transfer or a purchase hands it to another. Only the owner may replace, transfer, reprice or delete it with a document transition. A document can also leave by other paths, a moderators' deletion or an expired [`ttl`](ttl.md); see [Deletion](deletion.md).
+
+`$ownerId` may be indexed, and several keywords read it, where it usually stands for the writer of the document:
+
+- [`distinctFrom`](distinct-from.md): `"$ownerId"` makes a property differ from the owner.
+- [`propertyConstraints`](property-constraints.md): a rule may compare an identifier property with `$ownerId`.
+- [References](refers-to.md): a `propertyAgreement` pair, a lookup key and `identityProperty` may name it, and [`ownerRefersTo`](owner-refers-to.md) checks the owner itself.
+- [`encryptedFor`](encrypted-for.md): `"recipient": "$ownerId"` marks a message the writer encrypts to themself.
+
+## `$revision`
+
+| | |
+|---|---|
+| **Where** | Documents of a type whose documents can be replaced (`documentsMutable`, true by default), transferred (`transferable: 1`) or sold (`tradeMode: 1`) |
+| **Value** | An integer, from 1 |
+| **Recorded** | On those types; absent on every other |
+| **Since** | protocol version 1 |
+| **Errors** | `InvalidDocumentRevisionError` (40106): a transition whose revision is not the stored one plus one |
+
+A new document has revision 1. Every replace, transfer, price update and purchase raises it by one, and the transition must state the new revision: the stored revision plus one. A transition built against an older copy of the document is refused, rather than silently overwriting a newer one. A type whose documents can never change after creation carries no revision. `$revision` is not one of the system properties an index may name.
+
+## Timestamps
+
+| | |
+|---|---|
+| **Properties** | `$createdAt`, `$updatedAt`, `$transferredAt` |
+| **Value** | A block time, in milliseconds since the Unix epoch |
+| **Recorded** | Only when listed in the document type's `required` |
+| **Since** | protocol version 1 |
+| **On update** | The set is fixed: a contract update may not add one to `required` or remove one (`DataContractInvalidRequiredFieldsUpdateError`, 10276) |
+
+The platform sets these from the block that processes the transition; the writer never supplies them. A timestamp that is not in `required` is never recorded, and documents of the type do not have it.
+
+| Event | Sets |
+|---|---|
+| Create | every listed timestamp, so `$updatedAt` and `$transferredAt` start at the creation time |
+| Replace | `$updatedAt` |
+| Price update | `$updatedAt` |
+| Transfer | `$transferredAt` |
+| Purchase | `$transferredAt` |
+
+Timestamps may be indexed. Some keywords need one in `required`, since they read it:
+
+- [`ttl`](ttl.md) counts from `$createdAt`.
+- `canBeDeletedByModeratorsFor` counts from `$updatedAt`, or from `$createdAt` on a type whose documents cannot be replaced (see [Deletion](deletion.md)).
+- A [time-range index](time-range.md) needs the timestamp it buckets.
+
+## Block heights
+
+| | |
+|---|---|
+| **Properties** | `$createdAtBlockHeight`, `$updatedAtBlockHeight`, `$transferredAtBlockHeight`, `$createdAtCoreBlockHeight`, `$updatedAtCoreBlockHeight`, `$transferredAtCoreBlockHeight` |
+| **Value** | The `BlockHeight` forms: the Platform block height. The `CoreBlockHeight` forms: the Core chain height recorded with that block. |
+| **Recorded** | Only when listed in the document type's `required` |
+| **Since** | protocol version 1 |
+| **On update** | The set is fixed, as for the timestamps (10276) |
+
+The same three events as the timestamps, measured in blocks instead of time. Each is set on the same events as the timestamp of its name, and may be indexed.
+
+## `$creatorId`
+
+| | |
+|---|---|
+| **Where** | Documents of a type that sets `transferable: 1` or `tradeMode: 1`, in a contract of format 1 whose config is version 1 or later |
+| **Value** | An identifier: the id of the identity that created the document |
+| **Recorded** | On those types, from protocol version 10 |
+| **Since** | protocol version 10 |
+| **Errors** | `UndefinedIndexPropertyError` (10209): an index naming `$creatorId` on a type that does not record it |
+
+On a type whose documents can change hands, `$ownerId` follows the document while `$creatorId` stays with the identity that created it: a transfer or a purchase never changes it. On a type whose documents never change hands the creator is always the owner, and no `$creatorId` is recorded. Neither is it on a contract of format 0 or with a config of version 0, whatever its types allow.
+
+`$creatorId` is not written in `required` or `properties`. It may be named:
+
+- in an index;
+- on the referenced side of a reference's `propertyAgreement`, and as a key reference's `identityProperty`;
+- by [`creatorRefersTo`](owner-refers-to.md), which checks the creator. A type that does not record creators may not declare it (`InvalidContractStructure`, 10231).
+
+## See also
+
+- [What Lives Inside a Document](../data-model/documents.md#what-lives-inside-a-document), for the fields of a document
+- [Document Shape](document-shape.md#required), for the `required` list that records timestamps
+- [Document Serialization](../serialization/document-serialization.md#high-level-structure), for where each system property sits in the stored bytes
+- [Creation, Transfers and Trading](ownership-and-trading.md), for the flags that decide `$revision` and `$creatorId`
diff --git a/book/src/contract-keywords/time-range.md b/book/src/contract-keywords/time-range.md
new file mode 100644
index 00000000000..4f7accf3a77
--- /dev/null
+++ b/book/src/contract-keywords/time-range.md
@@ -0,0 +1,96 @@
+# Time-Range Indexes
+
+`timeRange` groups an index's documents into time windows by one of their timestamps, so that "the most used hashtags this hour" or "posts per day" is a question about one window rather than a scan over time. Each window is `range` seconds long and a new one starts every `step` seconds; when windows overlap, a document belongs to each window that contains its timestamp. Combined with the count and ranking keywords, a time-range index serves trending lists and leaderboards per window, with proofs. It costs one set of index entries per window a document falls in, and a `ttl` lets old windows expire so that this data does not stay in state forever.
+
+| | |
+|---|---|
+| **Where** | index; buckets the index's first property |
+| **Value** | object: `on`, `range`, `step` (required), `phase`, `ttl` |
+| **Default** | absent: the timestamp is indexed as it is |
+| **Since** | protocol version 14 (`ttl` included) |
+| **On update** | Fixed, like every index (`DataContractInvalidIndexDefinitionUpdateError`, 10217) |
+| **Errors** | `DuplicateUniqueIndexError` (40105) on a unique time-range index |
+
+## Example
+
+```json
+"post": {
+ "type": "object",
+ "indices": [
+ {
+ "name": "hourlyByHashtag",
+ "properties": [{ "$createdAt": "asc" }, { "hashtag": "asc" }],
+ "timeRange": { "on": "$createdAt", "range": 3600, "step": 900, "ttl": 86400 },
+ "countable": "countable",
+ "rangeCountable": true
+ },
+ {
+ "name": "onePerDay",
+ "properties": [{ "$createdAt": "asc" }, { "$ownerId": "asc" }],
+ "unique": true,
+ "timeRange": { "on": "$createdAt", "range": 86400, "step": 86400 }
+ }
+ ],
+ "properties": {
+ "hashtag": { "type": "string", "minLength": 1, "maxLength": 63, "position": 0 },
+ "text": { "type": "string", "maxLength": 280, "position": 1 }
+ },
+ "required": ["$createdAt", "hashtag", "text"],
+ "additionalProperties": false
+}
+```
+
+`hourlyByHashtag` keeps one-hour windows starting every 15 minutes, so each post is counted in four windows, and the counts per hashtag in the window that covers the last hour are one query. Its entries expire a day after their window starts. `onePerDay` has non-overlapping daily windows and is unique, so an identity may post once per day: a second post in the same day is refused with `DuplicateUniqueIndexError` (40105).
+
+## The keys
+
+| Key | Value | What it does |
+|---|---|---|
+| `on` | `"$createdAt"`, `"$updatedAt"` or `"$transferredAt"`, required | The timestamp to bucket. It must be the index's first property and be listed in the type's `required`, since a timestamp that is not required is never recorded. |
+| `range` | integer seconds, at least 1, required | The length of each window. An exact multiple of `step`. |
+| `step` | integer seconds, at least 1, required | The time between the starts of two windows. |
+| `phase` | integer seconds, default `0` | Moves the window boundaries: windows start at `phase + k * step` from the Unix epoch. Less than `step` and less than one year (31536000). Daily windows cut at 06:00 UTC take `"phase": 21600`. |
+| `ttl` | integer seconds, at least 1 | How long the index's entries live after their window starts. At least `range` and at most 604800 (one week) at protocol version 14. Absent, entries live forever. |
+
+## How it works
+
+**Buckets.** For each document, the index stores the start of every window that contains the document's timestamp, in milliseconds, in place of the timestamp itself. With `range` equal to `step` the windows do not overlap and a document is in exactly one; otherwise it is in `range / step` of them, the index's **overlap factor**, and costs that many sets of entries. The overlap factor is at most 24 at protocol version 14. Windows are declared in seconds because block time, which the timestamps come from, moves in steps of about five seconds.
+
+A source that changes, `$updatedAt` or `$transferredAt`, moves the document into the current windows each time it changes. Several indexes may bucket the same timestamp with different grids; each grid is stored apart from the others.
+
+**Queries.** A query selects one window of the index with an `IN_TIME_RANGE` clause on the bucketed timestamp:
+
+- `newest`: the window that started most recently, which covers the latest slice of time (up to one `step`).
+- `oldest`: the oldest window still open, which covers nearly a full `range` of history: the one for "trending over the last hour".
+- `byStart`: the window starting at a given millisecond time on the grid, which reaches past windows.
+
+`newest` and `oldest` are resolved against block time on the server and against the signed time of the response when verified. When the timestamp is bucketed by several grids, the query names the grid. A query may select at most one window. A window with no documents, including one that has not started, is a proved empty answer. Counts, sums and rankings declared on the index are then per window: a ranking below the bucketed timestamp is one leaderboard per window.
+
+**Uniqueness.** On a unique time-range index, two documents conflict when they fall in the same window and agree on the index's other properties. That is only meaningful when each document is in one window and cannot move, so uniqueness needs non-overlapping windows over `$createdAt`.
+
+**Expiry (`ttl`).** An entry can be queried until `ttl` seconds after its window's start; a query for an expired window is refused, so every window a query can reach is complete. The expired entries are removed lazily: each later write into the index removes some of the oldest expired entries, within a bounded budget, at no charge to the writer. An index that stops receiving writes keeps its expired entries.
+
+Everything written under an index with a `ttl` is billed as processing, at an ephemeral-bytes rate, instead of as storage, and nothing is refunded when it is removed. The rate prices a lifetime of at most a week, which is why `ttl` is capped there.
+
+A `ttl` removes entries from this index only. The documents stay, and so do their entries in the type's other indexes. To delete the documents themselves after a time, use the document type's [`ttl`](ttl.md).
+
+## Rules at registration
+
+- `on` names `$createdAt`, `$updatedAt` or `$transferredAt`, which is the index's first property and is listed in `required`. A user property cannot be bucketed.
+- `range` and `step` are at least 1, `range` is an exact multiple of `step`, and `range / step` is at most 24.
+- `phase` is less than `step` and less than 31536000.
+- `ttl`, when present, is at least `range` and at most 604800. Two indexes that bucket the same timestamp with the same `range`, `step` and `phase` share their storage and must declare the same `ttl`, or none.
+- A unique time-range index has `range` equal to `step` and `on` equal to `$createdAt`.
+- The index is not contested, does not set `nullSearchable: false`, and is not `preallocated`.
+- A ranking sits below the bucketed timestamp: a single-property time-range index cannot be ranked, and `rankedCountable.at` cannot name the timestamp. See [Ranked Indexes](ranked.md).
+- A `refersTo` lookup cannot resolve through a time-range index. See [Lookups](refers-to-lookup.md).
+- On an [index-only type](index-only.md), only `$createdAt` can be bucketed, and a bucketed index cannot serve as the type's proof index.
+
+A broken rule is refused as `InvalidContractStructure` (10231), or by the meta-schema as `JsonSchemaError` (10101). Before protocol version 14 the keyword is unknown and refused.
+
+## See also
+
+- [Time-Range Index TTL](../drive/time-range-ttl.md) for how expired windows are drained, the fee rate and the query gate.
+- [Counts, Sums and Averages](aggregates.md) and [Ranked Indexes](ranked.md) for totals and leaderboards per window.
+- [Time To Live (ttl)](ttl.md) for expiring whole documents.
+- [Indexes](indexes.md) for the index keywords every index uses.
diff --git a/book/src/contract-keywords/token-cost.md b/book/src/contract-keywords/token-cost.md
new file mode 100644
index 00000000000..d71dcdc19d7
--- /dev/null
+++ b/book/src/contract-keywords/token-cost.md
@@ -0,0 +1,126 @@
+# Token Costs (tokenCost)
+
+`tokenCost` makes an action on a document cost tokens: creating a card costs 10 gems, deleting one costs 1. The tokens are taken from whoever signs the transition, and either go to the contract owner or are burned. Reach for it when an app has its own token and wants documents to be paid for with it, or wants to hand out tokens that let users act for free, with the contract owner paying the gas.
+
+| | |
+|---|---|
+| **Where** | Document type |
+| **Value** | An object keyed by action: `create`, `replace`, `delete`, `transfer`, `update_price`, `purchase`. Each value is a cost object with the keys below. Actions left out cost no tokens |
+| **Default** | Absent: no action costs tokens |
+| **Since** | protocol version 9. `gasFeesPaidBy` is accepted from 9 and acted on from 14; `optional` is 14 |
+| **On update** | Fixed: a cost may not be added, changed or removed on an existing document type (`DocumentTypeUpdateError`, 40212) |
+| **Errors** | On a document transition: `RequiredTokenPaymentInfoNotSetError` (40115), `IdentityHasNotAgreedToPayRequiredTokenAmountError` (40116), `IdentityTryingToPayWithWrongTokenError` (40117), `IdentityTokenAccountFrozenError` (40702), `IdentityDoesNotHaveEnoughTokenBalanceError` (40700), `GasFeesPaidByNotAllowedError` (40129), `InconsistentGasFeesPaidByInBatchError` (40130), `GasSponsorInsufficientBalanceError` (40222). At registration: `InvalidTokenPositionError` (10451), `RedundantDocumentPaidForByTokenWithContractId` (10275), `TokenPaymentByBurningOnlyAllowedOnInternalTokenError` (10261), `DataContractNotFoundError` (40008), `InvalidTokenPositionStateError` (40009) |
+
+The keys of each cost object:
+
+| Key | Value | Default | Meaning |
+|---|---|---|---|
+| `tokenPosition` | integer, 0 to 65535 | required | Which token is charged: its position in this contract's `tokens`, or in the contract `contractId` names |
+| `amount` | integer, 1 to 281474976710655 | required | How many tokens the action costs |
+| `contractId` | identifier (32 bytes) | this contract | The contract whose token is charged, when it is not this one |
+| `effect` | `0` transfer to the contract owner, `1` burn | `0` | What happens to the tokens paid |
+| `gasFeesPaidBy` | `0` document owner, `1` contract owner, `2` prefer contract owner | `0` | Who the contract owner offers to have pay the gas of this action (acted on from protocol version 14) |
+| `optional` | boolean | `false` | `true` lets a transition skip the token and pay the gas in credits instead (protocol version 14) |
+
+## Example
+
+```json
+"card": {
+ "type": "object",
+ "documentsMutable": true,
+ "canBeDeleted": true,
+ "transferable": 1,
+ "tradeMode": 1,
+ "tokenCost": {
+ "create": { "tokenPosition": 0, "amount": 10, "gasFeesPaidBy": 2, "optional": true },
+ "replace": { "tokenPosition": 1, "amount": 1 },
+ "delete": { "tokenPosition": 1, "amount": 1, "effect": 1 }
+ },
+ "properties": {
+ "name": { "type": "string", "minLength": 1, "maxLength": 63, "position": 0 },
+ "attack": { "type": "integer", "minimum": 0, "maximum": 100, "position": 1 }
+ },
+ "required": ["name", "attack"],
+ "additionalProperties": false
+}
+```
+
+The contract has two tokens. Creating a card costs 10 of token 0, which go to the contract owner. A player who pays with the token and asks for it has the gas paid by the contract owner, when the owner's balance covers it; a player may also skip the token and pay the gas in credits. Replacing a card costs 1 of token 1, also to the owner. Deleting one burns 1 of token 1. Transfers, price updates and purchases cost no tokens.
+
+## Paying: `$tokenPaymentInfo`
+
+A document transition on an action with a token cost carries a `$tokenPaymentInfo` in its base, saying which token the signer agrees to pay with, how much at most, and who they ask to pay the gas:
+
+```json
+"$tokenPaymentInfo": {
+ "$formatVersion": "0",
+ "tokenContractPosition": 0,
+ "maximumTokenCost": 10,
+ "gasFeesPaidBy": "PreferContractOwner"
+}
+```
+
+| Field | Meaning |
+|---|---|
+| `paymentTokenContractId` | The contract of the token paid with. Leave it out for a token of the document's own contract: it must match the cost's `contractId` exactly, and a cost on the contract's own token has none |
+| `tokenContractPosition` | The token's position in that contract |
+| `minimumTokenCost`, `maximumTokenCost` | Optional bounds on the amount the signer agrees to pay; a cost outside them refuses the transition |
+| `gasFeesPaidBy` | `"DocumentOwner"`, `"ContractOwner"` or `"PreferContractOwner"`: who the signer asks to pay the gas (see [Who pays the gas](#who-pays-the-gas-gasfeespaidby)) |
+
+When the transition is processed:
+
+1. A required cost with no `$tokenPaymentInfo` is refused (`RequiredTokenPaymentInfoNotSetError`, 40115).
+2. A payment info naming another token than the cost is refused (`IdentityTryingToPayWithWrongTokenError`, 40117).
+3. A cost outside the signer's `minimumTokenCost` and `maximumTokenCost` is refused (`IdentityHasNotAgreedToPayRequiredTokenAmountError`, 40116).
+4. The signer's gas request must be one the cost offers, and the whole batch must name one payer (40129, 40130).
+5. Against state: a signer whose account for the token is frozen is refused (`IdentityTokenAccountFrozenError`, 40702), and so is one whose balance is below `amount` (`IdentityDoesNotHaveEnoughTokenBalanceError`, 40700).
+
+The signer pays: the creator for a create, the owner for a replace, delete, transfer or price update, and the buyer for a purchase.
+
+## `effect`: transfer or burn
+
+- `0`, the default, moves the tokens from the signer to the owner of the contract that holds the document type. When the contract owner performs the action themselves nothing moves, though their balance is still checked.
+- `1` burns the tokens from the signer's balance, lowering the token's supply. Only a token of the contract's own can be burned (`TokenPaymentByBurningOnlyAllowedOnInternalTokenError`, 10261, at registration).
+
+## Tokens of another contract: `contractId`
+
+A document type may charge a token another contract defines, for example a shared currency. `contractId` names that contract and `tokenPosition` the token in it. The effect must then be `0`: the tokens go to the owner of the contract holding the document type, not to the token's issuer. At registration the named contract must exist (`DataContractNotFoundError`, 40008) and have a token at that position (`InvalidTokenPositionStateError`, 40009), and it must not be the contract itself: leave `contractId` out for the contract's own tokens (`RedundantDocumentPaidForByTokenWithContractId`, 10275).
+
+## Who pays the gas: `gasFeesPaidBy`
+
+From protocol version 14 the contract owner can pay the gas (the storage and processing fees) of an action paid with a token. The cost states what the contract owner offers; the transition's `$tokenPaymentInfo.gasFeesPaidBy` states what the signer asks for. The two resolve like this:
+
+| Cost offers / signer asks | `DocumentOwner` | `PreferContractOwner` | `ContractOwner` |
+|---|---|---|---|
+| `0` document owner | signer pays | signer pays | refused (40129) |
+| `2` prefer contract owner | signer pays | contract owner, if their balance covers it | refused (40129) |
+| `1` contract owner | signer pays | contract owner, if their balance covers it | contract owner |
+
+- A signer can always pay for themself, can always state a preference, and can insist on the contract owner only where the cost offers `1`. A request the cost does not cover is refused (`GasFeesPaidByNotAllowedError`, 40129). An action without a token cost offers `0`, and a transition without `$tokenPaymentInfo` asks for `DocumentOwner`.
+- A batch has one payer. Transitions of one batch that resolve to different payers are refused (`InconsistentGasFeesPaidByInBatchError`, 40130); a token transition in the batch is never sponsored, so it counts as the signer paying.
+- When the contract owner's balance does not cover the gas (and any action fees they would owe), a batch that insisted is refused and charged nothing (`GasSponsorInsufficientBalanceError`, 40222), and a batch that only preferred falls back to the signer.
+- A transition that fails validation is never sponsored: its signer pays for the work that ran.
+- Storage refunds still go to the document's owner, whoever paid for the storage. Each token the contract owner hands out is therefore worth up to the storage fee of the largest document the type allows, so a type that offers to pay should bound its documents' size (`maxLength`, `maxItems`, [maxBytes](max-bytes.md)) and price the action to match.
+- A sponsor also pays any [action fee](action-fees.md) the action charges.
+
+Before protocol version 14 the key was accepted and stored but not acted on: the signer always paid.
+
+## Optional costs
+
+With `optional: true` a transition may leave `$tokenPaymentInfo` out. It then pays no token, its signer pays the gas in credits as on an action without a token cost, and no sponsorship applies. With `$tokenPaymentInfo` present the token is charged exactly as for a required cost, sponsorship included, and an insufficient token balance is a rejection, never a fallback to credits: the client chooses between token and credits before signing.
+
+Together with `gasFeesPaidBy` this gives a "free usage" pattern: an app hands out tokens, users act for free while their tokens last, and keep going on credits after.
+
+## Rules at registration
+
+- The meta-schema checks the shape (`JsonSchemaError`, 10101): only the six action keys; `tokenPosition` and `amount` required in each cost; values in the ranges above; no other key. Before protocol version 14 it also refuses `optional`.
+- Without `contractId`, `tokenPosition` must be a token of this contract (`InvalidTokenPositionError`, 10451).
+- With `contractId`: not this contract's own id (10275), no burn (10261), and a contract that exists with a token at that position (40008, 40009).
+
+## See also
+
+- [Gas paid by the contract owner](../fees/overview.md#gas-paid-by-the-contract-owner) and [Optional token costs](../fees/overview.md#optional-token-costs), the deep dive
+- [Action Fees (actionFees)](action-fees.md), fixed credit fees on the same six actions
+- [Creation, Transfers and Trading](ownership-and-trading.md), for the actions a type allows
+- [Contract-Level Keys and config](contract-config.md), for the contract's `tokens`
+- [Contract Keywords overview](../contract-keywords.md), for the conventions of these tables
diff --git a/book/src/contract-keywords/transient.md b/book/src/contract-keywords/transient.md
new file mode 100644
index 00000000000..c4f1fbb6806
--- /dev/null
+++ b/book/src/contract-keywords/transient.md
@@ -0,0 +1,72 @@
+# transient
+
+`transient` lists properties that a transition must carry but a stored document never holds. The value is validated with the rest of the document, can be read by the checks that run on the transition, and is then dropped. Reach for it when a write has to prove something with a value that should not be kept, such as a secret salt.
+
+| | |
+|---|---|
+| **Where** | The top of a document type |
+| **Value** | An array of names of top-level properties |
+| **Default** | Absent: every property is stored |
+| **Since** | protocol version 1. A replace drops the values from protocol version 14. |
+| **On update** | Fixed (`IncompatibleDocumentTypeSchemaError`, 10246). The list is compared as a set, so reordering or repeating a name is no change. |
+| **Errors** | `InvalidContractStructure` (10231) at registration |
+
+## Example
+
+The DPNS `domain` type, trimmed to two properties:
+
+```json
+"domain": {
+ "type": "object",
+ "documentsMutable": false,
+ "properties": {
+ "label": { "type": "string", "minLength": 3, "maxLength": 63, "position": 0 },
+ "preorderSalt": {
+ "type": "array",
+ "byteArray": true,
+ "minItems": 32,
+ "maxItems": 32,
+ "description": "Salt used in the preorder document",
+ "position": 1
+ }
+ },
+ "required": ["label", "preorderSalt"],
+ "transient": ["preorderSalt"],
+ "additionalProperties": false
+}
+```
+
+Registering a name takes two steps. A `preorder` document first commits to a hash of the salt and the name, and the `domain` document then reveals both. Every domain create must carry its 32-byte `preorderSalt`, and DPNS checks it against the preorder. Once the domain is created the salt has done its job, and the stored domain does not hold it.
+
+## How it works
+
+- **Checked like any other value.** A transient value is on the transition when the document is validated, so the schema holds it to its rules: a required transient property must be present, and its bounds apply. `maxBytes` and `distinctFrom` apply to it too, and so do checks such as the DPNS one above.
+- **Dropped before storing.** After validation, a create drops the transient values before the document is written. From protocol version 14 a replace drops them the same way. Before 14 a replace stored the values it carried, so a replaced document could hold values its create had dropped.
+- **Dropped by top-level name.** Only top-level properties can be transient. When a transient property is an object, the whole object is dropped with everything in it.
+- **Never stored, never found.** No stored document holds a transient value, so a query cannot find one and a later reader cannot see it.
+- **Sent again on replace.** A replace is validated as a whole document, so a required transient property must be carried by every replace, not only the create.
+- **Storage.** A transient property is stored as optional, with a presence byte, even when it is required: in a stored document it is always absent.
+
+## Rules at registration
+
+From protocol version 14, a document type is refused (`InvalidContractStructure`, 10231) when:
+
+- an entry does not name a top-level property of the type. A nested path, a system property or an undeclared name is refused; to drop a nested value, list the object around it;
+- an index reads a transient property, or a property inside a transient object. Every stored document would lack the value, so the index could find nothing and a unique index would enforce nothing;
+- a reference reads one where the value would have to be stored: a `refersTo` lookup may not read one on either side, a `propertyAgreement` may not name one on its referenced side, a key reference may not store its key id with a transient identity, and a `listElement` reference may not find its list's document through one. The referring side of a `propertyAgreement` may be transient: it is checked on the transition;
+- `encryptedFor` names one as its recipient or key id;
+- `generatedFrom` sits on one or names one as a param;
+- `immutable` lists one. A transient property is always absent from the stored document, so every replace that carries it would count as changing it;
+- a `propertyConstraints` rule reads one;
+- the type is `indexOnly` and declares any transient property.
+
+Before protocol version 14 none of these was checked.
+
+On a contract update the list is fixed. It decides which values stored documents hold and how every property is encoded, so documents written under one list could not be read under another.
+
+## See also
+
+- [Transient Properties](../data-model/documents.md#transient-properties) in the Documents chapter
+- [Document Serialization](../serialization/document-serialization.md#user-defined-properties), for the presence byte a transient property always takes
+- [Document Shape](document-shape.md#required), for `required`
+- [References (refersTo)](refers-to.md), [encryptedFor](encrypted-for.md), [generatedFrom](generated-from.md), [Mutability](mutability.md), [propertyConstraints](property-constraints.md) and [Index-Only Types](index-only.md), whose rules refuse transient properties
diff --git a/book/src/contract-keywords/ttl.md b/book/src/contract-keywords/ttl.md
new file mode 100644
index 00000000000..a5b8aed97c8
--- /dev/null
+++ b/book/src/contract-keywords/ttl.md
@@ -0,0 +1,64 @@
+# Time To Live (ttl)
+
+`ttl` gives every document of a type a lifetime. The platform deletes each document once that many seconds have passed since it was created, whoever owns it and whatever the type says about who else may delete it. Use it for content that is meant to disappear: stories, invitations, offers, session records. Such documents are also cheaper: they pay for the time they occupy the state rather than for storage forever.
+
+This is the document type keyword. An index can carry a `ttl` of its own inside `timeRange`, which expires index entries and leaves the documents in place; see [Time-Range Indexes](time-range.md).
+
+| | |
+|---|---|
+| **Where** | document type |
+| **Value** | integer, seconds, 3600 (one hour) to 31536000 (one year) |
+| **Default** | absent: documents live until someone deletes them |
+| **Since** | protocol version 14 |
+| **On update** | Fixed (`DocumentTypeUpdateError`, 40212): an update may not add, remove or change it |
+| **Errors** | `DocumentExpiredError` (40140) for a replace, transfer, purchase or price update of a document past its expiry |
+
+## Example
+
+```json
+"story": {
+ "type": "object",
+ "ttl": 86400,
+ "canBeDeleted": true,
+ "properties": {
+ "caption": { "type": "string", "maxLength": 200, "position": 0 },
+ "mediaUrl": { "type": "string", "maxLength": 500, "position": 1 }
+ },
+ "required": ["$createdAt", "mediaUrl"],
+ "additionalProperties": false
+}
+```
+
+Each story is deleted one day (86,400 seconds) after its creation. Its author may delete it sooner. `$createdAt` must be in `required`, since the expiry is counted from it.
+
+## How it works
+
+- **When a document expires.** At its `$createdAt` plus `ttl` seconds. `$createdAt` is the block time of the create, so nothing the writer sends moves the expiry, and documents created in the same block expire together. A replace, transfer, purchase or price update never moves it either: a buyer of an expiring document buys what is left of its life.
+- **When it is deleted.** After each block's state transitions, the platform deletes expired documents, oldest first: at most 128 per block, and at most 1,024 in weight, where a document weighs 1 plus the index levels of its type (the values at protocol version 14). The rest wait for the next block. The deletion is an ordinary one: the document and every index entry go, and counts and sums are brought down.
+- **Between expiry and deletion.** A document past its expiry that the cleanup has not reached yet can still be queried and referenced, and it still holds its values in the type's unique indexes, so a create with the same unique value is refused as a duplicate until the cleanup has run. It can no longer be replaced, transferred, bought or repriced, and a moderator can no longer restore it: each is refused, paid, with `DocumentExpiredError` (40140). Its owner may still delete it where `canBeDeleted` allows.
+- **Earlier deletion.** The owner may delete a document before it expires when `canBeDeleted` allows it, and the contract's moderators when `canBeDeletedByModerators` does. `canBeDeleted: false` only stops the owner; the platform still deletes the document when it expires.
+- **What it costs.** The document is stored without storage flags and refunds nothing when it is deleted, by anyone. Instead of the price of permanent storage, each byte it writes pays a price for the time it will live: five tiers up to seven days, then a price per 9.125 days spanned. Creating it also prepays, as processing, the cost of its later deletion. A replace, transfer, purchase or price update pays for the bytes it adds at the price of the lifetime left. See [Fees](../data-model/document-ttl.md#fees).
+- **Proofs near the expiry.** A write accepted in the last block before a document expires proves the document present. A proof fetched after the next block's cleanup finds it gone. The one-hour minimum keeps a newly created document in the state well past the moment its writer fetches the proof of the create.
+
+## Rules at registration
+
+The refusals below are `InvalidContractStructure` (10231) unless a bullet says otherwise. They hold on every parse of the contract, except the bounds, which are checked when a contract is registered or updated.
+
+- `$createdAt` must be in `required`.
+- Refused together with `documentsKeepHistory: true` (Drive never deletes a document whose type keeps history), with `indexOnly: true` (there is no stored row to delete by id), and on a type with a contested index (a contested document waits in its vote poll, and could expire before it is stored).
+- At least `min_document_ttl_seconds` and at most `max_document_ttl_seconds` of `SystemLimits`: 3600 and 31536000 at protocol version 14. The meta-schema itself admits 1 to 4294967295, so a `ttl` of 0 is a `JsonSchemaError` (10101) and one outside the narrower bounds is 10231.
+- For references, a type with a `ttl` is deletable. A `permanentDocument` reference and a `listElement` reference may not point at it, a lookup included (`ReferencedDocumentTypeDeletableError`, 40122); a `deletableDocument` reference may. See [References](refers-to.md).
+
+Everything else combines with a `ttl`: mutable types, `transferable`, `tradeMode`, `canBeDeletedByModerators`, `creationRestrictionMode`, count, sum and ranked indexes, `timeRange` indexes with or without their own `ttl`, references declared on the type, action fees and token costs.
+
+### On update
+
+An update may not add, remove or change the `ttl` of an existing document type. Every stored document has the expiry it was written and paid with: adding one would leave stored documents that the cleanup cannot find, removing it would leave entries that delete documents the type says live forever, and changing it would move expiries away from what was paid for. A document type added by an update may declare a `ttl` freely.
+
+## See also
+
+- [Document Time To Live](../data-model/document-ttl.md), for the expirations tree, the fee tiers, the payout to epochs and the cleanup
+- [Deletion](deletion.md), for the other two ways a document is deleted
+- [Time-Range Indexes](time-range.md), for the index `ttl`
+- [History](history.md), for why a type that keeps history cannot expire
+- [System Properties](system-properties.md), for `$createdAt`
diff --git a/book/src/contract-keywords/typed-arrays.md b/book/src/contract-keywords/typed-arrays.md
new file mode 100644
index 00000000000..fba443417ce
--- /dev/null
+++ b/book/src/contract-keywords/typed-arrays.md
@@ -0,0 +1,107 @@
+# Typed Arrays
+
+A typed array is a list property: a document holds any number of values, up to a limit, all of one simple type. A post's tags, the scores of a match or the members of a team are typed arrays. The `items` keyword makes an array a typed array and gives the schema every element follows. Before protocol version 14 an array property had to be a byte array.
+
+| | |
+|---|---|
+| **Where** | A property of type `array`, at the top of a document type or inside an object, in place of `byteArray: true` |
+| **Value** | The schema of one element: an integer, a number, a string, a boolean, a byte array or an identifier |
+| **Since** | protocol version 14 |
+| **On update** | `items` may be neither added nor removed (`IncompatibleDocumentTypeSchemaError`, 10246). The keywords inside it follow their own update rules, and a change to how an element is stored is refused (`DocumentTypeUpdateError`, 40212). |
+| **Errors** | `JsonSchemaError` (10101); on elements, `DocumentPropertyMaxBytesExceededError` (10421), `DocumentPropertyNotDistinctError` (10419) and the reference errors of [refersTo](refers-to.md) |
+
+## Example
+
+```json
+"tags": {
+ "type": "array",
+ "minItems": 0,
+ "maxItems": 10,
+ "uniqueItems": true,
+ "items": { "type": "string", "minLength": 1, "maxLength": 32, "maxBytes": 64 },
+ "position": 3
+}
+```
+
+A post holds up to 10 tags, none repeated, each 1 to 32 characters and at most 64 bytes.
+
+The elements may also be identifiers that point at other documents. The moderation charters contract lists the reasons a moderation team may act on like this:
+
+```json
+"reasons": {
+ "type": "array",
+ "minItems": 0,
+ "maxItems": 64,
+ "uniqueItems": true,
+ "items": {
+ "type": "array",
+ "byteArray": true,
+ "minItems": 32,
+ "maxItems": 32,
+ "contentMediaType": "application/x.dash.dpp.identifier",
+ "refersTo": { "type": "permanentDocument", "documentType": "reason" }
+ },
+ "position": 2
+}
+```
+
+A charter names up to 64 distinct `reason` documents, and each one must exist when the charter is written.
+
+## How it works
+
+**On the array**, `minItems` and `maxItems` count elements, and `uniqueItems: true` refuses a document that repeats one. A list that is too long or too short, repeats an element, or holds an element of the wrong type is refused with `JsonSchemaError` (10101).
+
+**On the elements**, the `items` schema is checked for every element, as a property of that schema would be:
+
+- the JSON Schema keywords: an element's `enum`, bounds, length and `pattern` (`JsonSchemaError`, 10101);
+- [`maxBytes`](max-bytes.md) on string elements (`DocumentPropertyMaxBytesExceededError`, 10421), the error naming the element, such as `tags[2]`;
+- [`distinctFrom`](distinct-from.md) on identifier elements: every element must differ from the named property (`DocumentPropertyNotDistinctError`, 10419);
+- [`refersTo`](refers-to.md) on identifier elements: each element is checked as a single reference would be, in list order, and the first one that fails refuses the write with that reference's error, naming the element (`reasons[2]` for the third). An empty or absent list checks nothing.
+
+A replace re-checks the references of the elements the stored list did not hold, and every element when the reference's rules call for it. See [References on the Elements](../data-model/documents.md#references-on-the-elements) for the details.
+
+**Storage.** A typed array is stored inline in the document: a count of its elements, then each element encoded as a required property of the element's type would be. An identifier element takes 32 bytes, an integer element the width its bounds give it (see [Numbers](property-schemas.md#numbers)), a fixed-size byte array element its bytes, and a string element a length prefix and its bytes. Elements never carry a presence byte. The `reasons` list above is therefore one count byte and 32 bytes per reason. The 5120-byte limit on a single value applies to each element, not to the list.
+
+**When a document is read from JSON**, identifier and byte array elements are converted from their string form element by element, as a single identifier or byte array is.
+
+## Rules at registration
+
+The array:
+
+- declares `items` and `maxItems`, and not `byteArray`;
+- has a `maxItems` of at most 1024, and a `minItems` no higher than its `maxItems`;
+- carries no `contentMediaType`, `refersTo`, `distinctFrom` or `maxBytes` of its own: these go on the `items`.
+
+The element schema (`items`):
+
+- is written inline: a `$ref` is refused;
+- has a `type` of `integer`, `number`, `string`, `boolean` or `array`, and an `array` element must be a byte array (`byteArray: true`). Objects and lists of lists are refused;
+- takes these keywords: `type`, `minimum`, `maximum`, `exclusiveMinimum`, `exclusiveMaximum`, `multipleOf`, `minLength`, `maxLength`, `pattern`, `format`, `minItems`, `maxItems` (bytes, on a byte array element), `enum`, `byteArray`, `contentMediaType`, `maxBytes`, `distinctFrom`, `refersTo`, `$comment` and `description`. Anything else, `position`, `const`, `uniqueItems` and `examples` included, is refused. A one-value `enum` does what `const` would, and an update can still widen it;
+- may carry an `enum` only on a string, integer, number or boolean element, and every member must be a value of the element's type;
+- on an integer element, has an integer `minimum` and `maximum`, the minimum no higher than the maximum;
+- may carry `refersTo` only on an identifier element, and not with the `identityPublicKey` target: its key id is a single sibling property, which cannot pair with many elements.
+
+Where a typed array cannot be used:
+
+- in an index (`InvalidIndexPropertyTypeError`, 10206), or as an index-only type's terminal or entry payload: nothing is written per element;
+- on either side of a `propertyAgreement` in a reference;
+- as an operand of a [propertyConstraints](property-constraints.md) rule, other than in a `present` or `absent` test;
+- with [`encryptedFor`](encrypted-for.md), which only a byte array takes.
+
+References count against the limit of 256 per document at `maxItems` each: a list of up to 64 references counts as 64. An `immutable` property may not hold a list of `deletableDocument` references (see [Mutability](mutability.md)).
+
+A meta-schema violation is refused with `JsonSchemaError` (10101) and the other rules with `InvalidContractStructure` (10231).
+
+## On update
+
+- `items` may not be added or removed, so a byte array cannot become a typed array or the reverse (10246).
+- Inside `items`, each keyword follows its own update rule (see [Property Schemas](property-schemas.md)): for example `maxLength` and `maxBytes` may be raised, and `enum` may gain values. `refersTo` and `distinctFrom` are fixed.
+- A change to how an element is stored is refused (`DocumentTypeUpdateError`, 40212): an integer element whose width or sign would change (by its bounds or its `enum`), or a byte array element that would switch between fixed and variable size or change its fixed size.
+- On the array, `maxItems` may be raised, up to 1024 and within the reference limit, and `minItems` lowered. `uniqueItems` may be removed, not added.
+
+## See also
+
+- [Typed Arrays](../data-model/documents.md#typed-arrays) and [References on the Elements](../data-model/documents.md#references-on-the-elements) in the Documents chapter
+- [Property Schemas](property-schemas.md), for the keywords an element takes
+- [References (refersTo)](refers-to.md), [distinctFrom](distinct-from.md) and [maxBytes](max-bytes.md), which apply to every element
+- [Document Serialization](../serialization/document-serialization.md#value-encoding-by-type), for the stored form
diff --git a/book/src/contributing/coding-conventions.md b/book/src/contributing/coding-conventions.md
index 1767435c069..d564bab8f3b 100644
--- a/book/src/contributing/coding-conventions.md
+++ b/book/src/contributing/coding-conventions.md
@@ -84,7 +84,15 @@ dpp method whose own gate is `None` there), or the edit is a pure refactor with
identical output. "Probably inert" is not enough. If the argument takes more
than a sentence, or rests on a runtime check inside the shipped module, add a
generation instead. Inside a new generation the capability is a constant fact
-(`Index::try_from_value_map(map, true)`), not a runtime check.
+(`Index::try_from_value_map(map, true)`), not a runtime check. For new
+changes, the patterns that compare `protocol_version` on purpose are the
+protocol upgrade ladder, the `is_allowed` gate for new transition kinds, and
+chain-creation content (`create_genesis_state` v1 and the Drive helpers that
+build the initial state structure); the [Versioned
+Dispatch](../versioning/versioned-dispatch.md) chapter describes them. Older
+comparisons elsewhere (for example the pre-version-9 arithmetic in
+`DocumentPropertyType`) stay because history replays through them; they are not
+a pattern to copy.
Why: replay safety is structural when a shipped file stays byte-identical, and
becomes a proof the reviewer has to check the moment it does not. An in-place
@@ -257,7 +265,8 @@ Rules that fall out of the table:
- Preserve mempool coverage. `Batch` runs advanced structure with state during
`check_tx`, while full state validation is skipped there
(`validates_full_state_on_check_tx` defaults to `false`; masternode votes
- are the one transition that opts in, because they are unpaid). Moving a
+ are the one transition that opts in, because a block refuses them unpaid,
+ and they run advanced structure with state there too). Moving a
contract-dependent structural check into state validation would remove that
rejection from mempool admission.
- Validation outcomes are `ConsensusValidationResult`, returned as `Ok`. A
@@ -370,8 +379,9 @@ error for a user mistake; the block loop treats the two classes differently.
the crate root; do not extend it for convenience.
- **State access is snapshot-based.** `Platform.state` is an `ArcSwap`;
read with `load()` and pass `PlatformRef` or `PlatformStateRef` down. The
- only locks are the ABCI application's `transaction` and
- `block_execution_context`. In async code, a guard's scope, not a `drop()`
+ only locks are the ABCI application's `transaction`,
+ `block_execution_context` and `unsigned_withdrawal_txs_by_round`, taken in
+ that order. In async code, a guard's scope, not a `drop()`
call, is what clears `clippy::await_holding_lock`; wrap the guard in a block
that ends before the first `.await`.
diff --git a/book/src/data-model/contested-documents.md b/book/src/data-model/contested-documents.md
index 7ab62dfd3be..12e0a18cbcb 100644
--- a/book/src/data-model/contested-documents.md
+++ b/book/src/data-model/contested-documents.md
@@ -16,6 +16,39 @@ amount from that balance. Contenders may join for the **join window** (one week
the first document; the contest runs for the **poll duration** (two weeks on mainnet). The first
document's owner may not be joined by the same identity twice.
+From protocol version 14, a contest accepts at most 1,000 contenders
+(`max_contenders_per_contest`): a document that would add one more is refused, paid, with
+`DocumentContestMaximumContendersReachedError` (40141). The bound is what lets one block end a
+contest: its end tallies up to `maximum_contenders_to_consider` contenders (10,000 from version 14)
+and removes the entries of those it tallied, so a contest within the cap is tallied and cleaned up
+whole. Before 14 a contest accepted any number of contenders and its end tallied at most 100; a
+contest that grew past 10,000 contenders before 14 is tallied and cleaned up for its first 10,000
+only.
+
+From protocol version 14, the fund a contender pays doubles once the contest holds 250 contenders
+(`contested_document_contenders_before_fund_doubling`) and again for every 50 more
+(`contested_document_contenders_per_fund_doubling`), so a contest stops growing long before its
+cap:
+
+| Contenders the contest holds | DPNS fund to join | Moderation election fund to join |
+| --- | --- | --- |
+| 0 to 249 | 0.1 Dash | 0.5 Dash |
+| 250 to 299 | 0.2 Dash | 1 Dash |
+| 300 to 349 | 0.4 Dash | 2 Dash |
+| ... | doubles every 50 | doubles every 50 |
+| 700 to 749 | 102.4 Dash | 512 Dash |
+| ... | doubles every 50 | doubles every 50 |
+| 950 to 999 | 3,276.8 Dash | 16,384 Dash |
+
+Filling a DPNS contest to 1,000 contenders costs 327,695 Dash (100 at a flat 0.1 Dash). A contender's
+prefunded voting balance is the most it is willing to pay, and it must hold that much: it is
+charged the fund to join the contest it joins, and what it stated beyond that stays with it. One stating less, the first
+contender of a new contest included, is refused, paid, with `DocumentContestNotPaidForError`,
+which carries the fund it has to pay. The SDKs read how many contenders a contest holds and state
+that fund unless the caller names the most it will pay, which lets a join go through while others
+join ahead of it. Before 14 every contender stated exactly the contest's fund, however many had
+joined, and paid what it stated.
+
The index's `contested.resolution` says how the contest is decided.
From protocol version 14, an identifier property among the index values is written as an
@@ -51,8 +84,9 @@ An `electedCharter` contest of the moderation charters contract (protocol versio
the target contract id, is a **moderation election** and does not take the generic parameters:
- Its join window and vote window are the `joinWindow` and `voteWindow` of the target contract's
- elected moderation declaration (one day to four weeks each, one week by default), on every
- network. A single applicant wins when the join window closes; a second applicant moves the end
+ elected moderation declaration (at most four weeks each, one week by default; at least a day
+ on mainnet, while any other network takes 0), in place of the generic windows of the network.
+ A single applicant wins when the join window closes; a second applicant moves the end
to the join window plus the vote window. A late applicant is refused with
`DocumentContestNotJoinableError` naming the target's join window.
- Each application prefunds the votes with the moderation fund, 0.5 Dash
@@ -68,14 +102,18 @@ other contest, DPNS included, keeps the generic windows and fund.
## Ties
From protocol version 14, a tie among the top contenders goes to the **earliest** contender:
-creation time, then block height, then core block height, then document id. This holds for both
-resolutions; contests ending before version 14 awarded the latest contender.
+creation time, then block height, then core block height, then document id, among every tied
+contender. This holds for both resolutions; contests ending before version 14 awarded the latest
+contender.
## Storage
A contest's state lives under `votes / contested_resource / active_polls`, laid out like the
contested index it decides: the contenders' documents, one votes sum tree per contender, and the
-abstain and lock tallies. The masternodes' vote references live under
+abstain and lock tallies. From protocol version 14 the tree below a contest's last index value,
+which holds its contenders, stored result and tallies, is a count tree, so a join reads how many
+contenders there are in one fetch; a contest started before 14 keeps its plain tree, and a join
+counts its contenders by reading their keys. The masternodes' vote references live under
`votes / contested_resource / identity_votes`, and the end dates under `votes / end_date_queries`,
one tree per end date holding an entry for each contest ending then.
Once the contest ends, the winning document is awarded, the losers are removed, and the stored
diff --git a/book/src/data-model/contract-moderation.md b/book/src/data-model/contract-moderation.md
index 9298c7ac091..6c3d7b556d1 100644
--- a/book/src/data-model/contract-moderation.md
+++ b/book/src/data-model/contract-moderation.md
@@ -108,7 +108,7 @@ Deletions (`Delete` and `IndexOnlyDelete`) are never refused: a barred identity
### The Errors
-Basic, in their own band (10900-10949): `InvalidContractModerationConfigError` (10900), `ContractModerationSelfTargetError` (10901), `ContractModerationReasonTooLongError` (10903; 10902 is reserved), `InvalidContractModerationReasonDocumentsError` (10904). State, in their own sub-band: `ContractModerationNotEnabledError` (41100), `IdentityNotContractModeratorError` (41101), `ContractModerationTargetNotAllowedError` (41102), `ContractUserAlreadyBannedError` (41103), `ContractUserNotBannedError` (41104), `ContractUserNotSuspendedError` (41105), `ContractSuspensionNotInFutureError` (41106), `ContractUserBannedError` (41107), `ContractUserSuspendedError` (41108), `ContractModerationTargetNotFoundError` (41109), `ContractModeratorIdentityNotFoundError` (41110, from the contract create and update, not from the moderation transition), `ContractModerationCounterpartyBarredError` (41114, from the document gate; 41111 to 41113 are reserved), `ContractUserNotWarnedError` (41117), `ContractUserWarningLimitReachedError` (41118). A contract update that turns a list on or off is refused with the existing `DataContractConfigUpdateError` (40002). Elected moderation has its own band (41200-41299): `ContractModeratedDocumentTypeNotYetUsableError` (41200), `ContractModerationAbilityNotGrantedError` (41201), `ModerationCharterAddedModeratorLimitReachedError` (41202) and `ModerationReasonNotListedError` (41203). A discounted action fee the seated charter does not give is refused with `DocumentActionFeeModeratorsShareMismatchError` (40139), beside the other fee agreement errors.
+Basic, in their own band (10900-10949): `InvalidContractModerationConfigError` (10900), `ContractModerationSelfTargetError` (10901), `DocumentActionFeesWithoutModerationError` (10902, see Fee Pots and the Claim), `ContractModerationReasonTooLongError` (10903), `InvalidContractModerationReasonDocumentsError` (10904). State, in their own sub-band: `ContractModerationNotEnabledError` (41100), `IdentityNotContractModeratorError` (41101), `ContractModerationTargetNotAllowedError` (41102), `ContractUserAlreadyBannedError` (41103), `ContractUserNotBannedError` (41104), `ContractUserNotSuspendedError` (41105), `ContractSuspensionNotInFutureError` (41106), `ContractUserBannedError` (41107), `ContractUserSuspendedError` (41108), `ContractModerationTargetNotFoundError` (41109), `ContractModeratorIdentityNotFoundError` (41110, from the contract create and update, not from the moderation transition), `ContractModerationCounterpartyBarredError` (41114, from the document gate; 41111 to 41113 are reserved), `ContractUserNotWarnedError` (41117), `ContractUserWarningLimitReachedError` (41118). A contract update that turns a list on or off is refused with the existing `DataContractConfigUpdateError` (40002). Elected moderation has its own band (41200-41299): `ContractModeratedDocumentTypeNotYetUsableError` (41200), `ContractModerationAbilityNotGrantedError` (41201), `ModerationCharterAddedModeratorLimitReachedError` (41202) and `ModerationReasonNotListedError` (41203). A discounted action fee the seated charter does not give is refused with `DocumentActionFeeModeratorsShareMismatchError` (40139), beside the other fee agreement errors.
## Deleting Documents
@@ -144,7 +144,7 @@ ContractUserModerationAction::DeleteDocument {
It names no identity (`identity_id()` is `None`): whose document it is is only known once the document is read. The transform checks, in order and each refusal paid: the document type exists (10406), it carries the keyword (`DocumentTypeNotDeletableByModeratorsError`, 41115), the signer is the owner or a moderator (41101), the document exists (`DocumentNotFoundError`), its owner is neither the contract owner nor a moderator (41102, the rule that protects them from a ban protects what they wrote), and block time is within the type's window after the document's last modification (`$updatedAt`, else `$createdAt`), when the type sets one (41116). The document is read the way a document's own deletion reads it, billed the same. The action carries the contract, the document's owner and the block time, so Drive reads nothing again. Nothing the document type prices is charged: neither its deletion token cost nor its `actionFees` deletion fee, both of which are what a document's own owner pays for deleting it.
-Drive then runs `DocumentOperationType::DeleteDocumentByModerator`, the ordinary deletion (so every index and aggregate of the type stays right) without its `canBeDeleted` guard, which is the owner's rule and not the moderators', and writes a **removal record**:
+Drive then runs `DocumentOperationType::ForceDeleteDocument`, the ordinary deletion (so every index and aggregate of the type stays right) without its `canBeDeleted` guard, which is the owner's rule and not the moderators', and writes a **removal record**:
```rust
pub struct ContractDocumentRemoval {
@@ -275,7 +275,7 @@ A contract may hand the choice of its moderators to the network instead of keepi
```rust
pub struct ElectedModerators {
- pub join_window: u32, // seconds; 1 day to 4 weeks, 1 week by default
+ pub join_window: u32, // seconds; at most 4 weeks, at least 1 day on mainnet (0 elsewhere), 1 week by default
pub vote_window: u32, // the same
pub challenge_cool_down: Option, // Some = seat contestable, seconds, 2 weeks to 3 years; None = never contested again
pub election_delay: Option, // seconds after creation before the first charter; unbounded, none = at once
@@ -288,7 +288,7 @@ pub struct ElectedModerators {
The declaration lives in `packages/rs-dpp/src/data_contract/config/moderation/elected.rs`. On the wire it is the third `$type` of the moderators, flat: `{"$type": "elected", "seatContestable": true, "challengeCoolDown": 1209600, "moderatedDocumentTypes": {"post": ["ban", "deleteDocuments"]}, "interim": {"$type": "notYetUsable"}}`, or `"seatContestable": false` without a `challengeCoolDown`, with `joinWindow`, `voteWindow`, `electionDelay`, `maxAddedModerators` and `ownerProtected` optional. Its parts:
-- **The election parameters** are fixed once set (`SystemLimits`: `min_contract_moderation_election_window_seconds` and `max_contract_moderation_election_window_seconds` bound both windows). The join window is how long applicants may join an election once the first one applied, the vote window how long masternodes then vote. The **election delay** is the one parameter the contract sets freely: how many seconds after its creation the first charter may be filed against it, the notice the contract gives before its first election can be called. It is optional and unbounded; left out, the election may be called at once. Because the declaration is made at the contract's creation and never changes, the creation is the declaration's own time. The charter contract's `targetContractId` reads it through the `moderation: "electionOpen"` requirement below. **`maxAddedModerators`** says how many members the leader of a seated team may add after the election, each one an identity that asked to join the team's proposal: additions ever filed, so a removal or a resignation frees no slot. It is 0 when left out, a team then being exactly what was elected, and at most `SystemLimits::max_contract_moderation_added_moderators` (15).
+- **The election parameters** are fixed once set (`SystemLimits`: `max_contract_moderation_election_window_seconds` caps both windows at four weeks, and on mainnet `min_mainnet_contract_moderation_election_window_seconds` keeps them at least a day; every other network takes a window of 0, so a test election can be run through in a block or two). The join window is how long applicants may join an election once the first one applied, the vote window how long masternodes then vote. The **election delay** is the one parameter the contract sets freely: how many seconds after its creation the first charter may be filed against it, the notice the contract gives before its first election can be called. It is optional and unbounded; left out, the election may be called at once. Because the declaration is made at the contract's creation and never changes, the creation is the declaration's own time. The charter contract's `targetContractId` reads it through the `moderation: "electionOpen"` requirement below. **`maxAddedModerators`** says how many members the leader of a seated team may add after the election, each one an identity that asked to join the team's proposal: additions ever filed, so a removal or a resignation frees no slot. It is 0 when left out, a team then being exactly what was elected, and at most `SystemLimits::max_contract_moderation_added_moderators` (15).
- **The seat** is contestable or not, and the contract says which: `seatContestable` is required, with no default. A default of false would make every team permanent, leaving a contract nothing to do about a leader who lost its keys or went rogue, since the leader can not change and a challenge is the only remedy; a default of true would opt every contract into challenges without it asking. A contestable seat declares its **challenge cool-down**, how long a seated team is safe from a challenge after a seat change (`min_contract_moderation_challenge_cool_down_seconds` to `max_contract_moderation_challenge_cool_down_seconds`, two weeks to three years), and a seat that can not be contested declares none, since a cool-down means nothing there: a declaration missing the key, or `seatContestable: true` without the cool-down, or `false` with one, does not parse. In Rust the two are one field, `challenge_cool_down: Option` (`ElectedModerators::seat_contestable` is `is_some`), so a declaration can not disagree with itself however it is encoded. Nothing reads the seat yet: challenges come after protocol version 14, a challenge then being a new contest on the same `byTargetContract` index of the charter contract, allowed only when the target declares its seat contestable. The key is there now because the declaration is frozen at the contract's creation. Until challenges ship a seat is never contested again, whatever the key says.
- **The moderated set** is the document types the team moderates, each with the abilities the seated team holds on it: non-empty, each type a document type of the contract, each ability set non-empty and backed by the contract (`ban` needs the banlist, `suspend` the suspension list, `warn` the warning list, `deleteDocuments` the type itself flagged `canBeDeletedByModerators`, so deletions reach only flagged types, within their window). The charter of a team will say how those types are moderated, never which. The lists stay contract-wide: an ability on a type is what a team may do over the documents of that type. The set also bounds the interim block. A charter does not price the moderators part of an action: a type's own `actionFees.moderators` amount is the most a team may charge, a charter charges a share of it (the charter contract's business, not the declaration's), and the owner part stays what the type declares, immutable as before.
- **The interim** says who moderates until a team is seated. `ContractOwner` and `AppointedModerators(set)` are the merged kinds, with their authority, their limit and their existence check (41110 at create): they moderate, they are protected, and they are the team that claims the moderators pot, all of it until a charter is seated and none of it after (see The seated team below). `NotYetUsable` names nobody: nobody moderates, nobody claims the pot (it accumulates for the team to come, `ContractFeeClaimNotAllowedError` for everyone), and the moderated document types can not be used. A contract that never attracts a team keeps those types unusable for good; the other types work as on an unmoderated contract. `NoModeration` names nobody too, with the moderated types usable meanwhile: nobody moderates and nobody claims the pot, and every type works as on an unmoderated contract until a team is seated.
diff --git a/book/src/data-model/document-ttl.md b/book/src/data-model/document-ttl.md
new file mode 100644
index 00000000000..c7178797785
--- /dev/null
+++ b/book/src/data-model/document-ttl.md
@@ -0,0 +1,240 @@
+# Document Time To Live
+
+A document type may declare a **time to live**. Every document of such a type is deleted by
+the platform once its time to live has passed, whoever owns it and whatever the type says
+about who else may delete it. Its owner pays, when the document is written, for the time
+the document will occupy the state rather than for perpetual storage, and prepays its
+deletion. Available from protocol version 14.
+
+```json
+"note": {
+ "type": "object",
+ "ttl": 1209600,
+ "properties": { "text": { "type": "string", "maxLength": 280, "position": 0 } },
+ "required": ["$createdAt", "text"],
+ "additionalProperties": false
+}
+```
+
+Documents of `note` above are deleted two weeks (1,209,600 seconds) after their creation.
+
+This is a different feature from the `ttl` key of a `timeRange` index (see
+[Time-Range Index TTL](../drive/time-range-ttl.md)), which expires index entries and leaves
+the document in place.
+
+## Semantics
+
+- A document expires at its `$createdAt` plus `ttl` seconds. `$createdAt` is set from block
+ time when the document is created, so nothing the writer sends moves the expiry, and every
+ document of a type created in one block expires at the same time.
+- A replace, a transfer, a purchase or a price update never moves the expiry: `$createdAt`
+ never changes. A buyer of an expiring document buys what is left of its life.
+- After each block's state transitions the platform deletes the expired documents, oldest
+ first, at most `max_document_expirations_per_block` per block (128 at protocol version
+ 14), and no more than `max_document_expiration_weight_per_block` (1,024) in weight, where
+ a document weighs 1 plus the index levels of its type. The rest wait for the next block. A state transition in the block a document expires
+ in still sees it; a document can therefore outlive its expiry by the part of a block
+ before the cleanup, and by more when a backlog builds.
+- The expiry belongs to the document: it is its own `$createdAt` plus the type's `ttl`, both
+ fixed, and the platform reads it from there. The expirations tree is only the index the
+ cleanup finds documents through.
+- From its expiry on, a document can no longer be replaced, transferred, bought or repriced,
+ and a moderator can no longer restore it: each is refused, paid, with
+ `DocumentExpiredError` (40140), whether or not the cleanup has reached the document. Its
+ owner may still delete it where `canBeDeleted` allows, which only removes it sooner. Until the cleanup deletes it, it
+ can still be queried and referenced, and it keeps its values in the type's unique indexes:
+ a create of the same unique value is refused as a duplicate until the cleanup has run. The
+ cleanup deletes a fixed number per block, so a sustained flood of creations with a short
+ time to live builds a backlog it drains at that rate, and that lag grows with it.
+- The document may be deleted earlier as usual: by its owner when `canBeDeleted` allows it,
+ by the contract's moderators when `canBeDeletedByModerators` does. `canBeDeleted: false`
+ only stops the owner; the platform still deletes the document when it expires.
+- The deletion is an ordinary deletion: the document and every index entry go, count and sum
+ trees are decremented, and nothing is left behind.
+
+## Where a time to live is refused
+
+The keyword is refused, on every parse of the contract (registration, update and a stored
+contract read back), when:
+
+| The document type | Why |
+|---|---|
+| does not list `$createdAt` in `required` | The expiry is computed from it. |
+| sets `documentsKeepHistory: true` | Drive refuses to delete a document whose type keeps history. |
+| sets `indexOnly: true` | There is no stored row to delete by id. |
+| has a contested index | A contested document waits in its vote poll until the poll awards it, keeping the `$createdAt` of its create, so it could expire before it is stored. |
+| declares `ttl: 0` | A time to live lasts at least a second. |
+
+When a contract is registered or updated, a `ttl` below `min_document_ttl_seconds` (one hour at
+protocol version 14) or above `max_document_ttl_seconds` (one year) is refused too. The floor
+keeps a document in state well past the moment its writer fetches the proof of its create,
+which proves the document present; a document the cleanup had already deleted would fail that
+proof although the create succeeded. A contract update may not add, remove or change the `ttl`
+of an existing document type: every stored document carries the expiry it was written and paid
+with. A document type added by an update declares it freely.
+
+The same holds for any write close to a document's expiry: a replace, transfer, purchase or price
+update accepted in the last block before the expiry proves the document present, and a proof
+fetched after the next block's cleanup finds it gone.
+
+References treat a type with a `ttl` as deletable, like one with `canBeDeleted` or
+`canBeDeletedByModerators`. A `permanentDocument` reference, a lookup one included, and a list
+element reference may not target it; a `deletableDocument` reference may, a lookup one included.
+The check is `DocumentTypeV2Getters::documents_can_disappear`.
+
+Everything else composes: mutable types, `transferable`, `tradeMode`,
+`canBeDeletedByModerators` (a moderator's restore puts the document back with its original
+`$createdAt`, and is refused once that document has expired),
+`creationRestrictionMode`, countable, summable and ranked indexes, `timeRange` indexes with
+or without their own `ttl`, `refersTo` declared on the type, action fees and token costs.
+
+## Storage
+
+Nothing a document of such a type writes carries storage flags: its primary item, its index
+entries and the index trees it creates are flagless, and its deletion refunds nothing.
+Drive drops the writer's flags when the document is inserted or changed
+(`DocumentAndContractInfo::without_storage_flags_if_expiring`), before any element is sized,
+and the re-tagging described below strips any left.
+
+Each document also gets an entry in the **documents expirations tree** under `Misc`:
+
+```text
+Misc (104) / E / / -> contract id (32 bytes) ++ document type name
+```
+
+The entry is written with the document and removed with it, whoever deletes it (the hook is
+in `force_delete_document_for_contract_operations`, which the owner's, the moderators' and the
+cleanup's deletions share). The last entry of an expiry time takes the tree of that time with
+it, so every tree of an expiry time holds at least one entry, and a document deleted early
+leaves nothing the cleanup would have to read.
+
+The storage fees of such documents wait in the **lifetime storage fee pools** under `Pools`,
+a sum tree so the pools' total counts them, one sum item per epoch they were collected in and
+number of epochs their storage lives:
+
+```text
+Pools (48) / l / -> credits
+```
+
+Both trees are created with the initial state structure of protocol version 14 and on the
+first block of protocol version 14, through one helper (`Drive::insert_document_ttl_trees`).
+
+## Fees
+
+The fee schedule's `document_ttl` group (`FeeDocumentTtlVersion`, `FEE_VERSION3`) prices a
+document of such a type:
+
+- **Bytes.** Every byte the document writes, its expirations tree entry included, costs the
+ price of the lifetime it has left. Up to seven days a tier applies; past that a price per
+ `pricing_period_seconds` spanned, rounded up. The period is part of the schedule (788,400
+ seconds, mainnet's epoch length), not the node's epoch length, so a network configured with
+ short epochs, like testnet's hour, prices a lifetime as mainnet does.
+
+ | Lifetime | Credits per byte (protocol version 14) |
+ |---|---|
+ | up to 1 hour | 1 |
+ | up to 1 day | 4 |
+ | up to 2 days | 8 |
+ | up to 4 days | 15 |
+ | up to 7 days | 26 |
+ | longer | 34 per 9.125 days spanned |
+
+ The values are the first year's share of the perpetual storage price (27,000 credits per
+ byte, 5% of it paid out in the first year) pro rata, rounded up. A one-year `ttl` pays
+ 40 × 34 = 1,360 credits per byte, about what a permanent document deleted after a year
+ keeps paying net of its refund.
+- **Payout.** That amount is a storage fee, paid out to the epochs the document lives in
+ rather than over the 50 eras of the perpetual storage distribution. Drive counts the
+ epochs its remaining lifetime spans, rounded up and at most one era (40 epochs), with the
+ network's epoch length (the node's `epoch_time_length_s`, handed to Drive through
+ `DriveConfig`) and reports the amount under that count
+ (`FeeResult::lifetime_storage_fees`). At the end of the block the amounts go to the
+ lifetime storage fee pools of the current epoch
+ (`add_distribute_block_fees_into_pools_operations` v1), and the next epoch change spreads
+ each pool of an earlier epoch evenly over its number of epochs from the new epoch on, the
+ remainder of the division to the new epoch, and removes it
+ (`add_distribute_storage_fee_to_epochs_operations` v1). A block never adds to a pool its
+ epoch change removes, so the first block of an epoch does both in one batch. A network with
+ hour-long epochs therefore pays a year-long document out within 40 hours.
+- **Deletion.** Creating the document prepays, as processing, what its deletion will cost:
+ `cleanup_base_processing_cost` (1,200,000), plus `cleanup_processing_cost_per_index_level`
+ (400,000) per index level of the type, where an index counts its properties, times the
+ overlapping windows of a `timeRange` index, plus `cleanup_processing_cost_per_document_byte`
+ (420, what removing and reading a byte costs) per byte of the stored document. The cleanup
+ itself bills nobody.
+- **Changes.** A replace, transfer, purchase or price update prices the bytes it adds by the
+ lifetime left at that block, and prepays the deletion of the document bytes it adds at
+ `cleanup_processing_cost_per_document_byte`. A deletion by the owner or a moderator pays its
+ own processing like any deletion; the prepaid deletion is the platform's, and is not
+ refunded.
+
+The price never decreases with the lifetime, so an estimate made at an earlier block time
+(check_tx) stays an upper bound of the execution; where the amount is paid out does not change
+what the writer pays, and a fee increase the writer offers, which multiplies processing only,
+never multiplies it. A dry
+run estimates a change as an insert of the changed document
+(`estimate_document_change_as_insert_operations_v1`): without the entry and the deletion the
+creation prepaid, which a change never pays, and with the deletion of all the document's
+bytes, at least what the change adds. In Drive the document's grove operations are
+re-tagged `EphemeralGroveOperation(_, EphemeralPricing::DocumentTtl { .. })` and applied as
+their own GroveDB batch, so their added bytes can be priced apart from the rest of the
+transition (see `apply_batch_low_level_drive_operations`); operations already tagged for a
+`timeRange` index's `ttl` keep that rule.
+
+## Expired documents
+
+`validate_document_not_expired` (drive-abci, `state_transition/common`) is the one rule: a
+document of a type with a `ttl` has expired when block time is at or past its `$createdAt`
+plus the `ttl` (Drive's `document_expires_at`, the time its entry is keyed by), the same
+boundary the cleanup deletes at. The batch's advanced structure validation (v1, protocol
+version 14 only), which check_tx runs too, calls it for every document action through
+`validate_document_action_not_expired`, an exhaustive match that refuses, paid, a replace,
+transfer, purchase or price update of an expired document. The moderator restore calls it too, judging the
+`$createdAt` of the document the removal record's hash pins. A document deletion by its owner
+does not.
+
+## Cleanup
+
+`Platform::expire_documents` runs after the block's state transitions, right after the address
+balance cleanup in `run_block_proposal`, and calls `Drive::remove_expired_documents` with
+`max_document_expirations_per_block` and `max_document_expiration_weight_per_block`:
+
+1. `fetch_expired_documents` reads, in one query, the entries of the expiry times at or before
+ the block time, oldest first, at most the limit of them. Every tree of an expiry time holds
+ an entry, so the query visits at most the limit of trees.
+2. Each expired document is read and checked against state (its contract, document type,
+ `ttl` and stored document, and that the document expires when its entry says), then
+ deleted from what was read: the deletion an owner runs, without the `canBeDeleted` guard
+ (`delete_read_document_for_contract_operations_v0`). Its entry, and the tree of its expiry
+ time when it was the last, go with it.
+3. An entry without a document to delete is logged and only removed. None is expected, and
+ failing the block over one would halt the chain.
+4. The cleanup stops before a removal that would pass the weight budget (an entry's removal
+ weighs 1), except the block's first, so the backlog always drains.
+
+Every removal goes into one GroveDB batch, applied without a fee, each built against the
+removals queued before it, so every emptiness check sees them.
+
+## Versioning
+
+Everything rides protocol version 14, unreleased when this landed: the keyword joined document
+meta-schema v3 and the generation 3 parser, the limits joined `SYSTEM_LIMITS_V4`, the fee group
+joined `FEE_VERSION3`, the update rule joined `validate_update` v1, and `expire_documents` is
+`Some(0)` in `DRIVE_ABCI_METHOD_VERSIONS_V10` only. The shipped generations edited in place are
+inert before 14:
+
+- the deletion hook in `delete_document_for_contract_operations` v0, the reference checks
+ (`documents_can_disappear`) and the restore check of `contract_user_moderation` state v0:
+ `documents_ttl_seconds` is only ever `Some` on a document type parsed from the keyword,
+ which no earlier protocol version reads. The deletion's post-read part moved into
+ `delete_read_document_for_contract_operations_v0` with its operations unchanged;
+- the `expire_documents` call in `run_block_proposal` v0: the method is `None` before 14;
+- the tree's creation in `transition_to_version_14` (`perform_events_on_first_block_of_protocol_change`
+ v0), which only an upgrade to 14 runs;
+- one batch per pricing rule in `apply_batch_low_level_drive_operations` v0 and the
+ `DocumentTtl` arm of `consume_to_fees_v0`: nothing is tagged ephemeral before 14;
+- `add_distribute_block_fees_into_pools_operations` v0, split into a helper that v1 shares,
+ with its operations unchanged, and `fetch_pending_epoch_refunds` v0, whose query and reading
+ moved unchanged into a helper the lifetime storage fee pools share;
+- the pattern-only edits in `batch_insert_empty_tree_if_not_exists` v0 and
+ `convert_drive_operations_to_grove_operations` v0, whose output is unchanged.
diff --git a/book/src/data-model/documents.md b/book/src/data-model/documents.md
index 18d815cf5ce..e173a4c39ae 100644
--- a/book/src/data-model/documents.md
+++ b/book/src/data-model/documents.md
@@ -678,9 +678,36 @@ The parser (generation 3, meta-schema v3, `apply_max_bytes`) folds the bound int
The check runs where the JSON schema validation of a document's properties runs, `DataContract::validate_document_properties`, right after it: on every document create and replace, and in every client that validates a document before sending it. A longer string is refused with `DocumentPropertyMaxBytesExceededError` (basic code 10421), which names the property (`tags[2]` for an element) and both lengths. The document validation (version 0, extended in place) is inert before protocol version 14, where no string carries a byte cap and the `validate_max_bytes` method slot is `None`. In Rust the check is `DocumentTypeBasicMethods::validate_max_bytes_properties()`; in JavaScript the error reaches an app as `DocumentMaxBytesErrorCode.MaxBytesExceeded`.
+## Generated Properties (`generatedFrom`)
+
+Protocol version 14 adds the property keyword `generatedFrom`: the platform generates a string property's value with a system function of other properties of the same document, its params. The first system functions change the case of a string (`lowercase`, `uppercase`, `capitalize`, `camelCase`, `snakeCase`) or apply the DPNS `domain` rule (`normalizedLabel` is `label` lowercased, with `o`, `i` and `l` replaced by `0`, `1` and `1`), so any contract can build a unique index that treats look-alike names as one.
+
+```json
+"normalizedLabel": {
+ "type": "string", "maxLength": 63,
+ "generatedFrom": {
+ "function": "sys.stringTransformations.homographSafeASCII",
+ "params": ["label"]
+ },
+ "position": 1
+}
+```
+
+The functions are a closed list of system functions (`SystemFunction`), named under `sys.` so that functions a contract may bring later can be told apart by name; each declares how many params it takes. `SystemFunction` is an enum of namespaces, each an enum of its own in a module of its own: `SystemFunction::StringTransformation(StringTransformation)` for `sys.stringTransformations`, in `system_function::string_transformations`, whose six functions each take one string. `lowercase`, `uppercase` and `capitalize` change the case of ASCII letters; `camelCase` and `snakeCase` split the string into words (at every ASCII character that is neither a letter nor a digit, and before an ASCII uppercase letter that starts a word) and join them as `helloWorld` or `hello_world`; `homographSafeASCII` maps `A` to `Z` to lowercase, then `o` to `0` and `i` and `l` to `1`. Every one changes ASCII characters only and keeps every other character, with no Unicode table, because Unicode case mappings differ between releases of the standard library, and two nodes must never generate different values; every one gives back its own output unchanged, and a test pins that the meta-schema lists exactly the functions the parser knows. On ASCII `homographSafeASCII` equals `convert_to_homograph_safe_chars`, the function the DPNS trigger uses; a test pins that over every ASCII character and over strings from the DPNS alphabets. It refuses nothing: the characters a value may hold are each param's `pattern`'s to decide, and the generated property needs no pattern of its own, since every value it holds is generated from params that passed theirs. A param is a property path for now (`GenerationParam::Property`); literals, system values and nested calls can be added later without changing what parses today.
+
+The parser (generation 3, meta-schema v3, `apply_generated_from`) reads the declaration onto `DocumentProperty::generated_from` (`Option`, absent on every property parsed before protocol version 14) and checks that `params` lists as many params as the function takes. `validate_generated_from_declarations` checks every param against the other properties on every parse: another string property, not generated itself, not transient nor inside a transient object (and neither is the declaring property), and inside every object that holds the declaring property. The meta-schema refuses the keyword beside `$ref`, whose definition replaces every keyword written next to it. On contract update a changed, added or removed `generatedFrom` is an incompatible schema change, and a property the update adds may declare it only when one of its params is new too (`validate_update` 1 refuses one whose params all existed with `DocumentTypeUpdateError`, 40212: documents stored before the update were never generated). The declaring paths and their declarations are cached on the document type (`DocumentTypeV2Getters::generated_from_fields`), so a write to a type without declarations pays nothing.
+
+Three methods do the work at write time:
+
+- `DocumentTypeBasicMethods::fill_generated_properties` writes every declared property the document leaves out while supplying every param. The action transformers of document create, replace and index-only delete call it first, before the contest resolution and every check read the data, so the stored document, its indexes and its contest all hold the generated value. `Document::try_from_create_transition` and `try_from_replace_transition` call it too, and so does `index_only_transition_entry_path_query`, the one builder the prover and the verifier share for index-only entries, so proofs are built and checked against the document the platform stored. `DocumentTypeBasicMethods::data_as_stored` returns a transition's data with the generated properties written into a copy, or borrows it as it is on a type that declares none; the document subscription filter (`DriveDocumentQueryFilter::matches_document_transition`) reads a create's and a replace's data, and an index-only delete's values, through it, so a subscription on a generated property sees the value the platform stores.
+- `DocumentTypeBasicMethods::regenerate_generated_properties` is the client-side twin: it sets every declared property to what its params generate, replacing a value the document holds and removing it when a param is absent. The transition builders (`from_document` of create, replace and index-only delete), the SDK's contest fund lookup and the property-constraint pre-checks of the JavaScript and FFI SDKs call it, so a document fetched, edited and sent back carries the value of its new params rather than the stale one the platform would refuse, and its contest is detected from it. Random documents call it too, after drawing the params of every generated property they drew.
+- `DocumentTypeBasicMethods::validate_generated_from_properties` runs in `DataContract::validate_document_properties`, after the JSON schema and `maxBytes`: a declared property must equal what its function generates from its params, and be absent when a param is. A document that repeats a key in an object on the way to the property or to a param is refused too: the schema validation and the stored document keep the last of repeated keys, where the path reads the platform generates from find the first. A property that does not pass refuses the write with `DocumentPropertyNotGeneratedError` (basic code 10424). A document that arrived has been generated, so on the platform the check only refuses a value the client sent, a value sent without its params, or a repeated key.
+
+Every call site was extended in place and is inert before protocol version 14: the `apply_generated_from`, `fill_generated_properties` and `validate_generated_from` slots are `None` there, and the meta-schemas refuse the keyword. In JavaScript the error reaches an app as `DocumentGeneratedFromErrorCode.DocumentPropertyNotGenerated`.
+
## Property Constraints (`propertyConstraints`)
-Protocol version 14 adds the doctype-level `propertyConstraints` keyword: named rules the integer properties of every created or replaced document must meet, where JSON Schema can only bound one property at a time. Each rule compares two integer expressions.
+Protocol version 14 adds the doctype-level `propertyConstraints` keyword: named rules the properties of every created or replaced document must meet, where JSON Schema can only bound one property at a time. Each rule is a condition: a comparison of two integer expressions, a test of whether an integer expression takes one of listed values, a comparison of a string property with string constants, a test of whether the document holds a property, or `anyOf`, `allOf` or `not` over conditions.
```json
"propertyConstraints": {
@@ -693,27 +720,59 @@ Protocol version 14 adds the doctype-level `propertyConstraints` keyword: named
"wholeLots": { "equal": [{ "modulo": ["quantity", 10] }, 0] },
"minimumOrder": {
"greaterThanOrEqual": [{ "multiply": ["price", { "ifAbsent": ["quantity", 1] }] }, 100]
+ },
+ "feeWaivedOrAtLeastTen": {
+ "anyOf": [{ "equal": ["fee", 0] }, { "greaterThanOrEqual": ["fee", 10] }]
+ },
+ "discountGivenAboveZero": {
+ "anyOf": [{ "absent": "discount" }, { "greaterThan": ["discount", 0] }]
+ },
+ "tieredFee": { "in": ["fee", [0, 10, 25, 50]] },
+ "closedNeedsClosedAt": {
+ "anyOf": [{ "notEqual": ["status", { "const": "closed" }] }, { "present": "closedAt" }]
}
}
```
-The first rule reads `((price + fee) * quantity) <= deposit`. A rule's name is 1 to 64 letters, digits or underscores, and the rule is an object with one key, its comparison: `equal`, `notEqual`, `lessThan`, `lessThanOrEqual`, `greaterThan` or `greaterThanOrEqual`, listing the left and the right expression. An expression is one of:
+The first rule reads `((price + fee) * quantity) <= deposit`, `feeWaivedOrAtLeastTen` reads `fee == 0 || fee >= 10`, `discountGivenAboveZero` lets an offer leave its discount out but not give a discount of 0, `tieredFee` holds the fee to four tiers, and `closedNeedsClosedAt` says a closed offer carries a `closedAt`. A rule's name is 1 to 64 letters, digits or underscores, and the rule is a condition, an object with one key:
+
+- a comparison, `equal`, `notEqual`, `lessThan`, `lessThanOrEqual`, `greaterThan` or `greaterThanOrEqual`, listing the left and the right expression;
+- `{ "in": [expression, [values]] }`, holding if the integer expression takes one of two or more distinct integer values. It says what an `anyOf` of `equal`s says, in one node per value instead of three, so a set of up to 30 values fits the node limit where the `anyOf` fits 10. A value is a literal, never a path or an expression;
+- a string comparison: `{ "equal": [path, { "const": "closed" }] }` or `notEqual`, with the constant on either side; `{ "notEqual": ["fromCurrency", "toCurrency"] }`, two bare paths that both name string properties, which compares their strings; or `{ "in": [path, ["open", "pending"]] }`, whose values are two or more distinct strings. The path names a string property, typically one with an `enum`. A string on its own is a path, so a constant is written as `{ "const": ... }`, while the values an `in` lists are literals and need no wrapper. Strings are only compared for equality, never ordered or used in arithmetic. A string property the document leaves out equals no constant and no other string property, not even one also left out, so `notEqual` holds for it and `equal` and `in` do not, unless `{ "ifAbsent": [path, "open"] }` gives it a string default, which it then reads as (it may stand wherever the bare path does, and makes the comparison one of strings); `present` and `absent` test it directly. When the property declares an `enum`, every constant compared with it must be one of the enum's values, so a misspelling is refused at registration rather than making the rule quietly never hold;
+- an identifier comparison, the same three forms for identifier properties, those declaring `refersTo` included: `{ "equal": ["paymentToken", { "const": "" }] }` or `notEqual`, `{ "notEqual": ["buyerId", "sellerId"] }`, or `{ "in": ["paymentToken", ["", ""]] }`. Constants are base58 identifiers of 32 bytes, checked at registration, and compared by their bytes, whatever form the document gives the identifier in. An identifier property the document leaves out equals no identifier, not even another one left out; identifiers take no `ifAbsent` default and are never ordered. `$ownerId`, the document's owner, is an identifier operand too: `{ "equal": ["authorId", "$ownerId"] }` holds the author to the owner, and `{ "in": ["$ownerId", ["", ...]] }` lets only the identities listed own a document of the type. It is no property, so `present` or an integer operand refuses it, and comparing it with itself is refused. Since a transfer and a purchase give the document a new owner, each is judged against the rules reading `$ownerId`, with the new owner, and refused when it would break one; an indexOnly type refuses such a rule, since its deletes carry no owner;
+- `{ "startsWith": [text, affix] }` and `{ "endsWith": [text, affix] }`, holding if the first string starts or ends with the second, byte for byte with no case folding: each side a `{ "const": ... }`, a string property or an `ifAbsent` string default, at least one a property, never the same one twice. `{ "startsWith": ["url", { "const": "https://" }] }` holds a link to https, and `{ "startsWith": ["path", "parentPath"] }` a path under its parent's. A string property left out without a default takes no string, and the condition does not hold for it; a constant tested against a property that declares an `enum` must start or end one of its values;
+- `{ "contains": [path, value] }`, holding if the typed array property at the path holds an element equal to the value, looked for as the array's elements are: an integer expression among integers, a string constant, string property or `ifAbsent` string default among strings, an identifier constant, identifier property or `$ownerId` among identifiers. `{ "not": { "contains": ["labels", { "const": "used" }] } }` refuses a `"used"` label, and `{ "contains": ["participants", "$ownerId"] }` holds the owner to the participants (so a transfer or purchase to a non-participant is refused). An array the document leaves out holds nothing, a string or identifier property it leaves out is among no elements, and a string constant must be one of the elements' `enum` values when they declare one;
+- `{ "present": path }`, holding if the document holds the property, and `{ "absent": path }`, holding if it leaves it out (a property set to null counts as left out, and so does an object none of whose members is present, such as `{}`, which a stored document does not keep). An operand reads a property the document leaves out as 0, so only these tell "not given" from "given as 0". They may name a property of any type, an object or a member of one included, since they read no value;
+- `{ "anyOf": [...] }`, holding if at least one of two or more conditions holds;
+- `{ "allOf": [...] }`, holding if every one of two or more conditions holds;
+- `{ "not": condition }`, holding if its one condition does not;
+- `{ "ifThen": [a, b] }`, holding if `b` holds whenever `a` does: `b` is evaluated only when `a` holds, so `{ "ifThen": [{ "greaterThan": ["discount", 0] }, { "greaterThanOrEqual": [{ "divide": ["price", "discount"] }, 10] }] }` never divides by zero, and a fault in either breaks the rule. It says what `{ "anyOf": [{ "not": a }, b] }` says, in one node fewer. The two may not be alike;
+- `{ "ifThenElse": [a, b, c] }`, holding if `b` holds when `a` does and `c` holds when it does not; only the branch `a` selects is evaluated. `{ "ifThenElse": [{ "greaterThanOrEqual": ["price", 1000] }, { "lessThanOrEqual": ["fee", 50] }, { "lessThanOrEqual": ["fee", 10] }] }` allows a higher fee on an expensive offer. No two of the three may be alike;
+- `{ "notIn": [expression, [values]] }`, an `in` negated, listed the same way and in as many nodes: `{ "notIn": ["fee", [7, 13]] }` refuses two fees.
+
+Conditions nest: `{ "not": { "allOf": [{ "equal": ["price", 0] }, { "greaterThan": ["quantity", 10] }] } }` refuses a free order of more than 10. An `anyOf` or `allOf` may not list two alike conditions, nor hold one of its own kind directly (it says what one flat list says), and a `not` may not hold a `not` or a `notIn` directly. An expression is one of:
- an integer value (`100`; a float with no fractional part, `100.0`, reads as that integer, as the meta-schema's `integer` type admits it);
-- a string, the dotted path of an integer property of the document type (`"price"`, `"meta.total"`), whose value it takes, 0 when the document leaves the property out;
-- `{ "ifAbsent": [path, value] }`, the property's value, or `value` when the document leaves it out;
+- a string, the dotted path of an integer or boolean property of the document type (`"price"`, `"meta.total"`, `"waiveFee"`), whose value it takes, 0 when the document leaves the property out. A boolean reads as 1 for true and 0 for false, so `{ "equal": [{ "multiply": ["waiveFee", "fee"] }, 0] }` says a waived fee is 0;
+- `{ "ifAbsent": [path, value] }`, the property's value, or `value` when the document leaves it out (an integer value here; a string value gives a string property a default in a string comparison instead);
- `{ "add": [...] }` or `{ "multiply": [...] }` over two or more operands;
-- `{ "subtract": [a, b] }`, `{ "divide": [a, b] }`, `{ "modulo": [a, b] }` or `{ "power": [a, b] }`.
+- `{ "min": [...] }` or `{ "max": [...] }`, the least or greatest of two or more operands, every one evaluated (a fault in any breaks the rule), and `{ "abs": a }`, the absolute value of one: `{ "lessThanOrEqual": ["fee", { "max": [10, { "divide": ["price", 10] }] }] }` caps a fee at 10 or a tenth of the price, whichever is more, and `{ "lessThanOrEqual": [{ "abs": { "subtract": ["a", "b"] } }, 5] }` keeps two values within 5;
+- `{ "subtract": [a, b] }`, `{ "divide": [a, b] }`, `{ "modulo": [a, b] }` or `{ "power": [a, b] }`;
+- a size: `{ "length": path }`, the characters of a string property (counted as `maxLength` counts them), `{ "byteLength": path }`, its UTF-8 bytes (as `maxBytes` counts them), or `{ "count": path }`, the items of an array property or the bytes of a byte array property. Where `maxLength`, `maxBytes` and `maxItems` bound one property by a fixed number, a size can be compared with another property or bounded only under a condition: `{ "lessThanOrEqual": [{ "count": "tags" }, "maxTags"] }` holds a list to its own limit, and `{ "anyOf": [{ "greaterThan": ["fee", 0] }, { "lessThanOrEqual": [{ "length": "title" }, 20] }] }` keeps a free listing's title short. A property the document leaves out or sets to null has size 0, and a size never breaks a rule by itself (a value of another type would read as 0 too, but the schema validation refuses it first);
+- a system time or height: `"$createdAt"`, `"$updatedAt"` and `"$transferredAt"`, block times in milliseconds, and each with `BlockHeight` or `CoreBlockHeight` appended, the Platform and Core block heights: those of the document's creation, of its last create, replace or price update, and of its last create, transfer or purchase. A rule may read one only on a type that records it by listing it in `required`, so every stored document holds it; it takes no `ifAbsent`, and an indexOnly type, whose deletes carry none, reads none. `{ "lessThanOrEqual": [{ "subtract": ["endsAt", "$createdAt"] }, 604800000] }` keeps a listing to a week from its creation, and `{ "lessThanOrEqual": ["$updatedAt", "endsAt"] }` refuses a replace or a price update after it ends;
+- a total read from state: `{ "countOf": [type] }` or `{ "countOf": [type, filter] }`, how many documents of a type of the same contract there are, or how many match the filter, and `{ "sumOf": [type, property] }` or `{ "sumOf": [type, property, filter] }`, the total of an integer property over them. A filter maps each key, a property of the counted type or `$ownerId`, to the value it must take, read from the document being written: one of its properties, `$ownerId`, an integer or a `{ "const": ... }`. The total is the one a count or sum tree keeps, as it will be once the write is done (the document itself counted when the type is its own, at its new values, and moved to its new owner by a transfer or purchase), so `{ "lessThanOrEqual": [{ "countOf": ["listing", { "$ownerId": "$ownerId" }] }, 10] }` on `listing` keeps every owner at ten listings or fewer. A whole-type total needs `documentsCountable` or `documentsSummable`, and a filtered one an index of the counted type whose properties are exactly the filter's keys, countable or summing the property. A rule is judged only when its own type is written, so a fact about another type can go stale after the write.
A JSON number is always a value and a string always a path, so a property named `100` is not confused with the number, and the rule is a tree the meta-schema can check rather than a string with precedence rules to parse. Consensus holds nothing but this tree; an SDK may offer an infix spelling that compiles to it.
The arithmetic is exact over `i128`. Operands are evaluated left to right, and every intermediate result must fit: an overflow, a divisor of 0, a negative exponent or a property value that is not an integer (a float with no fractional part passes the schema's `integer` type) breaks the rule instead of wrapping or truncating. `divide` and `modulo` are Euclidean, so the remainder is never negative and the quotient is the one that goes with it (`-7` by `2` is `-4` remainder `1`); for operands that are not negative this is ordinary integer division. `0` to the power `0` is `1`. There are no floats: a `number` property cannot be read, which keeps every node's result bit-identical.
-The parser (generation 3, meta-schema v3) checks the keyword on every parse, stored contracts included: the shape, that every path names an integer property of the type (a nested one by its dotted path) that is neither `transient` nor inside a transient object (a transient value is never stored, so a stored document could not be held to the rule), that every rule reads at least one property, that no literal divisor is 0 and no literal exponent negative, and that no operand nests deeper than `MAX_PROPERTY_CONSTRAINT_PARSE_DEPTH` (64), a constant that keeps a parse without full validation from recursing without bound and that no registrable rule comes near. Under full validation, when a contract registers or updates, it also holds the limits: at most `SystemLimits::max_property_constraints` rules per type (16) and `max_property_constraint_nodes` nodes per rule (32), counting the comparison, every operator and every operand. The rules are fixed when the document type is created: adding, removing or changing one is an incompatible schema change (`IncompatibleDocumentTypeSchemaError`, 10246), since stored documents were judged against the rules as they were.
+Conditions are checked in declared order and no further than the outcome needs: a comparison evaluates its left side, then its right; `anyOf` stops at the first condition that holds and `allOf` at the first that fails. A fault in a condition that is checked breaks the rule whatever the others would say, and `not` does not turn it into a pass. So an earlier condition guards a later one: `{ "anyOf": [{ "equal": ["b", 0] }, { "equal": [{ "divide": ["a", "b"] }, 2] }] }` holds for a `b` of 0 without dividing by it, while the same two conditions the other way round divide by zero and break the rule.
+
+The parser (generation 3, meta-schema v3) checks the keyword on every parse, stored contracts included: the shape, that every path an operand reads names an integer or boolean property of the type, every path `length` or `byteLength` measures a string property, every path `count` counts an array or byte array property, every system time or height a rule reads is one the type lists in `required` (and an indexOnly type reads none), every path compared with a string names a string property (and every constant compared with one that declares an `enum` is one of its values) (a nested one by its dotted path) and every path `present` or `absent` tests names a property of the type, and that neither is `transient` nor inside a transient object (a transient value is never stored, so a stored document could not be held to the rule), that every comparison and `in` reads at least one property (a constant one would make its rule, or an `anyOf` around it, hold for every document or for none), that no `in` lists a value twice, that no `anyOf` or `allOf` holds one of its own kind directly and no `not` a `not`, that no literal divisor is 0 and no literal exponent negative, and that no condition or operand nests deeper than `MAX_PROPERTY_CONSTRAINT_PARSE_DEPTH` (64), a constant that keeps a parse without full validation from recursing without bound and that no registrable rule comes near. Under full validation, when a contract registers or updates, it also holds the limits: at most `SystemLimits::max_property_constraints` rules per type (16) and `max_property_constraint_nodes` nodes per rule (32), counting every comparison and logical operator, every `in` and each value it lists, every `const`, every `present` or `absent`, every arithmetic operator and every operand (a size is one), that no `anyOf` or `allOf` lists the same condition twice (conditions that parse alike, so `1` and `1.0` are the same value), and that the rules of one type read at most `max_property_constraint_aggregates` (4) distinct totals. Once every document type of the contract is parsed, a registration also checks every `countOf` and `sumOf`: the type it counts is one of the contract's and not indexOnly, nor the declaring type when it has a contested index (a document a contest awards is stored without the rules judged), a tree of it keeps the total, the filter's keys and values are integers, strings or identifiers of the same kind, and every property a value reads is required. The rules are fixed when the document type is created: adding, removing or changing one is an incompatible schema change (`IncompatibleDocumentTypeSchemaError`, 10246), since stored documents were judged against the rules as they were.
-Enforcement lives in `DataContract::validate_document_properties` (generation 0, extended in place: the call is inert before protocol version 14, where `validate_property_constraints` is `None`), after the schema validation. Document create and replace structure validation call it, so consensus applies the rules, and so does every client that validates a document before sending it. The rules are checked in name order against the document's properties, which for a replace is the whole document, and the first one broken fails with `DocumentPropertyConstraintViolatedError` (basic code 10422), naming the document type, the rule and why: the comparison does not hold, or evaluating it overflowed, divided by zero, raised to a negative power or read a value that is not an integer. The check reads no state and changes nothing stored, so it adds no fee; the limits bound its cost. Transfers, purchases and price updates change no property and are not judged.
+Enforcement lives in `DataContract::validate_document_properties` (generation 0, extended in place: the call is inert before protocol version 14, where `validate_property_constraints` is `None`), after the schema validation. Document create and replace structure validation call it, so consensus applies the rules, and so does every client that validates a document before sending it. The rules are checked in name order against the document's properties, which for a replace is the whole document, and the first one broken fails with `DocumentPropertyConstraintViolatedError` (basic code 10422), naming the document type, the rule and why: the rule does not hold, or evaluating it overflowed, divided by zero, raised to a negative power or read a value that is not an integer. The check changes nothing stored. It reads state only for the `countOf` and `sumOf` totals, which the document batch transformer reads when it builds the action (`Drive::fetch_property_constraint_aggregate`, billed with the write) and hands the check in `DocumentSystemValues::aggregates`; the limits bound its cost. `$ownerId` and the system times and heights read the values of the document version being written (`validate_document_properties` takes them as `DocumentSystemValues`): on a create the writer and the block's time and heights, on a replace the writer, the stored creation and transfer values and the block's as the update. A client passes what it knows: an owner it does not know equals no identifier, and a rule reading a time, a height or a total it is not given is not judged. Consensus reads every total the rules it judges read (`DocumentSystemValues::aggregates` is `Some`), so one missing there is an error in the code building the write, never a skipped rule. The SDK pre-checks use the device clock for the times a write records, and leave a rule reading a block height unjudged, since the height is unknown until the block. Transfers and purchases change no property, only the owner and the transfer's time and heights, so only the rules reading those are judged again, with the new values (`DocumentTypeV0Methods::validate_property_constraints_for_system_change`, next to the `distinctFrom` check). Price updates change only the update's time and heights, and are judged against the rules reading those the same way.
-In Rust the rules are `DocumentTypeV2Getters::property_constraints` (a map from name to `PropertyConstraint`, empty on types that predate the keyword), each rule's `violation` evaluates it against a document's data, and the document check is `DocumentTypeV0Methods::validate_property_constraints`.
+In Rust the rules are `DocumentTypeV2Getters::property_constraints` (a map from name to `PropertyConstraint`: a comparison, an `in`, a string comparison (`TextCompare`, `TextIn`), a `present` or `absent`, or an `anyOf`, `allOf` or `not` of them, empty on types that predate the keyword; `property_reads` lists what a rule reads and how: by value, by presence, by size (`Length`, `Count`) or in a comparison of strings or identifiers, `system_reads` the system times and heights it reads, and `aggregate_reads` the totals, each an `AggregateRead`), each rule's `holds` and `violation` evaluate it against a document's data and `DocumentSystemValues`, and the document check is `DocumentTypeV0Methods::validate_property_constraints`.
## Rules and Guidelines
diff --git a/book/src/drive/document-count-trees.md b/book/src/drive/document-count-trees.md
index 2a6ac5b1415..51b9717e985 100644
--- a/book/src/drive/document-count-trees.md
+++ b/book/src/drive/document-count-trees.md
@@ -393,6 +393,10 @@ A few notes about the index-level flag:
A migration check from `dapi-grpc` server logic: every count query requires either `documentsCountable: true` (for unfiltered totals) or a `countable: true` / `rangeCountable: true` index whose properties **exactly match** the query's where-clause fields. No covering index → the call returns a clear `InvalidArgument` describing what the picker was looking for ("requires a `range_countable: true` index whose last property matches the range field" for range queries, "requires a countable index whose properties exactly match the where clause fields" for Equal/In queries). Pick your indexes deliberately at contract creation time — per-index `countable: true` / `rangeCountable: true` flags can't be added later (contract indexes are immutable post-creation).
+### Counts Are Public
+
+Anyone can run a count query and verify its proof, so a countable index publishes everything its counts reveal. Encrypting a document's fields does not hide its existence or its indexed values. A countable index keyed first by a recipient and then by the document owner answers "who sent documents to this recipient, and how many" for every recipient. On the DashPay `contactRequest` type, a countable `[toUserId, $ownerId]` index would publish every user's inbound contacts. If a UI only needs a badge, a countable index on the recipient alone (`["toUserId"]`) reveals a total and no per-sender edges.
+
## SDK Access at Three Layers
### `rs-sdk` (native Rust)
diff --git a/book/src/drive/document-sum-trees.md b/book/src/drive/document-sum-trees.md
index 932b240704c..0eb0510a859 100644
--- a/book/src/drive/document-sum-trees.md
+++ b/book/src/drive/document-sum-trees.md
@@ -291,7 +291,7 @@ Set at the same level as `type` / `properties` / `indices` on a document type:
"properties": {
"recipient": { "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32, "position": 0,
"contentMediaType": "application/x.dash.dpp.identifier" },
- "amount": { "type": "integer", "minimum": 1, "position": 1 },
+ "amount": { "type": "integer", "minimum": 1, "maximum": 4294967295, "position": 1 },
"sentAt": { "type": "integer", "minimum": 0, "position": 2 }
},
"required": ["recipient", "amount", "sentAt"],
diff --git a/book/src/drive/grovedb-structure.md b/book/src/drive/grovedb-structure.md
index 7a31a6f28a9..108373f8d31 100644
--- a/book/src/drive/grovedb-structure.md
+++ b/book/src/drive/grovedb-structure.md
@@ -181,6 +181,38 @@ shape is that the fixture writes the same keys in the same batches as the
block pipeline does, since a Merk batch of several keys gives another tree
than the same keys written one at a time.
+## One document type's layout
+
+The description covers every document type at once, so its document layers
+are templates: an index property, one of its values, the `[0]` where an index
+ends, and which tree types each of them can be. Which of those a given
+document type gets depends on its keywords (`unique`, `countable`,
+`summable`, the ranked keys, `timeRange`, `indexOnly`, `documentsKeepHistory`,
+...), and the rules that pick them live in the index walkers.
+
+`drive::document::layout::document_type_layout` applies those rules to one
+document type and returns its concrete layout: the document type tree, the
+documents by id, and for each index the property and value trees down to
+where it ends, each with the tree or element type Drive writes, the
+zero-contribution wrapper a continuation tree gets under an aggregating value
+tree, the ranking axes of an indexed tree, the indexes that use the layer,
+and conditions such as the tree a unique index falls back to when a value is
+null. Each layer names the node of the description it is an instance of, so a
+viewer can link to it. It needs only the contract, so it is compiled with the
+`verify` feature as well, and the JavaScript SDKs expose it as
+`documentTypeLayout(contract, documentTypeName, platformVersion)`. It follows
+the v2 index walkers, so it refuses a platform version before 14.
+
+The tree types come from the functions the walkers call, and the element
+choices the walkers make inline are held to them by
+`should_lay_out_what_drive_writes`. It applies 19 test contracts (plain,
+unique, compound, countable, summable, ranked and chained indexes, history,
+time windows, indexOnly types with flat and preallocated indexes), inserts
+documents, and fails on any element whose kind or wrapper the layout does not
+predict, and on any layer of the layout Drive did not write. It does not cover
+contested indexes, time windows with a `ttl`, or document updates and
+deletes.
+
## Pull requests that change the structure
When a pull request changes `grovedb-structure.json`, the
diff --git a/book/src/drive/index-only-document-types.md b/book/src/drive/index-only-document-types.md
index 34b72a4454a..b1ce228332c 100644
--- a/book/src/drive/index-only-document-types.md
+++ b/book/src/drive/index-only-document-types.md
@@ -384,11 +384,26 @@ the terminal (`terminal > `, with a limit) walks the entries
page by page — **keyset pagination**, the indexOnly replacement for
id-shaped `startAt` cursors, which cannot address a position whose
synthesized id is a one-way hash. Mixed shapes are served through a
-**prefix pivot**: one range or `in` clause may sit on a prefix property
-instead of the terminal (`hashtag == h AND postId > p AND $ownerId ==
-me`), with everything above the pivot equality-bound, everything below
-it unconstrained, and the terminal clause an equality. All shapes prove
-and verify through the same shared path-query builder.
+**prefix pivot**: one `in` clause may sit on the index's last prefix
+property instead of the terminal (`hashtag == h AND postId IN [p, q] AND
+$ownerId == me`), with everything above it equality-bound, the terminal
+clause an equality, and a limit of at least the number of `in` values.
+
+A range pivot (`postId > p` in the same query), or an `in` pivot with
+prefix properties below it, is refused, and the error names the index
+shape that serves the query: one that lists the equality-bound
+properties, the terminal's included, before the ranged property. A
+pivot walk opens one branch per pivot value, and grovedb charges a
+branch that holds no row one slot of the limit, so a page of such a
+query could hold fewer rows than exist, and the response carries no
+cursor to say where it stopped. These shapes stay refused until the
+storage layer can report where a page stopped. An `in` pivot on the last
+prefix property opens at most one branch per value, so a limit that
+covers its values is never used up early. When another index serves the
+same query without an incomplete pivot, index selection prefers it over
+a pivot index that would win the name-order tie-break.
+
+All shapes prove and verify through the same shared path-query builder.
Not supported on the read surface: by-`$id` fetches (no primary tree —
rejected with guidance) and `startAt` cursors (rejected with the keyset
diff --git a/book/src/drive/indexes.md b/book/src/drive/indexes.md
index 347136c8c74..77656aedc14 100644
--- a/book/src/drive/indexes.md
+++ b/book/src/drive/indexes.md
@@ -52,7 +52,7 @@ pub struct IndexProperty {
### `name`
-A short, human-readable identifier for the index (e.g. `"byOwnerAndType"`). It shows up in error messages and is the key used in `document_type.indexes()` (`BTreeMap`). If omitted in the schema, a random alphanumeric name is generated. Two indexes within the same document type cannot share a name.
+A short, human-readable identifier for the index (e.g. `"byOwnerAndType"`). It shows up in error messages and is the key used in `document_type.indexes()` (`BTreeMap`). Every document meta-schema requires it. A parse that skips schema validation (check tx, test fixtures) and meets an unnamed index derives the name from the properties and their directions, joined with `|`, so every parse of the same contract agrees on it. Two indexes within the same document type cannot share a name.
### `properties: Vec`
@@ -67,7 +67,7 @@ The schema form is:
]
```
-`asc` / `desc` controls sort order on result enumeration. Drive currently only uses ascending storage, but the field is preserved through the contract.
+Every document meta-schema accepts only `"asc"`. Drive stores index entries in ascending order; a query chooses its own result order.
### `unique: bool`
diff --git a/book/src/drive/sum-index-examples.md b/book/src/drive/sum-index-examples.md
index 48b5509f91f..e4edfec08e8 100644
--- a/book/src/drive/sum-index-examples.md
+++ b/book/src/drive/sum-index-examples.md
@@ -18,7 +18,7 @@ The `tip` document type carries four properties (`recipient`, `amount`, `sentAt`
"properties": {
"recipient": { "type": "array", "byteArray": true, "minItems": 32, "maxItems": 32,
"position": 0, "contentMediaType": "application/x.dash.dpp.identifier" },
- "amount": { "type": "integer", "minimum": 1, "position": 1 },
+ "amount": { "type": "integer", "minimum": 1, "maximum": 4294967295, "position": 1 },
"sentAt": { "type": "integer", "minimum": 0, "position": 2 },
"note": { "type": "string", "maxLength": 280, "position": 3 }
},
diff --git a/book/src/drive/time-range-ttl.md b/book/src/drive/time-range-ttl.md
index 0a5d46ffb6e..0108fc40347 100644
--- a/book/src/drive/time-range-ttl.md
+++ b/book/src/drive/time-range-ttl.md
@@ -7,6 +7,9 @@ shares. The storage primitive underneath is grovedb's flat-subtree drop
landed in grovedb PR #849); see
[the storage section](#grovedb-dependency-flat-subtree-drop).
+A document type can also expire whole documents with its own `ttl` keyword; that is a
+different mechanism, described in [Document Time To Live](../data-model/document-ttl.md).
+
## Motivation
A `timeRange` index stores every document once per containing window, and
diff --git a/book/src/error-handling/error-codes.md b/book/src/error-handling/error-codes.md
index 83b58e34750..57a1d90d4cb 100644
--- a/book/src/error-handling/error-codes.md
+++ b/book/src/error-handling/error-codes.md
@@ -53,7 +53,7 @@ Error codes are organized into ranges that correspond to error categories and su
| 10200-10277 | Data Contract | `DataContractMaxDepthExceedError` (10200), `DuplicateIndexError` (10201), `InvalidDataContractIdError` (10204), `DataContractInvalidRequiredFieldsUpdateError` (10276), `PreProgrammedDistributionAmountOverLimitError` (10277) |
| 10350-10359 | Groups | `GroupPositionDoesNotExistError` (10350), `GroupExceedsMaxMembersError` (10354) |
| 10360-10367 | Contract Groups | `ContractGroupMembershipsOverLimitError` (10360), `InvalidContractGroupAdminsError` (10364), `InvalidContractGroupDescriptionLengthError` (10367); 10365 unassigned |
-| 10400-10422 | Documents | `DataContractNotPresentError` (10400), `DuplicateDocumentTransitionsWithIdsError` (10401), `DocumentPropertyNotDistinctError` (10419), `InvalidEncryptedPropertyShapeError` (10420), `DocumentPropertyMaxBytesExceededError` (10421), `DocumentPropertyConstraintViolatedError` (10422) |
+| 10400-10424 | Documents | `DataContractNotPresentError` (10400), `DuplicateDocumentTransitionsWithIdsError` (10401), `DocumentPropertyNotDistinctError` (10419), `InvalidEncryptedPropertyShapeError` (10420), `DocumentPropertyMaxBytesExceededError` (10421), `DocumentPropertyConstraintViolatedError` (10422), `DocumentPropertyNotGeneratedError` (10424) |
| 10450-10460 | Tokens | `InvalidTokenIdError` (10450), `TokenTransferToOurselfError` (10456) |
| 10500-10535 | Identity | `DuplicatedIdentityPublicKeyBasicError` (10500), `InvalidIdentityPublicKeyDataError` (10511) |
| 10600-10603 | State Transition | `InvalidStateTransitionTypeError` (10600), `StateTransitionMaxSizeExceededError` (10602) |
@@ -107,7 +107,7 @@ The fee category currently has a single code. The 30000 range is reserved for fu
| Range | Category | Examples |
|-------|----------|----------|
| 40000-40009 | Data Contract | `DataContractAlreadyPresentError` (40000), `DataContractIsReadonlyError` (40001), `DataContractNotFoundError` (40008) |
-| 40100-40139 | Documents | `DocumentAlreadyPresentError` (40100), `DocumentNotFoundError` (40101), `DuplicateUniqueIndexError` (40105), `DocumentActionFeeAgreementNotSetError` (40132), `DocumentActionFeeAgreementMismatchError` (40133), `DocumentActionFeeMultiplierNotToleratedError` (40134), `DocumentActionFeeModeratorsShareMismatchError` (40139) |
+| 40100-40140 | Documents | `DocumentAlreadyPresentError` (40100), `DocumentNotFoundError` (40101), `DuplicateUniqueIndexError` (40105), `DocumentActionFeeAgreementNotSetError` (40132), `DocumentActionFeeAgreementMismatchError` (40133), `DocumentActionFeeMultiplierNotToleratedError` (40134), `DocumentActionFeeModeratorsShareMismatchError` (40139), `DocumentExpiredError` (40140) |
| 40200-40217 | Identity | `IdentityAlreadyExistsError` (40200), `InvalidIdentityRevisionError` (40203), `IdentityInsufficientBalanceError` (40210) |
| 40300-40307 | Voting | `MasternodeNotFoundError` (40300), `MasternodeVoteAlreadyPresentError` (40304), `VoteChoiceNotAllowedForVotePollError` (40307) |
| 40400-40401 | Prefunded Balances | `PrefundedSpecializedBalanceInsufficientError` (40400) |
diff --git a/book/src/evo-sdk/dashpay-contact-requests.md b/book/src/evo-sdk/dashpay-contact-requests.md
index beaf9e0953e..10f1a88081d 100644
--- a/book/src/evo-sdk/dashpay-contact-requests.md
+++ b/book/src/evo-sdk/dashpay-contact-requests.md
@@ -31,7 +31,15 @@ The current DashPay contract schema requires the system field
`encryptedPublicKey` is exactly 96 bytes:
- 16 bytes: AES-CBC initialization vector
-- 80 bytes: AES-CBC ciphertext for the sender's 78-byte serialized contact xpub
+- 80 bytes: AES-CBC ciphertext for the sender's 69-byte compact contact xpub
+
+The compact xpub is the parent fingerprint (4 bytes), chain code (32 bytes)
+and public key (33 bytes), without the version, depth and child number of a
+full 78-byte BIP32 serialization. Receivers, including the reference mobile
+wallets and the Rust, Swift and Kotlin SDKs, refuse any other plaintext
+length. Both 69 and 78 bytes pad to the same 80-byte ciphertext, so the
+contract cannot catch the mistake: a request that encrypts the full 78 bytes
+is stored on chain, and the recipient's wallet then drops it.
The sender derives the contact xpub from the sender identity, recipient
identity, account, and address index. The sender then encrypts that xpub with an
@@ -99,14 +107,17 @@ function deriveSharedKey({
return crypto.createHash('sha256').update(Buffer.concat([compressedPrefix, x])).digest();
}
-function serializedXpubPayload(xpub: string): Buffer {
+function compactXpubPayload(xpub: string): Buffer {
const payload = dashcore.encoding.Base58Check.decode(xpub);
if (payload.length !== 78) {
throw new Error(`Invalid DashPay contact xpub length: ${payload.length}`);
}
- return payload;
+ // BIP32 layout: version(4) depth(1) parentFingerprint(4) childNumber(4)
+ // chainCode(32) publicKey(33). DIP-15 encrypts only the parent
+ // fingerprint, chain code and public key: 69 bytes.
+ return Buffer.concat([payload.subarray(5, 9), payload.subarray(13, 78)]);
}
function encryptContactXpub({
@@ -122,7 +133,7 @@ function encryptContactXpub({
privateKeyWif: senderEncryptionPrivateKeyWif,
publicKeyBytes: recipientDecryptionPublicKeyBytes,
});
- const payload = serializedXpubPayload(contactXpub);
+ const payload = compactXpubPayload(contactXpub);
const iv = crypto.randomBytes(16);
const cipher = crypto.createCipheriv('aes-256-cbc', aesKey, iv);
const encrypted = Buffer.concat([
diff --git a/book/src/fees/document-cost.md b/book/src/fees/document-cost.md
new file mode 100644
index 00000000000..55be9961488
--- /dev/null
+++ b/book/src/fees/document-cost.md
@@ -0,0 +1,111 @@
+# What a Document Costs
+
+`drive::document::cost` computes what creating one document costs, from the
+contract alone. The JavaScript SDKs expose it as
+`documentCreateCost(contract, documentTypeName, options, platformVersion)`,
+which is what the contract visualizer shows per document type.
+
+Amounts are in credits: 1 Dash is 100,000,000,000 credits.
+
+## Storage
+
+The storage fee is the bytes the insert adds times
+`storage_disk_usage_credit_per_byte` (27,000 credits from protocol version 14).
+It is usually almost all of what a document costs.
+
+The estimate is exact. It lists every element the insert writes (see
+[GroveDB Structure](../drive/grovedb-structure.md) for the layout), built with
+the functions the insert walkers use:
+
+- the document by id, or with `documentsKeepHistory` a tree of revisions and a
+ pointer to the newest;
+- for each index, the value trees along its properties (created only when no
+ earlier document has the value), the `[0]` terminal and the document's
+ reference in it, or an indexOnly type's entry;
+- the rows a ranked index keeps for the value in its secondary trees, one per
+ ranking axis;
+- the trees preallocated for other document types' `preallocated` indexes
+ whose entries will reference the document, charged to its creator.
+
+Each element is priced with GroveDB's byte formulas: the key with the 32-byte
+subtree prefix, the serialized element (or a fixed size standing for a tree),
+the value and node hashes, the aggregate feature of the tree it sits in (8 or
+16 bytes in a count or sum tree), and the link its parent keeps to it. A
+document create's elements carry 35 bytes of storage flags (the owner and the
+epoch); index trees carry them only when the documents are mutable, the
+contract can be deleted, or the type is indexOnly and its documents can be
+deleted.
+
+The storage depends on what is already stored, so it comes in two scenarios:
+
+- **every value new**: the first document with these index values creates
+ their trees;
+- **every value known**: a later document with the same values adds only its
+ own entries. A unique index still adds its value, which no earlier document
+ can hold, as does every tree keyed by the document's own id (a preallocated
+ index's, say) and a `ttl` document's expiration entry; a ranked index's row
+ for an existing value only moves, which GroveDB bills as replaced bytes.
+
+The estimate also splits the storage by index: the layers an index shares with
+other indexes (a common prefix of properties, paid once for the document) and
+the layers only it uses, so an index's cost on its own is its shared layers
+plus its own.
+
+The test `should_price_what_drive_charges` inserts documents into 19 contracts
+covering every index shape and requires the estimate to equal the storage fee
+Drive charged, byte for byte, with the trees already stored read from GroveDB
+before each insert.
+
+## Processing
+
+- **Exact:** verifying the signature (15,000 credits for an ECDSA key, 300,000
+ for BLS) and fetching the signing key and the identity's balance (18,000).
+- **Estimated:** the work of the writes (seeks, hashing and rewriting the path
+ to each new element in its tree and above), for an assumed number of stored
+ documents, each with its own values (1,000 by default); and the small reads
+ and writes around the insert (the document id check and the identity's
+ contract nonce). The test
+ `should_estimate_the_processing_of_the_writes_within_a_factor_of_two` holds
+ the write estimate to Drive's processing fee. Processing is a few percent of
+ a document's cost.
+- A `userFeeIncrease` raises the processing fee only.
+
+The per-document "minimum fee" of a batch (`document_batch_sub_transition`) is
+a balance check before processing, not a charge, and the unique index checks
+and the fetch of the batch's own contract are not billed.
+
+## What the contract adds
+
+- **Action fees** (`actionFees`): the create's fee, in credits, scaled by the
+ epoch's fee multiplier unless priced as fixed, paid into the contract's fee
+ pots.
+- **Token cost** (`tokenCost`): tokens transferred to the contract owner or
+ burned.
+- **Contest fund**: a contested index's vote fund (0.1 Dash, doubling past 250
+ contenders), paid when the value is contested.
+
+## Refunds
+
+Deleting a document refunds the storage fee of its flagged elements, less what
+the epochs already passed were paid: about 99.9% in the epoch it was created,
+about 95% a year later, then less each year for fifty years.
+
+## Documents with a `ttl`
+
+A document whose type declares a [`ttl`](../contract-keywords/ttl.md) is
+stored without flags and pays for its bytes by the lifetime it has left, all of
+its `ttl` when it is created, at the schedule's tier for that lifetime (from 1
+credit per byte for an hour to 26 for a week, then 34 per 788,400 seconds)
+instead of the 27,000 of storage kept for good. It also adds its entry to the
+documents expirations tree, and prepays its deletion as processing (a base
+cost, a cost per index level and one per document byte). Nothing of it is
+refunded.
+
+## Not covered
+
+- A create whose value starts a contest (a DPNS name matching the contest
+ rule, say) is stored in the contest's vote poll until the contest ends, not
+ in the index. The estimate prices an uncontested create and lists the contest
+ fund; the vote poll's storage is not priced.
+- A platform version whose insert methods differ from protocol version 14's is
+ refused: other versions write other elements for some shapes.
diff --git a/book/src/fees/overview.md b/book/src/fees/overview.md
index a771a14e7e3..94af5b6e964 100644
--- a/book/src/fees/overview.md
+++ b/book/src/fees/overview.md
@@ -52,6 +52,12 @@ Storage fees are **refundable**: when data is deleted, a portion of the original
storage fee is returned to the identity that paid it (see [Refunds](#refunds)
below).
+The documents of a type that declares a `ttl` (protocol version 14) are the exception: they
+carry no storage flags and refund nothing, their bytes are priced for the time they live, and
+their storage fees are paid out to the epochs they live in, through the lifetime storage fee
+pools, instead of over the perpetual distribution. See
+[Document Time To Live](../data-model/document-ttl.md).
+
### Processing Fees
Processing fees pay for computation that does not leave a permanent trace in
diff --git a/book/src/introduction.md b/book/src/introduction.md
index 7a3932cdfce..72728a4e873 100644
--- a/book/src/introduction.md
+++ b/book/src/introduction.md
@@ -138,6 +138,7 @@ pub trait TransactionalApplication<'a> {
pub trait BlockExecutionApplication {
fn block_execution_context(&self) -> &RwLock>;
+ fn unsigned_withdrawal_txs_by_round(&self) -> &RwLock;
}
```
diff --git a/book/src/sdk/identity-keys.md b/book/src/sdk/identity-keys.md
index 5f45d6ca6a5..1e21f48868c 100644
--- a/book/src/sdk/identity-keys.md
+++ b/book/src/sdk/identity-keys.md
@@ -90,7 +90,9 @@ security. This matters because many operations check
security levels:
- Adding/disabling other keys requires MASTER
- Credit transfers require CRITICAL (enforced via the TRANSFER purpose)
- - Document operations accept HIGH or MEDIUM depending on the contract
+ - Document operations accept CRITICAL down to the level each document type
+ requires (`signatureSecurityLevelRequirement`, HIGH by default), so MEDIUM only
+ where a type asks for it; a batch holding a token transition needs CRITICAL
3. **Which purposes allow which levels.** Not all combinations are valid for
externally added keys (i.e., keys added via identity create/update transitions):
diff --git a/book/src/serialization/document-serialization.md b/book/src/serialization/document-serialization.md
index 5f6b52230a3..f74d5e78561 100644
--- a/book/src/serialization/document-serialization.md
+++ b/book/src/serialization/document-serialization.md
@@ -104,9 +104,9 @@ The document's unique identifier, written as raw bytes. This is a 256-bit value
The identity that currently owns the document, written as raw bytes.
-### `$creatorId` (v2 only, conditional)
+### `$creatorId` (v2 and v3, conditional)
-Present only in serialization version 2, and only if the document type supports transfers (`documents_transferable`) or trading (`trade_mode != None`).
+Present only in serialization versions 2 and 3, and only if the document type supports transfers (`documents_transferable`) or trading (`trade_mode != None`).
```text
0x01 [32 bytes creatorId] — creator ID present
@@ -185,7 +185,7 @@ All numeric values use **big-endian** byte order.
| `identifier` | 32 bytes raw |
| `date` | 8 bytes big-endian f64 (when optional: `0xff` prefix + 8 bytes) |
| `array` (typed array, protocol v14) | varint element count + each element encoded exactly as a required property of the element's type (rows above): an identifier element is 32 raw bytes, an integer element takes the width its bounds give it, a fixed-size byte array element is raw, a string or variable-size byte array element has a varint length prefix. Elements never carry a presence byte |
-| `object` | Nested fields serialized recursively in their schema position order |
+| `object` | Nested fields serialized recursively, in the order the schema lists them (not sorted by `position`) |
**Note on date types**: User-property `date` fields are encoded as **f64** (8 bytes). System timestamps (`$createdAt`, `$updatedAt`, `$transferredAt`) are **u64** milliseconds. Both are 8 bytes big-endian but use different numeric representations.
diff --git a/book/src/state-transitions/validation-pipeline.md b/book/src/state-transitions/validation-pipeline.md
index 44801a7de92..859e4a31317 100644
--- a/book/src/state-transitions/validation-pipeline.md
+++ b/book/src/state-transitions/validation-pipeline.md
@@ -16,6 +16,11 @@ and again when rechecking mempool contents. It performs a lighter validation:
signature verification, basic structure, and balance checks. The goal is to filter out
obvious garbage cheaply, without doing expensive state lookups.
+A `MasternodeVote` is the exception. A block refuses a failed vote without charging anyone,
+and the proposer drops it from the block without a trace, so check_tx runs the vote's advanced
+structure and state validation as well. A vote that a block would refuse, such as a Lock vote
+on a contest without locking, is refused when it is broadcast, with its error.
+
This is implemented in
`packages/rs-drive-abci/src/execution/validation/state_transition/check_tx_verification/mod.rs`:
diff --git a/book/src/versioning/feature-versions.md b/book/src/versioning/feature-versions.md
index 0301f170b06..f2395089285 100644
--- a/book/src/versioning/feature-versions.md
+++ b/book/src/versioning/feature-versions.md
@@ -485,16 +485,19 @@ slots edited. Newer ones use struct update syntax, so that the file *is* the
diff:
```rust
-// packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/v3.rs
+// packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/v2.rs
-/// Differs from v2 in exactly one slot:
+/// Differs from v1 in two slots.
/// `document_query_helpers.compute_aggregate_mode_and_check_limit` is 2
-/// rather than 1. That is the boolean-`HAVING` routing gate. ...
-pub const DRIVE_ABCI_QUERY_VERSIONS_V3: DriveAbciQueryVersions = DriveAbciQueryVersions {
+/// rather than 0: the ranked and boolean-`HAVING` routing gate. ...
+pub const DRIVE_ABCI_QUERY_VERSIONS_V2: DriveAbciQueryVersions = DriveAbciQueryVersions {
document_query_helpers: DriveAbciDocumentQueryHelperVersions {
compute_aggregate_mode_and_check_limit: 2,
},
- ..DRIVE_ABCI_QUERY_VERSIONS_V2
+ data_contract_query_helpers: DriveAbciDataContractQueryHelperVersions {
+ latest_versions_read: 1,
+ },
+ ..DRIVE_ABCI_QUERY_VERSIONS_V1
};
```
@@ -564,7 +567,7 @@ version/
v1.rs .. v10.rs
drive_abci_query_versions/
mod.rs
- v0.rs .. v3.rs
+ v0.rs .. v2.rs
drive_abci_withdrawal_constants/
mod.rs # DriveAbciWithdrawalConstants (parameters, not method versions)
v1.rs .. v3.rs
diff --git a/book/src/versioning/platform-version.md b/book/src/versioning/platform-version.md
index e7317d3b924..32e7db207f5 100644
--- a/book/src/versioning/platform-version.md
+++ b/book/src/versioning/platform-version.md
@@ -171,7 +171,7 @@ pub const PLATFORM_V14: PlatformVersion = PlatformVersion {
methods: DRIVE_ABCI_METHOD_VERSIONS_V10, // changed: records the per-block total credits history
validation_and_processing: DRIVE_ABCI_VALIDATION_VERSIONS_V10, // changed: contested-index cross-check + refersTo validation
withdrawal_constants: DRIVE_ABCI_WITHDRAWAL_CONSTANTS_V3, // changed: prune bound for the total credits history
- query: DRIVE_ABCI_QUERY_VERSIONS_V3, // changed: ranked + boolean-HAVING routing gate
+ query: DRIVE_ABCI_QUERY_VERSIONS_V2, // changed: ranked + boolean-HAVING routing gate
checkpoints: DRIVE_ABCI_CHECKPOINT_PARAMETERS_V1,
},
dpp: DPPVersion {
diff --git a/book/src/versioning/versioned-dispatch.md b/book/src/versioning/versioned-dispatch.md
index ba303a9a835..7a59a994be1 100644
--- a/book/src/versioning/versioned-dispatch.md
+++ b/book/src/versioning/versioned-dispatch.md
@@ -687,6 +687,27 @@ stage of the validation pipeline reading the constants in
its initial version is rejected with a `StateTransitionNotActiveError` rather
than an unknown-version dispatch error.
+Genesis content is the other place a `protocol_version` comparison is the
+intended shape. `create_genesis_state` runs once per chain, under the protocol
+version the chain is born at. Mainnet and testnet were born at protocol
+version 1 and replay generation 0, which stays frozen. Generation 1 is selected
+only for chains born at protocol version 9 or later, and every such chain is a
+devnet, a local network or a test suite that is created again for each
+release, so no live node reproduces its genesis. When a protocol version adds
+a system contract or other genesis content, it goes into
+`create_genesis_state_v1` behind `if platform_version.protocol_version >= N`
+(document history at 13, app connect and moderation charters at 14), and a
+chain that already exists gets the same content from its
+`transition_to_version_N` rung. A new genesis generation would add code for no
+replay benefit. Never edit generation 0.
+
+The Drive helpers that build the initial state structure follow the same rule
+for the same reason: they run once, at chain creation, under the chain's
+initial protocol version, and a chain that already exists gets the same trees
+from its upgrade rung. `add_initial_withdrawal_state_structure_operations`
+adds the withdrawal sum trees behind `>= 4` and the credit history trees behind
+`>= 14`; replaying mainnet's genesis at protocol version 1 takes neither branch.
+
## Rules
**Do:**
diff --git a/docs/dashpay/BLOCK_SPEC.md b/docs/dashpay/BLOCK_SPEC.md
deleted file mode 100644
index 9770ea01b13..00000000000
--- a/docs/dashpay/BLOCK_SPEC.md
+++ /dev/null
@@ -1,107 +0,0 @@
-# DashPay ignore — cross-device design + DoS / social-graph-leak analysis
-
-Status: the single-device **"Block sender"** design this file originally carried was
-**superseded** by the shipped local-only **Ignore** (per-sender, reversible;
-`ignored_senders`, `ignore_sender`/`unignore_sender`, applied in `changeset/apply.rs`).
-That feature is documented in `SPEC.md` (G5) and `SYNC_CORRECTNESS_SPEC.md`; the
-single-device Block design — its state/persistence/UI/test plan and the review
-resolutions specific to it — is no longer reproduced here.
-
-This file is retained **only** for the two forward-looking pieces Ignore does not yet
-cover:
-
-- **(a) Cross-device ignore** — how to make ignore sync across a user's devices via a
- single owner-scoped, self-encrypted blocklist, and the privacy reason it is **not**
- carried via `contactInfo` (§1).
-- **(b) DoS / social-graph-leak analysis** — the fetch-cost / flood analysis and the
- countable-index social-graph leak that any query-level DoS filter must avoid (§2).
-
-Owner: platform-wallet / swift-sdk.
-Relates to: `SPEC.md` (G5, the shipped Ignore), `SYNC_CORRECTNESS_SPEC.md`,
-`CONTACTINFO_FORMAT_SPEC.md`.
-
----
-
-## 1. Cross-device ignore — a self-encrypted blocklist, NOT `contactInfo`
-
-Ignore is per-device local state today; you re-ignore a sender on each device. Making
-it sync is a **future** item on the contract / governance track — not built.
-
-**Why not `contactInfo`.** The tempting reuse — carry an ignore flag in the
-`contactInfo.privateData` blob we already sync — **breaks the DIP-15 ≥2-contacts
-unlinkability gate** and is rejected. An ignore targets a *non-established* sender, so
-carrying it would mean creating a `contactInfo` **about a non-contact**. That document's
-public existence + `$createdAt` correlates with the inbound `contactRequest` (via the
-public `userIdCreatedAt` index) to re-identify *who* you ignored: `encToUserId` is
-encrypted so *who* is hidden, but the doc's *existence/count* is not. The
-"`displayHidden` is precedent" argument is a false equivalence — `displayHidden` rides a
-document that exists anyway (an established contact), whereas an ignore-of-a-non-contact
-*creates* the leaking document. It is also mechanically blocked today:
-`set_contact_info_with_external_signer` → `set_contact_metadata` hard-requires an
-established contact, and the apply side drops non-established `contactInfo`.
-
-**The design if it ships.** A **single owner-scoped, self-encrypted blocklist
-document** — one document the owner encrypts to themselves (same key family as
-`contactInfo`'s `privateData`, but a single owner-private list, not per-contact and not
-gated by the 2-contact rule). Every device reads and applies it, so ignore (and
-optionally decline) apply everywhere. Costs: each edit is a document write (credits);
-it reveals only *that* a blocklist exists plus an edit count — **not** one document per
-ignored victim. Its update timing should be conflated with normal profile edits so it
-does not leak the per-sender existence/count. This is a contract change on the later
-governance track, and the metadata-leak analysis above must be settled before building
-it.
-
-## 2. DoS / spam, and the social-graph leak a countable index would create
-
-### 2.1 Fetch model, and why an ignore can't cut fetch cost
-
-The received-request query is keyed by recipient:
-
-```
-where toUserId == me, order_by $createdAt, limit: 100
-```
-
-An ignore is a **local read-filter applied after fetch**: the index has no
-`sender NOT IN (…)` axis and Sybil senders are unpredictable, so an ignore cannot avoid
-the fetch + GroveDB proof-verify cost of an incoming request — it only hides it once
-fetched.
-
-**Threat: a sender (or a funded Sybil swarm) creates many requests.** Invalid ones are
-the worst — they fail parse/validation but still cost fetch + proof-verify + parse. The
-only built-in deterrent is **economic**: each `contactRequest` costs the sender
-platform credits. Spam isn't free, but it isn't prevented. A naive `limit: 100,
-start: None` re-fetch also lets a flood of ≥100 junk requests **bury** legitimate ones
-past the first page.
-
-**Mitigation — incremental fetch (high-water).** Track the newest `$createdAt` seen per
-identity and query `WHERE toUserId == me AND $createdAt > high_water`, paginating
-forward: each request is fetched exactly once, pagination can't bury legit requests past
-100, and ignore/decline become one-time-on-first-sight. This bounds steady-state work to
-O(new requests per sweep). It does **not** stop the *first* fetch of a request from a new
-sender (impossible without server-side sender exclusion), but nothing in the protocol
-can. The existing `userIdCreatedAt` index `[toUserId, $createdAt]` already serves this
-range-after-equality query, so incremental fetch needs **no** contract change. *(This
-high-water incremental fetch has since shipped — see `SYNC_CORRECTNESS_SPEC.md` and the
-DIP-15 §8.8/§8.12 row in `DIP_CONFORMANCE_GAPS.md`.)*
-
-### 2.2 The trap: a countable `[toUserId, $ownerId]` index leaks the inbound social graph
-
-A natural-looking next step is a **countable** index on the recipient→sender axis so the
-wallet can answer "how many pending requests do I have" / "is one sender flooding me"
-from a count proof **without fetching documents**:
-
-```
-byRecipientSender = [{ toUserId: asc }, { $ownerId: asc }] // countable — DO NOT ship as drafted
-```
-
-(The `$ownerId` of a `contactRequest` *is* the sender.) This is a **social-graph leak**
-and must not ship in that form. Platform count / group-by proofs are **public, not
-recipient-private**, and return cleartext `{sender_id → count}`. A countable
-`[toUserId, $ownerId]` therefore lets *anyone* scrape "who contacted recipient R, with
-counts" in O(log n) — the inbound social graph, in the clear.
-
-**Resolution:** drop the per-sender `GROUP BY $ownerId` axis. At most keep an aggregate
-`COUNT(*) WHERE toUserId == me` (a single number, for a pending-request badge), which
-reveals only a total and not per-sender edges. Any real query-level DoS filter that
-excludes ignored/rejected senders *before* fetching is a contract change (DIP /
-maintainer coordination), and this graph-exposure analysis must be carried into that DIP.
diff --git a/docs/dashpay/CONTACTINFO_FORMAT_SPEC.md b/docs/dashpay/CONTACTINFO_FORMAT_SPEC.md
deleted file mode 100644
index 0ea08e454cc..00000000000
--- a/docs/dashpay/CONTACTINFO_FORMAT_SPEC.md
+++ /dev/null
@@ -1,306 +0,0 @@
-# contactInfo `privateData` — DIP-15 varint format (migrate off CBOR)
-
-Status: **IMPLEMENTED** (2026-06-18) — DIP-15 varint codec in `crypto/contact_info.rs`,
-byte-vector + compat tests. (The tolerant minor-version decode stays available for a
-future additive field, but **ignore state does NOT ride contactInfo** — R1 found
-that leaks who you ignored; ignore is local-only, cross-device via a future encrypted
-profile field. See the R1 item in the backlog, dashpay/platform#4020.)
-Owner: platform-wallet / platform-encryption
-Relates to: Spec 2 (Ignore, adds `relationshipState`), `BLOCK_SPEC.md`,
-`CONTACTINFO_FORMAT_SPEC.md` Appendix A.
-
-## Format decision: DIP-15 varint, NOT CBOR (2026-06-18)
-
-`contactInfo.privateData` is an **opaque encrypted byteArray** — the registered
-contract validates only its **length** (`byteArray:true, minItems:48,
-maxItems:2048` in `dashpay.schema.json`); the field description's "…encoded as an
-array in cbor" is **advisory documentation, not a structural constraint**. The
-plaintext inside the AES-256-CBC ciphertext is therefore a writer/reader
-convention we are free to choose — and we choose **DIP-15**, the authoritative
-protocol spec, so we interop with DIP-15-compliant clients (the reference
-`dash-wallet` / `kotlin-platform` will follow the DIP when it implements
-contactInfo). **No contract change is needed** (length-only validation accepts any
-48..2048-byte ciphertext). No client decodes contactInfo today, so this is a free
-window: we set the de-facto format and it matches the DIP.
-
-> An earlier pass briefly "reconciled" this the other way (keep CBOR, per the
-> schema *description* + `CONTACTINFO_FORMAT_SPEC.md` Appendix A). That over-weighted an advisory
-> description as binding. Corrected: the contract enforces length only, DIP-15 is
-> authoritative — use varint.
-
-This is **Spec 1** of the DashPay-privacy track. (The minor-version forward-compat
-seam — §3 — remains for any future additive field. Note: ignore state is **not**
-carried here — R1 found a per-sender `contactInfo` leaks who you ignored, so ignore
-is local-only with cross-device deferred to a future encrypted `profile` field.)
-
----
-
-## 1. Problem
-
-Our `contactInfo.privateData` codec (`crypto/contact_info.rs::encode/decode_private_data`)
-emits a **CBOR array** `[aliasName, note, displayHidden, padding?]`. **DIP-15
-defines a different format** (verified against `github.com/dashpay/dips/dip-0015.md`,
-§"Contact Info" / §"Encrypting Private Data"):
-
-- **Serialization:** "the private data should be serialized in the same way as
- done for **Dash message data**" (dip-0015.md:811) — i.e. the Bitcoin/Dash
- protocol binary format (var-int-length-prefixed strings/arrays), **not CBOR**.
-- **Fields (v0), in order:**
- | # | Field | Type | Encoding |
- |---|-------|------|----------|
- | 0 | `version` | uInt32 | `major << 16 \| minor` (dip-0015.md:771) |
- | 1 | `aliasName` | String | var-int length + UTF-8 |
- | 2 | `note` | String | var-int length + UTF-8 |
- | 3 | `displayHidden` | uInt8 | 1 byte |
- | 4 | `acceptedAccounts` | array | var-int count + u32s (dip-0015.md:805) |
-- **Crypto:** AES-256-CBC with the `rootEncryptionKey/(2^16+1)'/idx'` derived key
- (we already do this — only the *plaintext serialization* changes).
-
-**Our gaps vs DIP-15:** (a) CBOR instead of Dash-message varint; (b) **no
-`version` field**; (c) **no `acceptedAccounts`**.
-
-**Why now (the one cheap window):** verified 2026-06 that **no client decodes
-`contactInfo.privateData` today** — `android-dashpay` has no `ContactInfo` class
-(the schema is bundled as JSON only); `dash-wallet` has only a `// TODO: choose
-the contactRequest based on the ContactInfo.accountRef value`. So there is **no
-reader to break.** When `dash-wallet` implements its TODO it will follow DIP-15
-(varint), not our CBOR — so if we don't align now, the two clients won't interop.
-We're the only writer; fix the wire format while it's free.
-
-## 2. Goal
-
-- Replace the CBOR codec with the **DIP-15 Dash-message varint** serialization,
- including the `version` field and `acceptedAccounts`.
-- Adopt DIP-15's **major/minor version forward-compat** model.
-- **Define** (not yet populate) `reject` / `block` fields as a **minor-version
- extension**, so a later spec can sync reject/block via contactInfo without
- another format change.
-- No behavior change to alias/note/hidden; pure wire-format + versioning.
-
-## 3. DIP-15 versioning model (verbatim, because it drives everything)
-
-dip-0015.md:771-776: `version = major << 16 | minor`.
-- **Major** change = **incompatible**: a client that doesn't understand the major
- version **discards the whole contactInfo**.
-- **Minor** change = "most likely additional fields": an un-updated client
- "should still be able to parse the first fields … and **ignore data past the
- final field known in the version**."
-
-Consequence (this answers the "won't old clients break?" question): **adding
-`reject`/`block` is a MINOR bump** → a DIP-15-v0 reader parses `version …
-acceptedAccounts` and ignores our trailing fields. **No breakage.** Only a major
-bump locks old readers out. So:
-- our baseline = **major 0, minor 0** (DIP-15 v0 fields exactly);
-- our reject/block extension = **major 0, minor 1** (appended fields);
-- decoders MUST be **tolerant**: read the fields the known minor defines, ignore
- trailing bytes; on an unknown **major**, discard.
-
-## 4. The reject/block fields — DEFINED but NOT ADOPTED (R1, resolved 2026-06-18)
-
-> **Resolution.** This field was the proposed cross-device carrier for
-> reject/block. It is **not implemented** and is **not part of the shipped DIP-15
-> codec** (which carries only `aliasName` / `note` / `displayHidden` /
-> `acceptedAccounts`). Per R1, a `contactInfo` about a *non-established* sender
-> leaks *who* you ignored (the timing-correlation argument below), so **Ignore is
-> local-only** (Spec 2) and cross-device sync is deferred to a future **encrypted
-> field on the `profile` document** (contract / governance track) — NOT to
-> `contactInfo`. The design below is retained for reference only.
-
-Appended after `acceptedAccounts`, present from **minor 1** (design only — unused):
-
-| # | Field | Type | Meaning |
-|---|-------|------|---------|
-| 5 | `relationshipState` | uInt8 | 0 = active, 1 = declined, 2 = blocked (extensible) |
-
-Rationale for a single `relationshipState` byte over two bools: declined/blocked
-are mutually-exclusive states of one relationship; one enum is smaller, avoids
-the "both set" ambiguity, and extends cleanly (e.g. 3 = muted). `displayHidden`
-(field 3) stays as-is for backward DIP-15 compat; `relationshipState` is the
-richer superset we read first when present.
-
-**Scope boundary (critical):** this spec only **defined** the field + its
-encoding. *Whether and how* a `contactInfo` is created to carry it — especially
-for a **non-established** declined/blocked sender — was the **privacy question
-(R1 from the block review)**, **RESOLVED (2026-06-18): not via `contactInfo` at
-all.** Ignore is local-only (Spec 2); cross-device goes through an encrypted
-`profile` field later. Kept for context:
-
-> A `contactInfo` *about a non-contact* is a brand-new on-chain document whose
-> existence + `$createdAt` can be timing-correlated with the inbound
-> `contactRequest` (public `userIdCreatedAt` index) to re-identify *who* you
-> blocked — even though `encToUserId` is encrypted, and the ≥2-contacts gate
-> (dip-0015.md:697-699) can't cover a non-contact. **Spec 2 resolved this: a
-> per-sender `contactInfo` is leaky (above) and even a single owner-scoped list
-> on `contactInfo` still signals "an ignore happened", so ignore is kept
-> local-only and cross-device is deferred to an encrypted `profile` field whose
-> update timing is conflated with ordinary profile edits.** This format spec is agnostic to that
-> choice — it just provides the field.
-
-## 5. Padding / 48-byte floor
-
-The contract validates `privateData` at **48–2048 bytes** (dip-0015.md:727).
-Our CBOR codec appends a padding element to reach 48; the Dash-message format has
-no field for that, and trailing padding would collide with the "ignore data past
-the final known field" rule (a future reader could mis-read padding as a higher-
-minor field). **Decision needed (Q-pad):**
-- (a) Pad the **ciphertext region** only — encode the exact fields, then rely on a
- reserved **`padding` length-prefixed byte field** placed *last in every minor*
- and documented as "ignore"; or
-- (b) define the floor purely as an encryption-layer concern (pad plaintext to ≥
- the size that yields 48-byte ciphertext, inside AES-CBC, with the pad length
- recoverable) so the field stream itself carries no padding.
-Recommend **(a) an explicit trailing `padding` var-bytes field** that is *always
-the final field* and always skipped — it's self-describing (var-int length) so
-"ignore trailing" still works, and it's the closest analog to today's behavior.
-
-## 6. Migration / compatibility of *existing* data
-
-contactInfo docs are immutable on-chain. Any docs **we** already wrote (CBOR,
-no version field) become unreadable by the new varint decoder.
-- DashPay is **pre-release** (not on mainnet); existing CBOR docs are
- testnet/devnet UAT artifacts → **acceptable to abandon** (they'll simply fail
- to decode and be skipped, same as a foreign-root doc today).
-- Do **not** build a CBOR↔varint dual-reader unless we find we must preserve
- specific test data. (Open question Q-dual.)
-- The **local** SwiftData/SQLite mirror is rebuilt from chain on sync, so no
- local migration is needed beyond the decoder swap.
-
-## 7. Implementation surface
-
-- `packages/rs-platform-wallet/src/wallet/identity/crypto/contact_info.rs`:
- rewrite `encode_private_data` / `decode_private_data` to the Dash-message varint
- format (var-int string/array helpers; `version` first; tolerant decode that
- stops at the known-minor field count and skips trailing). Keep the AES-CBC layer.
-- `ContactInfoPrivateData` struct: add `version: u32` (or major/minor accessors),
- `accepted_accounts: Vec`, and `relationship_state: u8` (minor ≥ 1).
- `displayHidden` stays. (Note: the in-memory struct already flows through
- `set_contact_metadata(ContactInfoPrivateData)` after the recent refactor.)
-- No FFI/Swift signature change (privateData is opaque bytes across the boundary);
- only the bytes' internal layout changes.
-
-## 8. Test plan
-
-- **Round-trip:** encode→decode every field incl. empty/None strings, empty and
- non-empty `acceptedAccounts`, `relationshipState` 0/1/2.
-- **Forward-compat:** a **minor-0** decoder reading **minor-1** bytes parses
- v0 fields and ignores `relationshipState` (the DIP-15 guarantee) — pin it.
-- **Major-incompat:** a decoder reading an unknown **major** discards (returns
- None / skips), not a partial parse.
-- **Vector:** if any DIP-15 / reference test vector for privateData exists, match
- it byte-for-byte (none found in dashj/android-dashpay; we may be authoring the
- first — note that).
-- **Floor:** encoded output is ≥ 48 bytes after padding (Q-pad), ≤ 2048.
-
-## 9. Open decisions
-
-- **Q-pad** — explicit trailing `padding` field (recommended) vs encryption-layer
- padding.
-- **Q-dual** — abandon existing CBOR docs (recommended, pre-release) vs build a
- CBOR/varint dual-reader.
-- **Q-state** — single `relationshipState: uInt8` (recommended) vs separate
- `declined`/`blocked` flags.
-- **Q-minor-now** — define `relationshipState` (minor 1) in *this* spec/PR, or
- ship the pure DIP-15-v0 alignment first (minor 0) and add the field in Spec 2?
- (Leaning: ship v0 alignment here; add the field in Spec 2 where it's used —
- keeps this PR a clean wire-format fix.)
-
----
-
-
-
-## Appendix A — contactInfo wire conventions (research, 2026-06-12)
-
-Source-verified findings for implementing the DashPay `contactInfo`
-document (M3 task 13). Full citations at the bottom.
-
-### Decisive finding: no reference client implements contactInfo
-
-DashSync-iOS (`DSBlockchainIdentity.m` + Identity models), dashj /
-android-dpp / kotlin-platform, and dash-shared-core contain **zero**
-contactInfo creation or encryption code. DIP-15 + the deployed
-dashpay-contract schema are the only authoritative sources, and **this
-repo's implementation sets the de-facto wire convention.** There is no
-cross-client byte-compatibility constraint — only self-consistency and
-schema validity.
-
-### Conventions adopted (CONFIRMED unless marked INFERRED)
-
-#### Key derivation (DIP-15)
-
-The "root encryption key" is the identity's **registered ENCRYPTION
-key** (DIP-11 purpose 1); `rootEncryptionKeyIndex` is that key's id on
-the identity. Two child keys are derived from its extended form in the
-owner's HD tree (hardened CKDpriv):
-
-```text
-encToUserId key: rootEncryptionKey / 65536' / derivationEncryptionKeyIndex' (2^16)
-privateData key: rootEncryptionKey / 65537' / derivationEncryptionKeyIndex' (2^16 + 1)
-```
-
-The 2^16 offset is DIP-15's explicit "discount other potential
-derivations" choice. The AES-256 key is the raw 32-byte child private
-key scalar (INFERRED — no hash step is specified anywhere; matches how
-contactRequest ECDH consumes key material).
-
-`derivationEncryptionKeyIndex` is sequential per `$ownerId` starting at
-0 (one per contactInfo document; the unique index is
-`($ownerId, rootEncryptionKeyIndex, derivationEncryptionKeyIndex)`).
-
-#### encToUserId (DIP-15, verbatim justification in the DIP)
-
-`AES-256-ECB(toUserId)` — exactly 32 bytes = two blocks, **no IV, no
-padding**. ECB is sound here because the plaintext is itself a SHA-256
-output and the key is never reused for other purposes.
-
-#### privateData
-
-> **CORRECTION (2026-06-18): use DIP-15 varint, not CBOR.** The conclusion
-> below ("the deployed schema description wins → CBOR") over-weighted an
-> advisory note. The contract validates `privateData` by **length only**
-> (`byteArray`, 48–2048); its "array in cbor" text is documentation, NOT an
-> enforced structural constraint. The encrypted plaintext format is a free
-> writer/reader convention, so we follow **DIP-15** (the authoritative protocol
-> spec) with `version`/varstr/`acceptedAccounts`. See the spec above.
-
-`IV(16) ‖ AES-256-CBC(plaintext)` — IV prepended (INFERRED from the
-`encryptedPublicKey` convention; DIP-15 doesn't state placement for
-this field).
-
-Plaintext (~~CBOR~~ → **DIP-15 varint**, per the correction above): the original
-analysis adopted a **CBOR array `[aliasName, note, displayHidden]`** per the
-deployed schema's field description — positional, with CBOR `null` for absent
-strings (INFERRED). DIP-15 prose instead describes Bitcoin-varint "Dash message
-data" with `version` + `acceptedAccounts` — and that is what we now use (the
-schema enforces length only, so there's no conflict and no contract change).
-
-#### Privacy rule (DIP-15, spec-only — no client enforces it today)
-
-> "A client should not transmit a contact info document for a user to
-> the network until that user has at least two established contacts."
-
-Enforced at the publish gate: with <2 established contacts the local
-state still updates; the document write is deferred until the rule is
-satisfied.
-
-### Discrepancy table (DIP-15 prose vs deployed schema)
-
-| Question | DIP-15 prose | Deployed schema |
-|---|---|---|
-| Plaintext format | Bitcoin varint stream | CBOR array |
-| Fields | version, aliasName, note, displayHidden, acceptedAccounts | aliasName, note, displayHidden |
-| version | uInt32 present | absent |
-| acceptedAccounts | array of uInt32 | absent |
-
-### Sources
-
-- DIP-0015 (dashpay/dips) — derivation offsets, ECB/CBC modes, privacy rule
-- dashpay-contract `schema/v1/dashpay.schema.json` — CBOR-array description,
- unique index, 48–2048B bounds
-- DIP-0011 (key purposes), DIP-0013 (identity key paths), DIP-0009
- assignments (15'/16' are incoming-funds / auto-accept — no contactInfo path)
-- dashsync-iOS Identity models, android-dpp, kotlin-platform,
- dash-shared-core — checked: no contactInfo implementation anywhere
-- rs-dpp `lib.rs` `RootEncryptionKeyIndex` / `DerivationEncryptionKeyIndex`
- type aliases
diff --git a/docs/dashpay/DASHPAY_STATE_ENCAPSULATION_SPEC.md b/docs/dashpay/DASHPAY_STATE_ENCAPSULATION_SPEC.md
deleted file mode 100644
index 072b95d1024..00000000000
--- a/docs/dashpay/DASHPAY_STATE_ENCAPSULATION_SPEC.md
+++ /dev/null
@@ -1,463 +0,0 @@
-# DashPay State Encapsulation — extract `ManagedIdentity`'s DashPay fields into `DashPayState`
-
-Status: draft, rev 2 (review must-fixes folded)
-Scope: `packages/rs-platform-wallet` (+ mechanical re-paths in `rs-platform-wallet-ffi`;
-test-helper construction in `rs-platform-wallet-storage`). No on-disk format change, no FFI
-ABI change, no Swift change. Follow-up to PR #3841 — lands as its own PR after #3841 merges.
-
-Origin: review question on PR #3841 — "Having dashpay stuff right in identity wallet and
-identity manager and managed identity is mixing too much different stuff… shall we have a
-separate dashpaywallet and somehow encapsulate dashpay stuff from common identity things?"
-
-> **Review outcome (rev 2).** Four independent reviewers (feasibility, scope, adversarial
-> failure-modes, Rust/domain-fit) audited rev 1 against the code. Scope verdict:
-> right-sized; two-tier design and two-commit staging earn their keep. The load-bearing
-> corrections folded here: **(MF-1)** rev 1 undercounted the FFI cold-load restore path —
-> it raw-writes **six** DashPay fields including all three relationship maps
-> (`ffi/persistence.rs:4064/4067/4070`), so the relationship-map `apply_*` methods must be
-> `pub`, not `pub(crate)`, and the boundary claim is recalibrated from "sealed" to
-> "raw writes impossible; invariant-bypassing writes are named `apply_*` and auditable".
-> **(MF-2)** a `pub dashpay` field is bypassable by whole-value replacement
-> (`managed.dashpay = Default::default()` / `mem::take` compiles anywhere and silently wipes
-> the high-water cursors) — the field itself is now private with a `dashpay()` borrow
-> getter and per-field Tier B `_mut` accessors, and **no** whole-struct `dashpay_mut()`.
-> **(MF-3)** rev 1 answered only one of the three sites the origin comment names; §0 now
-> maps the design to all three honestly, and an optional facade-level `DashPayView`
-> (zero-cost borrowing namespace — materially different from the twice-reverted owned
-> facade) is added as decision point Q1. Plus: getters live on `DashPayState` itself;
-> `dashpay_` field-name stutter dropped; `pub(super)` replaces the long scoped-visibility
-> path; the in-crate test-fixture raw writes are inventoried and budgeted (§5); D4's method
-> doc keeps the caller-side half of the cursor contract explicit.
-
----
-
-## 0. What the origin comment names, and what this spec covers
-
-The PR comment names three sites. Honest mapping:
-
-1. **`ManagedIdentity` (state)** — the worst offender and **this spec's target**: 12 of 19
- fields are DashPay social state, all `pub`, invariants enforceable only by convention.
-2. **`IdentityWallet` (network facade)** — the *largest* mixing by volume (~10.2k lines of
- DashPay ops vs ~6.6k identity-core across the `network/` impl files), but already
- file-split by concern with documented layering (`network/mod.rs`). Two owned-facade
- splits were tried and deliberately reverted on this branch (§4.1). What this spec offers
- there is the optional zero-cost `DashPayView` namespace (§3 D8, decision Q1) — call-site
- visibility without a second handle.
-3. **`IdentityManager` / manager layer** — already clean: the manager holds buckets + a
- location index with no DashPay logic; DashPay sync orchestration is already its own
- coordinator (`manager/dashpay_sync.rs`). No change proposed.
-
-## 1. Problem
-
-`ManagedIdentity` (`src/wallet/identity/state/managed_identity/mod.rs`) mixes two concerns
-in one flat, fully-`pub` struct:
-
-- **Identity-core**: `identity`, `identity_index`, `wallet_id`, `status`, `dpns_names`,
- `contested_dpns_names`, two sync block-times.
-- **DashPay social state — 12 fields**: `established_contacts`, `sent_contact_requests`,
- `incoming_contact_requests`, `ignored_senders`, `auto_accept_verify_failed`,
- `dashpay_rescan_triggered`, `dashpay_profile`, `dashpay_payments`, `contact_profiles`,
- `high_water_received_ms`, `high_water_sent_ms`, `pending_contact_crypto`.
-
-Three concrete costs today (all verified against the code):
-
-1. **No boundary.** Nothing in the type answers "what is DashPay vs identity-core"; the
- distinction lives in field-comment prose. Any future extraction (or reasoning about one)
- starts from zero.
-2. **Invariants are bypassable — and one already lives outside the state layer.** All fields
- are `pub`, so the auto-establish invariant (reciprocal request ⇒ established contact), the
- `AUTO_ACCEPT_VERIFY_FAILED_CAP` eviction, and the ignore-emits-both changeset rule are
- upheld only by the convention of calling the right method. Worse, **high-water cursor
- monotonicity is enforced nowhere in the state layer**: the compare-and-advance rule
- (`advance_if_unchanged`, `network/contact_requests.rs:822`) is a free function in network
- code writing the fields raw (`:1363`, `:1370`). A miss reintroduces the lost-unignore bug
- that function's doc-comment describes.
-3. **The FFI crate reads 9 public fields directly** inside handle closures
- (`ffi/src/dashpay_profile.rs:150` et al.) and **raw-writes six fields on the cold-load
- restore path**: `ignored_senders` (`ffi/persistence.rs:3758`), `dashpay_payments`
- (`:3806`), `contact_profiles` (`:3900`), and — via `apply_contact_rows`
- (`:3961-4077`, production load path) — all three relationship maps (`:4064/:4067/:4070`).
- The compiler offers no help distinguishing "legit restore write" from "invariant bypass".
-
-## 2. Current architecture (facts the design relies on)
-
-From a three-agent inventory of the state layer, all access sites, and the persistence
-coupling, plus four review passes re-verifying the cites:
-
-- **The core↔DashPay coupling is narrow.** No method in
- `state/managed_identity/{contacts,contact_requests,identity_ops,sync}.rs` mutates both an
- identity-core field and a DashPay field in the same call. Coupling is exactly: (a)
- `snapshot_changeset()` → `IdentityEntry::from_managed` (`changeset/changeset.rs:332-349`)
- reads 4 DashPay scalar fields on every core-field persist; (b) the two constructors
- initialize both groups; (c) `disable_keys` reads `wallet_id`/`identity_index` alongside a
- snapshot. DashPay mutators read only the immutable `self.id()`.
-- **Mutation methods are already the norm.** Network code calls state-mutation methods 53×;
- direct production field writes outside the owner module: ~11 in `network/` (high-water ×2,
- `dashpay_rescan_triggered` ×1, `established_contacts.get_mut` ×3,
- `set_dashpay_profile`/`mark_auto_accept_verify_failed` pass-throughs, `contact_profiles`
- ×1), 12 in `state/manager/apply.rs` (changeset replay), 9 in `wallet/apply.rs` (changeset
- replay), and **9 in the FFI crate** (6 restore-path raw writes + 3 via methods). Test
- fixtures add raw writes in `wallet/apply.rs:562-567/:1348-1351`, `network/` test modules,
- `ffi/tests/test_data/mod.rs:283-286/:304`, and `contact_workflow_tests.rs:289`
- (`established_contacts.get_mut`).
-- **Persistence never serializes `ManagedIdentity` itself** — it derives `Debug, Clone` only.
- Three persistence tiers cover the 12 fields:
- - `IdentityEntry` scalar snapshot (serde-derived flat struct, `changeset.rs:270-323`):
- `dashpay_profile` (merge: LWW), `dashpay_payments` (extend, LWW per txid),
- `contact_profiles` (extend, LWW per contact), `ignored_senders` (union).
- - `ContactChangeSet` (`changeset.rs:634-663`): the three relationship maps + the
- `ignored`/`unignored` tombstone pair.
- - `PlatformWalletChangeSet.pending_contact_crypto_{added,cleared}` (`changeset.rs:1189/1193`)
- for the deferred-crypto queue.
- - **In-memory only, never persisted**: `dashpay_rescan_triggered`,
- `auto_accept_verify_failed`, `high_water_received_ms`, `high_water_sent_ms`.
-- **The FFI ABI is keyed off the flat entry types** (`IdentityEntryFFI::from_entry`,
- `ContactRequestFFI::from_*` — `#[repr(C)]` with pinned sizes), NOT off `ManagedIdentity`'s
- layout. Regrouping `ManagedIdentity` cannot move a single FFI byte as long as the entry
- types stay flat.
-- **Two restore paths construct/populate `ManagedIdentity` outside its methods**: the
- boot/load path (in-crate: pre-built identities arrive via `IdentityManagerStartState`,
- consumed at `manager/load.rs:100` — no field-level construction; FFI loader:
- `ManagedIdentity::new` + restore writes, `ffi/persistence.rs:3692-3700`) and the
- changeset-replay path (`state/manager/apply.rs`, `wallet/apply.rs` contacts block,
- `apply_established_contact`).
-- **One external struct-literal construction** exists in `rs-platform-wallet-storage`
- (`schema/identities.rs:206-237`) — test-gated (`#[cfg(any(test, feature="__test-helpers"))]`).
- It defaults the three relationship maps (loaded separately from the contacts table) and
- populates `ignored_senders` by wholesale clone.
-- **The WalletPersister is a method parameter**, not a `ManagedIdentity` field. Non-persisting
- mutators return a `ContactChangeSet` for the caller to store. A sub-struct inherits the same
- two patterns unchanged.
-- History: a separate DashPay surface was tried and deliberately reverted twice on this
- branch — `914e244401` folded `wallet/dashpay/` under `identity/` (duplicate files,
- bidirectional refs, "where does this live?"), `cdd0da880e` merged the `DashPayWallet`
- facade into `IdentityWallet` (two FFI handles, two clones per op, straddling ops like
- `accept_contact_request`).
-
-## 3. Design
-
-### D1 — `DashPayState` struct, privately owned by `ManagedIdentity`
-
-New file `state/managed_identity/dashpay.rs` (a child module of `managed_identity`, sibling
-of the four impl files — verified module chain makes `pub(super)` fields visible to all of
-them and to nothing outside `managed_identity/`):
-
-```rust
-/// Per-identity DashPay social state: the DashPay-contract layer
-/// (contacts, requests, profile, payments, deferred crypto) carried by
-/// a `ManagedIdentity` on top of its identity-core fields.
-#[derive(Debug, Clone, Default)]
-pub struct DashPayState {
- // -- Tier A: guarded (sibling-module fields, mutate via methods) --
- pub(super) established_contacts: BTreeMap,
- pub(super) sent_contact_requests: BTreeMap,
- pub(super) incoming_contact_requests: BTreeMap,
- pub(super) ignored_senders: BTreeSet,
- pub(super) auto_accept_verify_failed: BTreeSet<[u8; 32]>,
- pub(super) high_water_received_ms: Option,
- pub(super) high_water_sent_ms: Option,
-
- // -- Tier B: open (plain data / caches, no cross-field invariant) --
- pub profile: Option,
- pub payments: BTreeMap,
- pub contact_profiles: BTreeMap,
- pub rescan_triggered: BTreeSet,
- pub pending_contact_crypto: Vec,
-}
-```
-
-On `ManagedIdentity`, the 12 flat fields are replaced by a **private** field
-`dashpay: DashPayState` (private-to-`managed_identity`: visible in `mod.rs` and all child
-impl files, invisible outside — this also closes the whole-value-replacement bypass, see
-D3). The existing field doc-comments (several are load-bearing, e.g. the in-memory-only
-rationale on `rescan_triggered` and `auto_accept_verify_failed`) move verbatim. The
-`dashpay_` name prefix is dropped inside the struct (no stutter behind `dashpay()`).
-
-**Tier assignment rationale.** Tier A = every field with a cross-field or temporal invariant:
-the three relationship maps (auto-establish; rotation-supersede; both-exist precheck),
-`ignored_senders` (ignore must emit `removed_incoming` + `ignored` together),
-`auto_accept_verify_failed` (CAP eviction — already method-only today), the two high-water
-cursors (compare-and-advance; only writer besides the sweep is `unignore_sender`'s rewind).
-Tier B = independent per-key caches where a raw insert cannot corrupt sibling state, and where
-the replay/restore paths and profile/payment recorders already write directly today.
-`pending_contact_crypto` stays Tier B: its dedup invariant lives in the free function
-`upsert_pending_contact_crypto` shared with the changeset apply path, and its drain uses
-owned snapshots — capturing that in methods is real scope with no bypass bug on record
-(possible follow-up, out of scope here).
-
-### D2 — mutation methods stay on `ManagedIdentity`; signatures unchanged
-
-Every existing mutation/query method (`add_sent_contact_request`,
-`add_incoming_contact_request`, `accept_incoming_request`, `ignore_sender` /
-`unignore_sender`, `set_contact_metadata`, `apply_rotated_incoming_request`,
-`mark_auto_accept_verify_failed`, `should_enqueue_auto_accept`, `set_dashpay_profile`,
-`record_dashpay_payment`, …) keeps its receiver, name, signature, and persister-threading
-pattern; bodies reach through `self.dashpay.*`. The 53 existing method call sites don't
-change. This is deliberately NOT a `DashPayState`-methods design for mutations: they need
-`self.id()` and snapshot access, and moving them would churn every call site for zero
-invariant gain.
-
-### D3 — read access: one `dashpay()` borrow + getters on `DashPayState`
-
-`ManagedIdentity` gains exactly one read accessor:
-
-```rust
-pub fn dashpay(&self) -> &DashPayState
-```
-
-Tier B fields are `pub`, so all reads flow `managed.dashpay().payments`,
-`managed.dashpay().contact_profiles`, … Tier A fields get borrow getters **on
-`DashPayState` itself** (next to the fields): `established_contacts()`,
-`sent_contact_requests()`, `incoming_contact_requests()`, `ignored_senders()`, and by-value
-`high_water_received_ms()` / `high_water_sent_ms()` (`Option` is `Copy`).
-`auto_accept_verify_failed` gets NO getter — the two existing query methods on
-`ManagedIdentity` (`is_auto_accept_verify_failed`, `should_enqueue_auto_accept`) cover every
-reader. Existing query helpers (`is_sender_ignored`, `established_contact(&id)`,
-`prior_sent_account_reference`, …) stay on `ManagedIdentity` unchanged.
-
-Read sites (~30 network, ~18 FFI, ~55 test assertions) re-path mechanically
-(`managed.established_contacts` → `managed.dashpay().established_contacts()`). Verified: no
-name collisions with existing `ManagedIdentity` methods; the same-named methods on
-`IdentityWallet` are a different type.
-
-Tier B **writes** get per-field mut accessors on `ManagedIdentity`: `payments_mut()`,
-`contact_profiles_mut()`, `rescan_triggered_mut()`, `pending_contact_crypto_mut()`, and
-`set_profile_raw` is unnecessary (`set_dashpay_profile` already exists; the replay paths use
-the mut accessors). There is deliberately **no whole-struct `dashpay_mut()`** and the
-`dashpay` field is private: `managed.dashpay = DashPayState::default()`, `mem::take`, and
-`mem::swap` — whole-value replacements that would silently wipe Tier A state including the
-cursors — do not compile outside `managed_identity/`.
-
-`established_contact_mut(&id) -> Option<&mut EstablishedContact>` **stays, promoted to
-`pub`** (documented escape hatch; `contact_workflow_tests.rs:289` — an external compilation
-unit — needs it, as do three in-crate network sites that mutate contact sub-fields then
-persist a hand-built `ContactChangeSet`). Sealing per-contact sub-field mutation is
-follow-up scope; the boundary claim here is deliberately modest — see D5.
-
-### D4 — capture the high-water invariant (the one real behavior-adjacent move)
-
-`advance_if_unchanged` + `advance_high_water` (free fns,
-`network/contact_requests.rs:814-832`) move onto `ManagedIdentity` as the ONLY write path
-for the cursors:
-
-```rust
-/// Compare-and-advance: advance the received-direction cursor to
-/// `max_fetched` (never below its current value) ONLY if the cursor
-/// still holds `snapshot` — the value read at sweep start. A mid-sweep
-/// `unignore_sender` rewind (reset to None) must not be clobbered by a
-/// stale sweep max, or the un-ignored sender stays invisible until a
-/// cold restart.
-///
-/// Caller contract (unchanged from the free fn): invoke only when the
-/// paginate exhausted without error AND every ingest reached disk —
-/// fetch/persist-success gating stays at the call site.
-pub fn advance_high_water_received(&mut self, snapshot: Option, max_fetched: Option);
-pub fn advance_high_water_sent(&mut self, snapshot: Option, max_fetched: Option);
-```
-
-The two raw network writes (`:1363`, `:1370`) become calls; `unignore_sender`'s rewind stays
-internal to the state layer. The moved invariant is CAS + monotonicity **only**; the
-fetch-succeeded/persist-succeeded gating remains caller-side convention, stated in the doc.
-Semantics bit-identical: the snapshot stays a caller-supplied param, so the two-guard
-interleaving with a concurrent un-ignore is unchanged.
-
-### D5 — replay/restore writes become named `apply_*` methods
-
-The replay and cold-load paths currently write Tier A fields raw. They get intent-named
-methods that skip business invariants **by design** (establishment/ignore decisions were made
-before persist; replay must reproduce state, not re-decide it). Visibility follows the
-callers — the FFI crate restores relationship maps in production, so these are `pub`:
-
-- `pub fn apply_sent_contact_request(&mut self, ContactRequest)` /
- `apply_incoming_contact_request` — for `wallet/apply.rs:197-222` and the FFI loader
- (`ffi/persistence.rs:4067/:4070`) + FFI fixtures (`ffi/tests/test_data/mod.rs:304`).
-- `apply_established_contact` — exists (`contact_requests.rs:621`), promoted
- `pub(crate)` → `pub` (FFI loader `:4064`, fixtures `:283-286`). Parity note: its
- remove-both-pending-sides is a provable no-op on the cold-load path — `apply_contact_rows`'
- match arms emit exactly one of {established, sent, incoming} per contact into fresh maps.
-- `pub fn apply_ignored_sender(&mut self, Identifier)` / `apply_unignored_sender` — for
- `wallet/apply.rs:255/265`, the FFI restore write (`ffi/persistence.rs:3758`), and the
- storage-crate test helper. **Implementer note:** `state/manager/apply.rs:71/:115/:143` and
- the storage helper write the *whole set* (`.extend()` union / fresh-object assign) — loop
- `apply_ignored_sender` per element. Equivalent because `:115/:143` sit on fresh-insert
- branches where the set is constructor-empty (assign ≡ insert-loop) and `:71` is already a
- union; pin this equivalence with a test.
-- `pub(crate) fn apply_removed_sent(&mut self, &Identifier)` / `apply_removed_incoming` —
- only `wallet/apply.rs:225/:230` removes.
-- `state/manager/apply.rs`'s remaining writes touch Tier B only (`dashpay_profile` →
- `profile`, `dashpay_payments` → `payments`, `contact_profiles`) — they use the Tier B mut
- accessors, as do the FFI restore writes to `payments`/`contact_profiles`.
-
-Each `apply_*` doc-comment states why it bypasses the invariant and who may call it.
-
-**Boundary claim, calibrated** (this is what the refactor actually buys): raw *field* writes
-to Tier A are compile errors outside `managed_identity/`; invariant-*bypassing* writes still
-exist but are named `apply_*`, greppable, and auditable — the compiler cannot distinguish a
-new illegitimate `apply_*` caller from a restore path. The capability-level seal is real for
-exactly two things: the high-water cursors (D4 — the only public write path enforces CAS)
-and whole-value replacement of the DashPay group (D3). Everything else is
-naming-and-audit, which is the honest, proportionate win.
-
-### D6 — construction
-
-`DashPayState` derives `Default` (every field defaults empty/None — true today for both
-constructors and the cold-load path; all 12 field types are `Default`-able).
-`ManagedIdentity::new` / `new_out_of_wallet` set `dashpay: DashPayState::default()`. The FFI
-loader keeps `ManagedIdentity::new` + `apply_*`/Tier-B-mut writes. The storage test helper
-(`storage/schema/identities.rs:206-237`) switches its literal to
-`dashpay: DashPayState::default()` semantics via constructor + an `apply_ignored_sender`
-loop (its relationship maps are already defaulted there; contacts load separately).
-
-### D7 — persistence mapping updates (no wire change)
-
-`IdentityEntry::from_managed` (lives in `crate::changeset` — reads Tier A through the `pub`
-getters, Tier B through `dashpay()`). `IdentityEntry`, `ContactChangeSet`, all merge
-functions, the SQLite schema, and every `#[repr(C)]` FFI mirror are **untouched**. The
-on-disk format and FFI ABI provably cannot change: nothing serializes `ManagedIdentity`
-(derives `Debug, Clone` only), and no FFI struct or `const` size assert is edited. Only
-observable representation delta: `Debug` output nests the group under `dashpay:` — no test
-parses `Debug` output (verified).
-
-### D8 — OPTIONAL: facade-level `DashPayView` namespace (decision Q1)
-
-The origin comment's eye was on `IdentityWallet` — where DashPay ops outnumber identity-core
-ops by lines ~10.2k to ~6.6k. The twice-reverted design was a second **owned** facade (two
-handles through FFI, two clones per op). A **borrowing view** has none of those costs:
-
-```rust
-pub struct DashPayView<'a, B: TransactionBroadcaster + ?Sized>(&'a IdentityWallet);
-
-impl IdentityWallet {
- pub fn dashpay(&self) -> DashPayView<'_, B> { DashPayView(self) }
-}
-```
-
-The DashPay op definitions (already file-split: `contact_requests.rs`, `contacts.rs`,
-`contact_info.rs`, `payments.rs`, `profile.rs`) move their `impl IdentityWallet` blocks to
-`impl DashPayView<'_, B>` — one route per op, no forwarding shims. Call sites become
-`wallet.identity().dashpay().send_contact_request(…)`. FFI **function signatures are
-unchanged** (they re-path internally); Swift is untouched. Cost: a mechanical re-path of the
-FFI + internal DashPay call sites and the sync coordinator; zero new state, zero clones.
-This is the piece that makes the DashPay/identity boundary visible at every call site — but
-it is severable: commits 1–2 stand alone if this is declined.
-
-## 4. Alternatives rejected
-
-1. **Separate owned `DashPayWallet` facade (the PR comment's literal suggestion).** Tried and
- reverted twice on this very branch (§2 history). The ops straddle the boundary
- (`accept_contact_request` = identity signing + DashPay docs; payments = core broadcaster;
- contact crypto = identity DIP-9/14 keys), so a second owned facade re-creates the same
- handle with a different name, re-splits ops that straddle, and re-introduces FFI
- handle-juggling. DashPay is a *layer on the identity aggregate*, not a sibling domain.
- Rejected on evidence, not taste. (The borrowing view in D8 is the surviving kernel of
- this idea — namespace without ownership.)
-2. **Separate DashPay store keyed by identity id** (e.g. `IdentityManager.dashpay:
- BTreeMap`). Splits one aggregate into two maps that must stay
- key-synchronized through add/remove/apply/load; every combined read becomes a two-map
- join; the changeset apply and both restore paths get a second lookup + orphan mode. All
- cost, and the "is it one thing?" answer is unchanged — a DashPay state without its
- identity is meaningless.
-3. **Extension-trait split of `IdentityWallet`** (`DashPayOps` trait). Pure cosmetics: state
- stays mixed, callers add trait imports, and the network layer is already file-split by
- concern. The borrowing view (D8) achieves the namespacing without the trait ceremony.
-4. **Full lockdown (all 12 fields private, no `_mut` escape hatch).** Forces dedicated
- methods for per-contact sub-field mutation (3 network sites), the profile fetch-cache
- writer, payments recorder internals, and the pending-crypto upsert/drain — roughly
- doubles the new-method surface to protect fields with no cross-field invariants and no
- observed bypass bugs. Poor cost/benefit now; Tier A→B promotion later is cheap.
-5. **Stop after commit 1 (all-`pub` regroup, no encapsulation).** Fixes cost 1 of §1
- (boundary/naming) but leaves costs 2–3 untouched: the high-water cursors stay raw
- network writes guarding a documented lost-unignore bug, and the FFI keeps 6
- indistinguishable raw restore writes. The ~14 new methods in commit 2 are earned by
- exactly those two costs.
-6. **Docs only (comment banner grouping the fields).** Zero enforcement; the next reviewer
- asks the same question.
-
-## 5. Migration & staging
-
-Own PR, based on `v4.1-dev` **after #3841 merges** (this touches the same files as #3841's
-tail; doing it inside would bloat an already-huge diff and re-trigger full re-review).
-
-Commits, each independently green:
-
-1. **Mechanical regroup.** Introduce `DashPayState` with ALL fields temporarily `pub` (and
- the `dashpay` field `pub`); move the 12 fields (renaming the three `dashpay_`-prefixed
- ones); re-path every access (`managed.X` → `managed.dashpay.X`). No visibility change, no
- method change. Compile-error-driven; behavior-identical by construction. Also deletes
- the orphaned 0-byte `state/managed_identity/tests.rs` left by `914e244401`.
-2. **Encapsulate.** Apply tier visibilities + private `dashpay` field; add `dashpay()`,
- Tier A getters, Tier B mut accessors, `advance_high_water_*`, the `apply_*` family;
- convert the network + FFI + replay sites; move the two high-water free fns into the
- state layer with their tests. **Test-fixture conversions budgeted here** (not
- "mechanical re-paths"): `wallet/apply.rs:562-567/:1348-1351` (insert → `apply_*`),
- `network/contact_requests.rs` fixture sites (~3411-3418 flag write →
- `established_contact_mut`; ~3560-3564 `ignored_senders.clear()` — no direct equivalent,
- becomes clone-keys + `apply_unignored_sender` loop), `network/payments.rs:1535/1610/2537`
- + `network/contact_info.rs` helper (insert → `apply_*`), FFI fixtures
- (`ffi/tests/test_data/mod.rs`), `contact_workflow_tests.rs:289`
- (→ `established_contact_mut`).
-3. **(Optional, decision Q1) `DashPayView` facade namespace** per D8.
-
-Rollback story: each commit reverts independently of the ones before it.
-
-## 6. Failure modes & risks
-
-- **Missed access site** → compile error (loud, the mechanism working as intended). Zero
- runtime discovery.
-- **High-water semantics drift** (the only logic that *moves*): mitigated by porting the
- existing free-fn unit tests unchanged, plus new method-level tests written to pass against
- the free fn's behavior BEFORE the move (kill-the-mutant check: `snapshot != current` must
- leave the field untouched).
-- **Replay-path behavior change**: `apply_*` methods must reproduce today's raw writes
- exactly. Verified caller-side semantics that must NOT move into the methods:
- `wallet/apply.rs` keys inserts off `entry.request.recipient_id`/`sender_id`, warns on
- orphan inserts but is silent on orphan removes, orders inserts-before-removes and
- unignore-after-ignore (un-ignore wins) — all stay in `wallet/apply.rs`. The
- `ignored_senders` assign-vs-loop equivalence (D5) gets a pinning test.
-- **Borrow-checker fallout**: adversarial pass verified all existing sites compile under the
- new surface (the three `get_mut` sites touch only the contact + persister while the `&mut`
- is live; loops over Tier A maps are read-only in-body; mutating sites collect inputs before
- taking `&mut`). New sites holding a getter-returned borrow across a `&mut` call will fail
- to compile — clone first in tests.
-- **FFI restore path**: `apply_*` parity is exact (raw inserts have no side effects;
- `apply_established_contact`'s extra removes are a no-op on fresh maps — D5).
-- **Merge risk with in-flight DashPay work**: pure mechanics; land in a quiet window.
- Orthogonal to the pending-contact-crypto follow-ups (that spec moved the field *onto* the
- identity; this one only re-paths it).
-- **Residual escape hatches — the honest list**: `established_contact_mut` (`pub`), Tier B
- `pub` fields + mut accessors, and the **`pub apply_*` family itself** (any crate can call
- `apply_ignored_sender` instead of `ignore_sender`, skipping the tombstone contract — same
- exposure as today's `pub` fields, but now named and greppable). The boundary claim is
- D5's calibrated version, not "all DashPay state is sealed".
-
-## 7. Test & verification plan
-
-- **Existing suites are the harness** (behavior-preserving refactor): full
- `rs-platform-wallet` lib tests (the auto-establish, rotation, ignore/unignore, CAP-eviction,
- persist-before-commit pins all keep passing untouched), `contact_workflow_tests`,
- `rs-platform-wallet-ffi` lib + integration tests.
-- **New unit tests** (written against the CURRENT free-fn behavior first, then the method):
- `advance_high_water_{received,sent}` — advance, never-below, `None`-snapshot,
- mid-sweep-rewind-preserved; `apply_ignored_sender` loop ≡ wholesale-assign parity;
- `apply_sent/incoming_contact_request` parity with today's `wallet/apply.rs` raw inserts
- (including the no-auto-establish property of the replay path).
-- **Visibility is its own test — for what it actually seals**: after commit 2, a Tier A raw
- *field* write or a whole-`dashpay` replacement outside `managed_identity/` is a compile
- error. (`apply_*` misuse is not compiler-catchable — that's the calibrated D5 claim.)
-- **Local CI mirror**: `cargo clippy --workspace --all-features` + `cargo fmt --check --all`
- (targeted `-p` builds miss feature-gated callers).
-- **iOS**: rebuild the xcframework + run the existing FFI persistence round-trip tests; no
- Swift source change expected (assert: `git diff --stat` on `packages/swift-sdk` is empty).
-
-## 8. Open questions (decision points for the PR author)
-
-1. **Include commit 3 (`DashPayView` facade namespace, D8)?** Recommendation: yes — it is
- the only part of this spec that changes what the origin comment's author *sees* at the
- `IdentityWallet` layer, and it is zero-cost at runtime. But commits 1–2 deliver the
- state-layer value standalone; declining Q1 drops D8 with no other edits.
-2. Should `payments` be Tier A? `record_dashpay_payment` has rollback-on-persist semantics,
- but the FFI restore + overlay paths write it raw; sealing it means two more `apply_*`
- methods. Proposed: keep Tier B now.
-3. DPNS fields (`dpns_names`, `contested_dpns_names`): once `DashPayState` lands, their
- loose placement becomes the next obvious question. Position: deliberately-separate
- follow-up using the identical pattern ("dashpay first, dpns next"), not scope here.
diff --git a/docs/dashpay/DIP15_INVITATIONS_SPEC.md b/docs/dashpay/DIP15_INVITATIONS_SPEC.md
deleted file mode 100644
index 1f2152e994b..00000000000
--- a/docs/dashpay/DIP15_INVITATIONS_SPEC.md
+++ /dev/null
@@ -1,835 +0,0 @@
-# DashPay Invitations (DIP-13 sub-feature 3') — Implementation Spec
-
-> **Status:** SHIPPED on PR #4041 (2026-07-14) — create + claim + reclaim + persistence + UI,
-> three review-fix rounds folded, funded testnet e2e green (TEST_PLAN DP-12..19, `AI_QA/QA004`).
-> The original design pass (2026-07-08, §1–§14 below) is kept as rationale; **§0 records where
-> the as-built implementation deliberately diverged.** Where §0 and a later section disagree,
-> §0 wins.
-
-Tracked as the "NEXT" item in the DashPay backlog (dashpay/platform#4020); called out in
-`SPEC.md` Milestone 5 and `DIP_CONFORMANCE_GAPS.md`.
-
----
-
-## 0. As-built delta (supersedes the marked sections below)
-
-1. **Link envelope = the LEGACY query format, not the §6 binary blob (supersedes §6, §7).**
- The 2026-07-13 legacy-compat rework (owner decision; contract in §0A) replaced the hand-rolled versioned payload with the
- query form shared with dash-wallet Android / dashwallet-iOS, so links are field-level
- cross-claimable: `dashpay://invite?du=&assetlocktx=&pk=&islock=`
- `[&display-name=…][&avatar-url=…]` (also parses `https://invitations.dashpay.io/applink?…`).
- Emit strict / parse lenient. Consequences:
- - The link carries the funding **txid**, not the embedded proof → **claim-by-fetch**: the
- invitee refetches the funding tx by txid (bounded retry for DAPI propagation lag, both
- byte orders), reconstructs the proof, and selects the credit output by matching
- `voucher_credit_script(pk)`.
- - **No expiry field on the wire** — the §5.1/§8/§10 "claim refuses a past-expiry link"
- mechanism does not exist in the as-built claim; `expiry_unix` survives only as inviter-side
- local display metadata. The economic bounds are the amount caps.
- - **No inviter identity id on the wire** (`inviter_id` always zeroed) — the contact bootstrap
- resolves the id from the `du` username via DPNS at claim time. `InviterInfo.username` is
- `Option`: a display-name/avatar-only link is metadata-only (`has_inviter == true`,
- `inviter_username == nil`, no bootstrap).
- - Amount is not on the wire (claim preview shows "—").
-2. **Claim accepts ChainLock invites too (amends §5.1).** `islock` absent or literal `"null"`
- ⇒ a `ChainAssetLockProof` is reconstructed (requires the funding tx to be chain-locked).
- Create still emits only InstantSend links — a slow-IS ChainLock fallback at create is
- rejected as a *link* but the funded lock is recorded first and stays reclaimable.
-3. **Amounts (amends §5/§8/§9):** `MIN_INVITATION_DUFFS = 300_000` (0.003 DASH — a smaller
- voucher can fund neither a claim nor a register-reclaim, discovered by funded e2e),
- `MAX_INVITATION_DUFFS = 26_000_000` (0.26 DASH), Swift default **0.03 DASH**.
-4. **Persistence as-built (amends §4.2):** the `InvitationChangeSet` flows through
- `PlatformWalletPersistence::store()` to each backend — the SQLite backend's
- `V003__invitations` table, and on iOS the FFI `on_persist_invitations_fn` bridge into the
- SwiftData `PersistentInvitation` model (SwiftData is the UI source; no Rust rehydrate; §0B). Persist failures are signaled end-to-end
- (nonzero callback → rolled-back round → `create_invitation` errors), not best-effort.
-5. **Durability + ordering hardening (review rounds 1–3; the per-finding log lives in the
- PR #4041 review threads + commit messages):** the pre-broadcast gate persists **and flushes** the
- invitation funding-index pool (aborting before broadcast on failure); creation refuses
- non-durable backends (`PlatformWalletPersistence::persists_durably()`); the funded-asset-lock
- flow is split so the invitation record is persisted immediately **after broadcast, before the
- proof wait** — an interrupted create can no longer orphan a funded voucher.
-6. **Reclaim shipped (extends §1 scope):** an unclaimed voucher is recovered as identity
- **credits** (top-up an existing identity or register a new one; the L1 amount was
- OP_RETURN-burned). Already-consumed handling is classified via the persisted
- `reclaimInFlight` marker: marker unset ⇒ provably a foreign claim (neutral "already
- claimed"); marker set ⇒ **explicitly ambiguous** (the marker proves only that a local
- consume attempt started, not that it landed — a racing claim is indistinguishable, so the
- row resolves to the conservative terminal `Claimed` with an ambiguity message, never an
- inferred `Reclaimed`) — see `AI_QA/QA004` step 6 for the exact classifier arms.
-7. **QA contract as-built:** TEST_PLAN §4.10 rows **DP-12..DP-19** (not just DP-12..15) +
- `AI_QA/QA004_invitation_reclaim.md`; funded e2e evidence recorded there.
-
----
-
-## 0A. As-built link envelope & legacy interop (absorbs the legacy-compat spec)
-
-The interop contract is **field-level parity with the live legacy wallets, emit
-strict/canonical, parse leniently** — exactly as tolerantly as the live Android wallet.
-Byte-for-byte parity is NOT the contract (the two legacy wallets differ in param order and in
-scheme/host). The on-chain primitive and derivation path (`m/9'/coin'/5'/3'/idx'`) are
-identical across all three wallets — no consensus change.
-
-### 0A.1 Wire format
-
-**Emit (canonical, what we produce):**
-```text
-dashpay://invite
- ?du= # required to emit; optional on parse
- &assetlocktx=
- &pk=
- &islock= # or omit (see below)
- [&display-name=]
- [&avatar-url=]
-```
-- Parse **by field name, order-independent**; accept **both** the `dashpay://invite` scheme
- and the `https://invitations.dashpay.io/applink` host (iOS legacy links use the latter).
-- **`pk`**: WIF, **compressed** flag set (the credit-output hash uses the *compressed*
- pubkey — wrong compression ⇒ wrong `hash160` ⇒ claim fails), network byte `0xCC` mainnet /
- `0xEF` testnet-family.
-- **`assetlocktx`**: emit lowercase big-endian display hex; on claim parse leniently — try
- as-given, then **retry byte-reversed** on a fetch miss (old iOS links are little-endian).
-- **`islock`**: OPTIONAL, with two absence forms — param missing **and the literal string
- `"null"`** (Android emits `"null"` for a chainlock-confirmed invite). Absent/`"null"` ⇒
- reconstruct a **`ChainAssetLockProof`** at claim, never reject. The hex is not
- self-describing: decode as the modern deterministic **ISDLOCK**; the ancient
- non-deterministic ISLOCK is unrepresentable in rust-dashcore and fails closed (documented
- limitation — no live producer exists).
-- **Validity (lenient superset of both wallets):** require `assetlocktx` + `pk`
- present/non-blank; never reject solely on a missing `du` or missing/`"null"` `islock`.
-
-### 0A.2 Claim-by-fetch
-
-The link carries the funding **txid**, not a proof, so claim reconstructs it (mirrors Android
-`TopUpRepository.obtainAssetLockTransaction`):
-1. Fetch the tx by `assetlocktx` via `Sdk::get_transaction` (bounded retry/backoff for DAPI
- propagation lag; reversed-retry per §0A.1).
-2. Fail-fast guards: fetched txid matches `assetlocktx` (either byte order); when an islock is
- present, `islock.txid == fetched tx.txid`.
-3. **Derive `output_index` by script match** — scan the fetched tx's `credit_outputs` for the
- output whose `script_pubkey` == `voucher_credit_script(pk)`; never hard-code index 0.
-4. Build `InstantAssetLockProof` (islock present) or `ChainAssetLockProof` (absent/`"null"`;
- requires the tx to be chain-locked), then submit through the **unchanged**
- `put_to_platform_and_wait_for_response_with_private_key`.
-
-Consensus enforces pk↔output, islock↔tx, and identity_id↔outpoint — all fail closed; the
-local guards are fast-fail UX + correct index selection, not theft prevention.
-
-### 0A.3 Consequences of the legacy format
-
-- **No inviter identity id on the wire** — only `du`. `InviterInfo = {username?, display_name?,
- avatar_url?}` and the invitee resolves the inviter's id from `du` via DPNS at
- contact-bootstrap. A `du`-less link is metadata-only (`has_inviter == true`,
- `inviter_username == nil`, no bootstrap possible).
-- **No expiry on the wire** — the pre-network staleness gate is gone; the real bounds are the
- amount caps + reclaim. The inviter-side local record keeps expiry for display only.
-- **Amount is not on the wire** — the claim preview shows "—" pre-fetch.
-
-### 0A.4 Amounts (onboarding tiers)
-
-`MIN_INVITATION_DUFFS = 300_000` (0.003 DASH, == Android `DASH_PAY_INVITE_MIN`; a smaller
-voucher can fund neither a claim nor a register-reclaim — found by funded e2e).
-`MAX_INVITATION_DUFFS = 26_000_000` (0.26 DASH). Swift create default **0.03 DASH** = identity
-+ a normal DPNS name (Android `DASH_PAY_FEE`). The cap covers the contested/premium tier as
-well (0.25, Android `DASH_PAY_FEE_CONTESTED`), with the remaining margin for the create/claim
-fees; the claim path is amount-agnostic, so nothing further gates a contested-tier invite.
-
-### 0A.5 Transport
-
-The custom `dashpay://` scheme is the shipped, first-class transport (QR / share sheet /
-in-person). The legacy wallets' AppsFlyer OneLink wrapper is **externally blocked** (Android
-team creds; brand domain + template) and tracked separately (#4096-adjacent); note that
-OneLink discloses the plaintext `pk` to AppsFlyer server-side — an accepted, documented
-regression vs a self-contained link, bounded by the amount cap + reclaim. The custom scheme's
-same-device interception limitation is documented in `Info.plist` + §6.1.
-
----
-
-## 0B. As-built persistence & reclaim (absorbs the Swift-persistence spec)
-
-### 0B.1 Persistence bridge
-
-`InvitationChangeSet` (structurally an `asset_locks`-style `BTreeMap` upserts +
-`BTreeSet` removals) flows through `PlatformWalletPersistence::store()` to each backend: the
-SQLite backend's `V003__invitations` table, and on iOS the **push-callback FFI bridge**
-(`on_persist_invitations_fn` → `persistInvitationsCallback` → SwiftData
-`PersistentInvitation`), mirroring the asset-lock wiring. Key properties:
-- **SwiftData is the UI source; push-only, no Rust→Swift rehydrate.** A SwiftData wipe loses
- only list *visibility* — never funds or key re-derivability (`funding_index` re-derives the
- voucher key).
-- **Persist failures are signaled, never swallowed:** a skipped write returns nonzero from the
- callback, failing the (invitation-only) `store()` round and surfacing an error from
- `create_invitation` instead of reporting a voucher that never reached SwiftData.
-- **Outpoint key seam:** both the upsert and the removal path derive the unique
- `outPointHex` via `PersistentAssetLock.encodeOutPoint` verbatim (key-form drift is pinned by
- `InvitationPersistenceTests`).
-
-### 0B.2 Reclaim
-
-The invitation's DASH is **burned into an `OP_RETURN`** at create time — the credit output
-exists only in the tx payload as a Platform-side authorization, never as an L1 UTXO — so
-"reclaim" means: **the inviter consumes the still-unclaimed voucher into a Platform identity
-of their own, recovering the value as credits** (mechanically, claiming your own invitation).
-UI copy always says "recovered as identity credits", never "DASH returned".
-
-- **Primitive:** consume the tracked lock via
- `FromExistingAssetLock { out_point, consume_invitation_voucher: true }` — the inviter's own
- signer re-derives the voucher key at `9'/coin'/5'/3'/funding_index'` internally (no key
- export). Two user-picked targets: **top-up an existing identity** or **register a new
- one**. The `consume_invitation_voucher` flag is the reclaim flow's **explicit
- authorization**: every generic resume/top-up path passes `false` and the funding resolver
- refuses `IdentityInvitation`-typed locks, so a shared voucher can never be silently
- consumed into an unrelated local identity (the Swift resumable-registrations surface also
- excludes `fundingTypeRaw == 3` rows).
-- **Race / already-consumed:** no L1 double-spend exists (no shared UTXO); Platform
- deterministically rejects the second consume
- (`IdentityAssetLockTransactionOutPointAlreadyConsumed` — the loser wastes only an ST fee).
- The Swift side classifies via the persisted `reclaimInFlight` marker, which is saved
- (required — the consume may not run on a failed save) only immediately before the on-chain
- consume: marker unset ⇒ provably the invitee claimed first (row → `Claimed`, neutral
- "This invitation was already claimed." — claimant not named); marker set ⇒ **explicitly
- ambiguous** — the marker proves only that a local consume attempt started, not that it
- landed (a racing claim between crash and retry is indistinguishable), so the row resolves
- to the conservative terminal `Claimed` with an ambiguity message, never an inferred
- `Reclaimed` (`Reclaimed` is written only by a success observed in-flow). The local
- "is not tracked" resume-guard failure with the marker set is surfaced as an explicit
- ambiguity error (status unchanged — there is no on-chain proof of consumption at all).
- The decision is the pure, unit-tested `classifyReclaimFailure(error:hadPriorReclaimInFlight:)`
- seam; see `AI_QA/QA004` step 6 for the verified classifier arms.
-- **Status lifecycle:** `Reclaimed`/`Claimed` are written by the Swift UI on the local row
- (SwiftData is the UI source; create is the only Rust emitter).
-
----
-
-## 1. Problem & goal
-
-DashPay onboarding today assumes the new user already **has** a Dash identity (which
-requires L1 Dash to fund the ~0.0002 DASH asset lock that registers it). That is a
-chicken-and-egg wall for inviting a friend who has never touched Dash: they can't receive a
-payment (no identity → no contact) and can't register an identity (no funds).
-
-**DIP-13 "Identity Invitation Funding keys" solves this.** An existing user (the *inviter*)
-pre-funds an asset lock at a dedicated derivation sub-feature, hands the one-time private key
-+ the asset-lock proof to a friend (the *invitee*) as a link, and the invitee registers
-**their own new identity** funded by that voucher — no L1 Dash required on the invitee's
-side. The invitation optionally bootstraps the DashPay contact in the same act (the invitee's
-contact request to the inviter carries a DIP-15 `autoAcceptProof`, so it auto-establishes).
-
-**Goal:** implement invitation **create** (inviter) and **claim** (invitee) end-to-end across
-`rs-platform-wallet` + `rs-platform-wallet-ffi` + `swift-sdk` + `SwiftExampleApp`, with unit
-+ integration tests, a testnet funded e2e, and QA-contract scenarios.
-
-### Non-goals
-- **No byte-for-byte interop with the production iOS/Android DashWallet invitation link.** We
- can't drive those builds in this environment (same constraint the auto-accept spec accepted:
- iOS-first, DIP-faithful where the DIP defines a format, normative-for-us where it is silent).
- The **on-chain** artifacts (asset lock, IdentityCreate, contactRequest) are consensus formats
- and *are* interoperable; only the off-chain **link envelope** is ours. See §7 for the interop
- decision once the reference format is confirmed.
-- **No new on-chain artifact.** Invitations reuse the existing AssetLock special-tx, the
- IdentityCreate transition, and a plain contactRequest.
-- **No auto-accept bearer key in the invitation (v1).** The contact-bootstrap is a *normal*
- contact request (see §2 design change); no `dapk` is embedded.
-- **No invitation for identity-less inviters in v1** beyond the pure funding voucher: the
- contact-bootstrap requires the inviter to hold a registered identity. A voucher from an
- identity-less funder still works as pure onboarding funding; it just carries no inviter to
- contact.
-- **Advisory expiry, not consensus revocation.** The voucher key controls an on-chain asset
- lock that never expires; the payload's `expiry` is an **advisory** bound (the claim UI refuses
- a stale link; the inviter is prompted to reclaim). True "revocation" is the inviter racing to
- *reclaim* the unclaimed lock (a race it can lose if the link already leaked — §8 Finding 6). A
- dedicated revoke UI is a follow-up.
-
----
-
-## 2. The model — two roles, three on-chain acts
-
-1. **Inviter (Bob, has funds + identity).**
- - Derives a one-time ECDSA **voucher key** at the DIP-13 invitation path
- `m/9'/coin'/5'/3'/funding_index'` (sub-feature `3'`).
- - Builds + broadcasts an **asset lock** paying `amount` duffs to that key, and waits for an
- **InstantSend** proof (§5.1 — fast, self-contained; a short IS-scoped expiry covers
- staleness).
- - **Optionally ticks "send a contact request back to me"** — if checked, the link carries the
- inviter's identity id + username; if not, it's a pure funding voucher.
- - Emits a `dashpay://invite?...` link carrying: **voucher private key**, **asset-lock
- proof (IS)**, **advisory expiry**, and *(if opted in)* **inviter identity id + username +
- display name**. The voucher key is re-derivable from `funding_index`, so it is **never
- persisted**; only the funding index + outpoint are tracked (for recovery + status).
-2. **Invitee (Carol, no funds).**
- - Opens the link → decodes (voucher key, proof, optional inviter info).
- - Registers **her own new identity** with keys derived from **her** seed at
- `m/9'/coin'/5'/0'/0'/identity_index'/…`, funded by the imported `(proof, voucher_key)` via
- the SDK's in-process raw-key path (§5.2). No L1 Dash on Carol's side.
- - **If the link carries inviter info, Carol is *asked* "establish contact with \?"** —
- on confirm, a *normal* contactRequest Carol→Bob is sent via the shipped
- `send_contact_request` path; Bob sees it in his Requests and accepts. Opt-in on both ends
- (inviter checkbox + invitee prompt); no bearer auto-accept key is embedded.
-
-> **Design change from the first draft (security review Finding 1 + reference behavior).** The
-> first draft embedded a DIP-15 auto-accept `dapk` in the link so the contact would auto-establish
-> with zero taps on the inviter. That is **removed**: auto-accept's safety rests entirely on a
-> **1-hour TTL**, which is fundamentally incompatible with an invitation that is claimed hours-to-
-> days later — a link long-lived enough to be useful would be a long-lived auto-accept bearer
-> credential against the inviter (anyone finding a stale/posted link could make the inviter
-> publish an encrypted friendship xpub to them). The production wallets don't do this either:
-> their claim flow (`sendContactRequestToInviterUsingInvitationURL`) sends a **plain** contact
-> request. So v1 auto-sends a normal contactRequest; zero-tap acceptance is the inviter's own
-> orthogonal auto-accept setting, not baked into the shared link. (Embedding a short-TTL dapk with
-> an explicit "expired → manual request" fallback is a possible v2 nicety — deferred.)
-
-The consensus acts (asset lock, IdentityCreate, contactRequest) are all already implemented and
-tested; invitations are the **orchestration + off-chain envelope + key-handoff** around them.
-
----
-
-## 3. What already exists (reuse inventory — first-hand code read)
-
-| Capability | Where | Reused for |
-|---|---|---|
-| **Invitation funding derivation** `AssetLockFundingType::IdentityInvitation` (sub-feature `3'`), `accounts.identity_invitation` xpub, storage/recovery/persistence all wired | `asset_lock/build.rs:200-216` (`peek_next_funding_address`), storage `schema/accounts.rs`, `asset_lock/sync/recovery.rs:427`, `persistence.rs:3633` | **Create**: derive the voucher key + build the voucher asset lock |
-| **Full funded-asset-lock flow** `create_funded_asset_lock_proof(amount, account_index, funding_type, identity_index, signer) -> (AssetLockProof, DerivationPath, OutPoint)` (build → track → broadcast → IS wait → CL-upgrade → attach proof) | `asset_lock/build.rs:305-417` | **Create**: build the voucher lock |
-| **IS→CL upgrade** `upgrade_to_chain_lock_proof(out_point, None)` | `identity/network/registration.rs:186-197,247-250` | **Create**: force a CL proof before export |
-| **Register identity from a raw asset-lock private key** `Identity::put_to_platform_and_wait_for_response_with_private_key(sdk, proof, asset_lock_proof_private_key: &PrivateKey, identity_signer, settings)` | `rs-sdk/.../put_identity.rs:50-59,146+` | **Claim**: register invitee identity funded by the imported voucher — **core claim needs no new SDK code** |
-| **Bare claim FFI (external proof + one-time key)** `dash_sdk_identity_put_to_platform_with_instant_lock` / `_with_chain_lock(sdk, …proof bytes…, private_key:[u8;32], signer, settings)` | `rs-sdk-ffi/src/identity/put.rs:29,211` | Lower layer under the platform-wallet `claim_invitation` wrapper (no Swift binding yet) |
-| **`AssetLockProof::Instant` embeds the full tx + islock** (self-contained); `Chain` = outpoint+height (Platform resolves tx) | `asset_lock_proof/instant/…:38`, `…/chain/…:24` | **Link**: serialize the proof directly — no separate txid + L1 fetch |
-| **Consensus verifies the create sig against the asset-lock output's P2PKH hash** | `identity_create/state/v0/mod.rs:222-245` | Security trust anchor (§8): holder of the voucher key == who may create the identity |
-| **Seedless register (self-funded)** `register_identity_with_funding(AssetLockFunding, identity_index, keys_map, identity_signer, asset_lock_signer, …)` | `identity/network/registration.rs:121` | Template; claim uses the raw-key variant instead |
-| **Sanctioned raw-scalar export (path-gated)** `ContactCryptoProvider::export_auto_accept_private_key(&path)` / resolver hook | `contact_requests.rs:63`, `mnemonic_resolver_core_signer.rs:353` | **Create**: template for the new path-gated `export_invitation_private_key` (§5.3) |
-| **Send a normal contactRequest** `platform_wallet_send_contact_request_with_signer(...)` | FFI `dashpay.rs:225` | **Claim**: auto-send the plain contact-bootstrap invitee→inviter (no dapk) |
-| **Register/resume identity FFI (external signer)** `platform_wallet_register_identity_with_funding_signer`, `platform_wallet_resume_identity_with_existing_asset_lock_signer` | FFI `identity_registration_funded_with_signer.rs` | Template for the new claim FFI marshaling |
-| **Asset-lock build FFI + tracked-lock listing** `asset_lock_manager_build_transaction`, `create_funded_proof`, `list_tracked_locks` | FFI `asset_lock/build.rs`, `asset_lock/manager.rs` | Create FFI + inviter-side status |
-
-**Net: the funding-derivation family and both consensus signing paths already exist.** The new
-code is (a) the create orchestration + voucher-key export, (b) the claim orchestration, (c) the
-`dashpay://invite` envelope codec, (d) inviter-side invitation persistence, (e) FFI + Swift + UI.
-
----
-
-## 4. Interface / data flow per layer
-
-### 4.1 Rust — new module `wallet/identity/network/invitation.rs` (+ codec in `crypto/invitation.rs`)
-
-**Create (inviter):**
-```
-async fn create_invitation(
- &self,
- amount_duffs: u64, // rejected if 0 or > MAX_INVITATION_DUFFS
- funding_account_index: u32, // BIP44 account supplying the L1 UTXOs
- inviter: Option, // id + username + display_name (contact-bootstrap)
- expiry_unix: u32, // advisory; the FFI sets now + MAX_INVITATION_TTL_SECS
- asset_lock_signer: &AS, // funds the asset-lock (funding-input + credit-output)
- crypto_provider: &CP, // exports the voucher scalar (path-gated resolver)
-) -> Result
-```
-where `inviter: Option` is `Some` only when the inviter ticked "send a
-contact request back to me" (§ owner decision). Steps: (1) **bound the amount**
-(`0 < amount_duffs ≤ MAX_INVITATION_DUFFS`) and the expiry (non-zero), else err;
-(2) `create_funded_asset_lock_proof(amount, funding_account_index, IdentityInvitation, signer)`
-→ `(IS proof, path, out_point)` — **the builder auto-selects the next unused funding index** and
-returns its derivation `path`; **keep the IS proof, no CL upgrade** (§5.1); (3) **export the
-voucher private key** via the seedless resolver hook, **path-gated to the fully-hardened
-`9'/coin'/5'/3'/idx'`** (§5.3); (4) build the `Invitation` struct + `dashpay://invite` URI (§6);
-(5) **persist an invitation record** through the wallet persister (§4.2) — created status,
-outpoint, funding_index (from `path`), amount, expiry, optional inviter info; **the voucher key is
-never persisted** (re-derived from `funding_index`).
-
-**Claim (invitee):**
-```
-async fn claim_invitation(
- &self,
- invitation: ParsedInvitation, // decoded from the URI
- identity_index: u32,
- keys_map: BTreeMap, // invitee's own new-identity keys
- identity_signer: &IS, // invitee's identity-key signer
- establish_contact: bool, // invitee's answer to "establish contact with ?"
-) -> Result
-```
-Claim **bypasses the wallet's `AssetLockFunding` machinery** — the deliberately-removed
-`UseAssetLock` variant (external proof through the tracked-lock resolver) is *not* revived; the
-invitee owns neither the lock's inputs nor its tracking and can't drive its IS→CL fallback, so
-claim submits the imported proof directly. Steps: (1) **validate the parsed invitation before
-any network act** (§8 Finding 5): proof is an **Instant** proof; the voucher pubkey is the
-credit-output's P2PKH target (`proof.output() → credit_outputs[output_index]`); expiry not
-past — fail loud with a specific error otherwise; (2) build the placeholder `Identity` with
-`keys_map`; (3)
-`placeholder.put_to_platform_and_wait_for_response_with_private_key(&sdk, invitation.proof,
-&invitation.voucher_key, identity_signer, settings)` → new `Identity` — **wrap this submit in
-`submit_with_cl_height_retry`** (feasibility Note A): the direct raw-key SDK call bypasses
-`register_identity_with_funding`, so it doesn't inherit that helper's retry on a transient
-CL-height-too-low (10506); without the wrapper a transient reject is a hard claim failure; (4)
-local bookkeeping
-(add to IdentityManager, breadcrumbs) — best-effort, non-propagating (mirrors
-`register_identity_with_funding` Step 4); (5) if `invitation.inviter` present **and
-`establish_contact`** (the invitee said yes to the prompt), **send a normal contactRequest**
-invitee→inviter via the shipped `send_contact_request` path (the new invitee identity as
-sender). Idempotent/re-sendable if step 5 fails after step 3 succeeds (§10). If the invitee
-declines, the identity is still created — just no contact.
-
-### 4.2 Rust — inviter-side persistence (proper persister integration — owner decision)
-**A first-class persisted invitation record, through the existing wallet persister system**
-(not an ad-hoc KV blob). Follow the established DashPay changeset → persister → SwiftData-model
-pattern already used for contact requests / payments (`rs-platform-wallet` changeset overlays +
-`rs-platform-wallet-storage` migration + the Swift `Persistence/Models` `@Query` models —
-research-swift map). Concretely:
-- **Rust storage (`rs-platform-wallet-storage`):** a new `invitations` table via a migration
- (mirroring `asset_locks` `V001__initial.rs:247`), columns `wallet_id, outpoint, funding_index,
- amount_duffs, expiry_unix, status (created|claimed|reclaimed), inviter_opt_in, created_at,
- claimed_identity_id?`. **No secret column** — the voucher key is re-derived from `funding_index`
- (§5.3), never stored.
-- **Rust changeset (`rs-platform-wallet`):** an `InvitationChangeSet` emitted by create/reclaim
- and by the sync that flips *created → claimed* (detected by the tracked asset-lock's outpoint
- being consumed on Platform / the invitee's inbound contactRequest), queued onto the persister
- exactly like `AssetLockChangeSet` / the DashPay overlays.
-- **Swift:** a `PersistentInvitation` SwiftData model registered in `DashModelContainer`, driving
- a `@Query` "Sent invitations" list (`InvitationsView`).
-
-Recovery still leans on re-derivation: an unclaimed invitation's voucher key is re-derived from
-its `funding_index` to re-package or reclaim (the asset-lock row already tracks the lock's
-lifecycle for the actual reclaim submit). The invitations table adds the durable, queryable
-*status* surface the UI needs.
-
-### 4.3 FFI (rs-platform-wallet-ffi) — new `invitation.rs`
-- `platform_wallet_create_invitation(wallet, amount_duffs, funding_account_index,
- inviter_identity_id: *const [u8;32] /*nullable*/, inviter_username: *const c_char /*nullable*/,
- expiry_unix: u32, core_signer_handle, out_uri: **c_char, out_outpoint: *mut OutPointFFI)
- -> Result`. **Only `core_signer_handle`** (the asset-lock/Core signer) is needed — pure voucher
- creation registers no identity, so there is no identity `signer_handle` (feasibility Note B).
- `now`/`expiry_unix` is passed in from Swift (FFI can't read the clock deterministically — same
- convention as `build_auto_accept_qr`).
-- `platform_wallet_claim_invitation(wallet, uri: *const c_char, identity_index,
- identity_pubkeys, identity_pubkeys_count, signer_handle /*invitee identity signer*/,
- establish_contact: bool, out_identity_id: *mut [u8;32], out_identity_handle: *mut Handle)
- -> Result`. `establish_contact` is the invitee's answer to the "establish contact with
- \?" prompt (only acted on if the link carries inviter info). Reuses
- `decode_identity_pubkeys` + the managed-identity insert from
- `identity_registration_funded_with_signer.rs`. Note: a **bare** identity-create-from-external-
- proof FFI already exists one layer down — `dash_sdk_identity_put_to_platform_with_chain_lock`
- / `..._with_instant_lock(sdk, …proof bytes…, private_key: *const [u8;32], signer, settings)`
- (`rs-sdk-ffi/src/identity/put.rs:29,211`). We do **not** call that bare FFI from Swift for
- claim: the platform-wallet `claim_invitation` wrapper is needed so the new invitee identity is
- registered in the wallet's `ManagedIdentity` storage **and** the contact-bootstrap fires — it
- calls `put_to_platform_and_wait_for_response_with_private_key` internally, then does bookkeeping
- + the bootstrap send. (No `core_signer_handle` is needed on claim: the asset-lock signature
- uses the imported raw voucher key, not a wallet-derived one.)
-- `platform_wallet_list_invitations(...)` + free helpers for the inviter status list.
-- String/URI input validation identical to the auto-accept FFIs (null checks, length caps).
-
-### 4.4 Swift (swift-sdk + SwiftExampleApp)
-Current services (note: `PlatformService`/`WalletService`/`UnifiedAppState` were **removed**):
-`AppState` (owns the `SDK`, network), `PlatformWalletManager` (per-network, DashPay sync
-lifecycle), `ManagedPlatformWallet` (**all identity/DashPay FFI calls live here**). **All Swift
-↔ Rust FFI work MUST go through the `swift-rust-ffi-engineer` agent** (repo `CLAUDE.md` rule).
-The **DIP-15 auto-accept QR flow is the copy-template** for both directions.
-- swift-sdk wrappers on `ManagedPlatformWallet`:
- - `createInvitation(amountDuffs:fundingAccount:expiry:) async throws -> InvitationLink`
- (idiom of `registerIdentityWithFunding` `ManagedPlatformWallet.swift:3370` — long-running L1
- build, so wrap with a Controller+Coordinator triad like `IdentityRegistrationController`).
- - `claimInvitation(uri:identityIndex:) async throws -> ManagedIdentity` (idiom of
- `sendContactRequestFromQR` `:1758`).
-- SwiftExampleApp UI (under the DashPay tab, `App/Views/DashPay/`):
- - **Create**: a "Create invitation" action (beside "Add me QR" in `DashPayProfileView.swift:74`)
- → amount entry **+ a "send a contact request back to me" checkbox** (drives the optional
- inviter info) → share sheet with the link + a QR (reuse `generateQRCode`).
- - **Claim**: a toolbar button + sheet mirroring `AddViaQRSheet` (`DashPayTabView.swift:830`)
- (paste/scan the `dashpay://invite` link) → register identity → **if the link carries inviter
- info, prompt "establish contact with \?"** → pass the answer as `establish_contact` →
- `kickDashPaySync` → the new identity (+ optional contact) land via `@Query`.
- - **Invitations list** (created + status): a new `InvitationsView` (`@Query` over
- `PersistentInvitation`, §4.2), reached via a toolbar `NavigationLink` (like the Ignored link
- at `:151`).
- - **Deep link (net-new plumbing):** no `onOpenURL`/`CFBundleURLTypes` exist today. Add the
- `dashpay` URL scheme to `SwiftExampleApp/Info.plist` and `.onOpenURL { … }` on the
- `WindowGroup` in `SwiftExampleAppApp.swift:105`, routing to `RootTab.dashpay` + the claim
- sheet; reuse the `AddViaQRSheet` URI-parse as the model.
-- `FundingType.identityInvitation = 3` already exists in Swift
- (`ManagedAssetLockManager.swift:36`, `KeyWalletTypes.swift:14`).
-- **Framework build:** `DashSDKFFI.xcframework` is a generated artifact (not committed); rebuild
- via `packages/swift-sdk/build_ios.sh --target sim` after any FFI/header change, then the
- `xcodebuild` app build (§ repo CLAUDE.md). Always clean+rebuild after header changes.
-
-### 4.5 QA contract
-The authoritative QA contract is **`packages/swift-sdk/SwiftExampleApp/TEST_PLAN.md`** (driven by
-the `simulator-control` skill; dashboard at `dashpay.github.io/qa-dashboard-site`). Add rows to
-**§4.10 DashPay** as **DP-12+** in the existing format:
-`| ID | Action | Layer | Tier | Status | Tags | Entry point & test notes |`. Planned rows:
-- `DP-12 | Create invitation | Cross | Common | … | funding | DashPay → Create invitation → platform_wallet_create_invitation (builds L1 asset lock; needs testnet funds).`
-- `DP-13 | Claim invitation | Platform | Common | … | | Paste/scan dashpay://invite → platform_wallet_claim_invitation → new identity + contact.`
-- `DP-14 | Invite→claim e2e (two wallets) | Cross | Thorough | … | multiwallet | Create on A, claim on B, contact auto-establishes both ends (cf. DP-11).`
-- `DP-15 | Reject malformed / already-claimed invitation | Platform | Uncommon | … | | Bad link + reused link both fail loudly, no side effects.`
-(Secondary: the `AI_QA/` MCP playbooks — add a `QA004`-style invite→claim walkthrough if useful.)
-
----
-
-## 5. The three technical cruxes (de-risked first-hand; §11 spikes confirm)
-
-### 5.1 Proof type — DECIDED: InstantSend (owner decision 2026-07-08)
-`AssetLockProof` has two variants with very different self-containment (confirmed
-`asset_lock_proof/mod.rs:40`):
-- **`InstantAssetLockProof { instant_lock, transaction, output_index }`** — embeds the **full
- funding tx + the InstantLock**. Self-contained (Platform validates the islock against the
- embedded tx). This is what the **reference iOS/Android wallets export** (`islock` + they carry
- the txid and re-fetch the tx). Fast to produce (just wait for the IS lock). **Risk:** Platform
- rejects an islock whose quorum has rotated or is too old relative to Platform's core height
- (`is_instant_lock_proof_invalid` + the IS→CL retry in `registration.rs`). An invitation that
- sits **unclaimed** for a long time can go stale.
-- **`ChainAssetLockProof { core_chain_locked_height, out_point }`** — tiny (outpoint + height);
- Platform resolves the tx from Core by outpoint. **No staleness window** (chain-locked is
- permanent), so an unclaimed invitation stays valid indefinitely. Cost: the inviter waits for a
- ChainLock at create (≈ up to a block or two, low-minutes).
-
-**DECISION (owner, 2026-07-08): export an InstantSend proof.** Faster create (no CL wait), matches
-the reference wallets, and the `InstantAssetLockProof` embeds the full tx + islock so the link is
-fully self-contained (the invitee never fetches anything from L1). `create_funded_asset_lock_proof`
-returns exactly this for a fresh tx (its `validate_or_upgrade_proof` only upgrades to CL when the
-tx is *old* — not the case at create), so the invitation path **keeps the IS proof, no forced CL
-upgrade**.
-
-> **Slow-IS fallback must be enforced (Rust-core review H1).** `create_funded_asset_lock_proof`
-> *also* falls back to a ChainLock proof if the IS lock doesn't propagate within its 300s
-> preference window. Since the invitee's `validate_claimable` accepts only an InstantSend proof,
-> `create_invitation` **must reject a returned ChainLock proof** — else it would emit a
-> `dashpay://invite` link the invitee silently rejects (a dead voucher: funds locked, no signal).
-> On this rare path create returns a clear error; the funding lock stays tracked/reclaimable, and
-> the inviter retries. *(A future robustness option is to accept a Chain proof on claim too —
-> it never goes stale — skipping the local credit-output pre-check since a Chain proof carries no
-> embedded tx; deferred, as it deviates from the literal Instant-only decision.)*
-
-**Staleness mitigation = a short, IS-scoped advisory expiry (not an IS→CL upgrade in v1).** The
-one real risk is that Platform rejects a *stale* islock (quorum rotated). Rather than build an
-invitee-side IS→CL upgrade (which needs the embedded tx re-tracked — non-trivial, and the
-external-proof `UseAssetLock` path was deliberately removed), v1 sets the invitation's advisory
-`expiry` conservatively **inside the IS validity window** (default ~24h, ≤ `MAX_INVITATION_TTL`):
-the claim path refuses a past-expiry link up front with a clear "invitation expired — ask the
-sender for a new one," so an about-to-go-stale proof is never submitted. Cheap, no fund risk (the
-inviter simply re-creates), and the inviter's asset lock is reclaimable after expiry. **Future
-enhancement (not v1):** an invitee-side IS→CL upgrade from the embedded tx to extend the window to
-days/weeks. *(Note: this makes the create FFI's identity-signer moot as before, and the claim's
-`submit_with_cl_height_retry` wrapper — feasibility Note A — still applies to the IS submit.)*
-
-### 5.2 Claim is ordinary identity registration with imported funding
-`put_to_platform_and_wait_for_response_with_private_key(proof, voucher_key, identity_signer)`
-already does exactly what claim needs. The invitee's identity keys come from the invitee's own
-seed (normal registration); only the **funding** `(proof, voucher_key)` is imported. **No new
-SDK code for the core claim.** The `identity_invitation` account is an inviter-only concept —
-the invitee never derives sub-feature `3'`.
-
-### 5.3 Exporting the voucher private key is a deliberate bearer-credential export
-The architecture's invariant is "private keys never cross the FFI boundary as raw bytes," and
-the signer-driven builder deliberately **withholds** the credit-output private key (it returns
-`AssetLockCreditKeys::Public((pubkey, path))`, `build.rs:117`). The invitation **is** a raw-key
-handoff (the whole point), so exporting it is a scoped, documented exception — exactly like the
-auto-accept `dapk` blob, which already exports a bearer private key in a QR.
-
-**Key choice:** **HD-derived at `m/9'/coin'/5'/3'/index'`** (not a JS-style random key). HD makes
-it DIP-13-recoverable — the wallet can re-derive/scan unclaimed invitation funding txs and let
-the user reclaim/resend (DIP-13's explicit recommendation) — at the cost of needing an export
-step. (A random ephemeral key, JS-SDK precedent `createAssetLockTransaction.ts:26`, exports
-trivially but is unrecoverable; rejected.)
-
-**Export = a NEW seedless resolver hook, path-gated to the exact invitation sub-feature
-(security review Finding 2 — normative).** The create FFI is **seedless** (it drives a
-`MnemonicResolverCoreSigner`, not a resident `Wallet`), so there is no `&Wallet` to
-`derive_extended_private_key` on for the real host — v1 must add a raw-scalar export on the
-resolver, exactly mirroring the sanctioned precedent
-`export_auto_accept_private_key(&path) -> SecretKey` (`mnemonic_resolver_core_signer.rs:353`,
-`ContactCryptoProvider` `contact_requests.rs:63`). **The new `export_invitation_private_key(&path)`
-MUST gate on the full path** `comps.len()==5 && comps[0]==9' && comps[2]==5' && comps[3]==3'` —
-**not** merely `comps[2]==5'`, because feature `5'` is shared with identity-registration
-(`5'/0'`,`5'/1'`), top-up (`5'/2'`), etc.; a loose gate would let a caller exfiltrate the user's
-**own** identity-funding keys. Add a negative test mirroring
-`export_auto_accept_private_key_gates_to_the_auto_accept_path`.
-
-**Never persist the key.** Because it is HD-derived, the inviter re-derives it from the seed
-whenever it re-packages or reclaims. Storage tracks only funding index + outpoint (§4.2). The
-returned URI (which *contains* the plaintext key) is treated as a secret end-to-end: no logging,
-no analytics, sensitive-pasteboard flag on the Swift side (§8 Finding 3).
-
-> **This export hook is v1 critical path, not a follow-up (feasibility Finding 5, BLOCKING).**
-> Production/example-app wallets are **seedless at steady state** (`Wallet::new_external_signable`,
-> no root key — `persistence.rs:158-163`); only the *first-ever* session has a resident seed. So
-> the "derive from a resident `Wallet`" idea is a **dead end**: create a wallet Monday (seed
-> resident), relaunch Tuesday (external-signable) → tap "Create invitation" →
-> `wallet.derive_extended_private_key(path)` errors and the existing
-> `export_auto_accept_private_key` rejects the `5'/3'` path (it gates to `16'`), so **no link can
-> be produced.** The fix is the new gated `export_invitation_private_key` on
-> `MnemonicResolverCoreSigner` + a `ContactCryptoProvider`-style method (seedless + seed impls,
-> cf. `contact_requests.rs:63/188`) + its FFI — a dedicated implementation slice (§13 slice 2).
-
----
-
-## 6. The `dashpay://invite` link envelope — a single versioned blob
-
-> **SUPERSEDED (2026-07-13, §0.1):** the shipped envelope is the legacy query format
-> (`du`/`assetlocktx`/`pk`/`islock` — see §0A), not this blob.
-> Kept for the design rationale it records (secret handling, caps, transport notes still apply).
-
-**Decision: one opaque, versioned payload** behind a `dashpay://invite?data=`
-deep link (keeping the reference's `dashpay://invite` scheme name for familiarity), **not** the
-reference's six loose query params. Rationale in §7. The payload is a small versioned blob in a
-**hand-rolled little-endian binary encoding** (deliberately *not* serde/bincode — the crate's
-`serde` feature is optional and off, and `AssetLockProof` is internally-tagged so bincode-serde
-rejects it), so the envelope can evolve without breaking older links. The as-built wire order
-(see `crypto/invitation.rs`) is:
-
-```text
-wire = version:u8 // = 0
- ‖ voucher_key:[u8; 32] // one-time ECDSA private key (secret; zeroized)
- ‖ expiry_unix:u32(LE) // ADVISORY, IS-scoped (§5.1); not consensus
- ‖ inviter_present:u8 // 0 = none, 1 = InviterInfo follows
- [ identity_id:[u8; 32]
- ‖ username:len-prefixed // DPNS name (whom the invitee's contactRequest targets)
- ‖ display_present:u8 [ display_name:len-prefixed ] ]
- ‖ asset_lock:len-prefixed // InstantSend proof (§5.1) — embeds tx + islock; LAST, length-prefixed
- // NO auto-accept dapk — v1 sends a normal contactRequest, invitee-confirmed (§2)
-```
-- Serializing the `InstantAssetLockProof` directly means the link **embeds the full funding tx +
- islock**, so the invitee needs **no L1 tx fetch** (an improvement over the reference, which
- carried only the txid). Link size is a few hundred bytes → base58 ~a few hundred chars: fine
- for a deep link and a QR.
-- **Length-cap the `data=` param before decode (§8 Finding 5, LOW).** The base58-**char** cap on
- the input *before* decoding is the DoS mitigation (mirrors the `dapk` cap in
- `parse_dashpay_contact_uri`). Note: `AssetLockProof`'s consensus bincode decode is **already
- bounded and panic-free** on arbitrary bytes (dashcore `MAX_VEC_SIZE`, finite cursor, all
- `Result`-based — verified), so the residual is only "a huge blob is fully buffered," which the
- pre-decode char cap closes. A fuzz test is cheap insurance, not a blocker.
-- The codec pair in `crypto/invitation.rs` is
- `encode_invitation_uri(voucher_key: &SecretKey, asset_lock: &AssetLockProof, expiry_unix: u32,
- inviter: Option<&InviterInfo>) -> Result` and
- `parse_invitation_uri(uri: &str) -> Result`, fully unit-tested (round-trip +
- every malformed rejection). A plain `https://…` fallback host can wrap the same `?data=` for
- users without the app installed — deferred (no hosting in v1; see §6.1).
-
-### 6.1 Transport security & the custom-scheme limitation
-
-The `data=` payload is a **bearer credential**: whoever reads the plaintext link controls the
-voucher and can claim (front-run) it. Because the app registers the `dashpay://` **custom URL
-scheme**, any other app that also registers `dashpay` can intercept an invite link on the same
-device and steal the claim. The load-bearing mitigation is therefore **economic, not transport**:
-`MAX_INVITATION_DUFFS` caps the loss at 0.26 DASH, and the inviter can reclaim an unclaimed voucher
-(best-effort race). The advisory expiry does **not** bound a leak (a leaked-link holder ignores it).
-
-A hardened production transport would use **Universal Links** (HTTPS + a hosted
-`apple-app-site-association`, `associated-domains` entitlement) or another verified handoff so the
-OS can't hand the link to an impostor app. That is **out of scope for this example app** — it
-needs hosting infrastructure the sample doesn't have, and the amount cap already bounds the blast
-radius — but it is the recommended path for the production wallet and is tracked as a follow-up.
-The `?data=` shape is transport-agnostic, so moving from the custom scheme to a Universal Link is a
-routing change, not an envelope change.
-
----
-
-## 7. Interop decision — ~~RESOLVED: ship our own self-contained envelope~~
-
-> **REVERSED (2026-07-13, §0.1):** the as-built codec adopts the reference wallets' legacy
-> query format for field-level cross-claimability with dash-wallet iOS/Android. The analysis
-> below (dead FDL delivery, JS SDK never had invitations) remains accurate — only the
-> conclusion changed, by owner decision, once cross-wallet claimability was prioritized.
-Research (research-reference, primary sources) settled this:
-- The production iOS (DashSync) + Android (dash-wallet) wallets use an **identical plaintext
- URL-query payload**: `du` (username), `display-name`, `avatar-url`, `assetlocktx` (**txid
- only**, 64-hex), `pk` (**WIF** private key), `islock` (hex InstantLock). The invitee **fetches
- the full funding tx from L1 by txid**, then registers using the embedded islock.
-- That link was distributed via **Firebase Dynamic Links**, which **Google shut down
- 2025-08-25** — the hosted `invitations.dashpay.io/link` short-links now **404**. So even the
- production wallets' *share layer is already broken* and must be reworked.
-- **The JS SDK never had an invitation API** — invitations existed only in the two native apps.
-
-**Conclusion:** there is little value matching a legacy wire format whose delivery mechanism is
-dead. We ship our own **self-contained, versioned** envelope (§6). The **only** things we must
-NOT diverge on are the **on-chain / consensus** semantics — the DIP-13 `3'` derivation and the
-islock / asset-lock-proof shapes Platform consensus accepts — because those are what actually
-interoperate. This mirrors the auto-accept spec's "iOS-first, DIP-faithful where defined,
-normative-for-us where silent" stance. (If byte-interop with a future reworked DashWallet is ever
-required, matching is a localized codec change; the on-chain acts already interoperate.)
-
----
-
-## 8. Security
-*(Folds a 4-lens security review: no CRITICALs — the core crypto is sound; findings are must-fix
-hardening + honest-framing fixes. Verified-clean floor: in-flight IdentityCreate is
-non-malleable, double-claim is deterministic, the invitee never risks its own funds.)*
-
-- **Consensus trust anchor (why this is safe at all).** Platform validates the IdentityCreate's
- outer signature against the **asset-lock output's P2PKH public-key hash**
- (`identity_create/state/v0/mod.rs:222-245`) and the identity id is `hash(outpoint)` — so a
- network observer who does *not* hold the voucher key cannot swap in their own keys and steal an
- in-flight claim, and two racers target the *same* id (consensus commits exactly one). Every
- claim-theft attack reduces to **"who holds the link."** The invitee's own identity keys sign
- the per-key witnesses separately.
-- **Bearer credential — the load-bearing leak mitigation is the amount cap + reclaim, NOT the
- expiry (Rust-security-review LOW-2 honesty fix):**
- - **Amount cap enforced in Rust (Finding 4).** `create_invitation` rejects
- `amount_duffs > MAX_INVITATION_DUFFS` — the *actual* bound on a leaked link's blast radius
- (a direct FFI caller / headless host / UI bug can't exceed it). Never UI-only.
- - **Expiry is a UX / reclaim signal, not a leak bound.** A malicious *finder* of a leaked link
- holds the voucher key + proof and can submit directly, **ignoring the honest UI's expiry
- check** — so `expiry_unix` does not bound a leaked-link window. What it *does* do: (a) stop an
- **honest** invitee from submitting an about-to-go-stale IS proof (§5.1), and (b) give the
- inviter a clear reclaim-after signal. Advisory, not consensus. (The FFI sets a sensible
- default expiry from `MAX_INVITATION_TTL_SECS`; clamping it in Rust is symmetry, not security.)
- - **Single-use** (asset lock consumed on first claim → deterministic reject thereafter), funds
- are the inviter's to give; the inviter can race to **reclaim** an unclaimed voucher (a race it
- can lose if already leaked — §8 Finding 6).
-- **The link is plaintext key material — treat the URI as secret end-to-end (Finding 3).** The
- create FFI returns the URI (which *contains* the voucher key) as a C string that flows through
- Swift + a `dashpay://invite` deep-link handler (handlers routinely log URLs) + clipboard
- (iOS Universal Clipboard syncs across devices) + the share sheet. Requirements: **no logging /
- no analytics** of the URI; secret/`Zeroizing` types Rust-side; a **sensitive-pasteboard** flag
- Swift-side; the voucher key is **never persisted** (re-derived from `funding_index`, §5.3).
-- **Inviter self-claim / front-run is a real griefing/DoS vector against the invitee (Finding 6 —
- honesty fix).** *Not* "no third-party risk." The inviter can front-run or reclaim after handoff,
- denying the invitee onboarding mid-flow with no signal it was the inviter's doing. No fund theft
- (funds are the inviter's), but real denial. Likewise **"reclaim = revocation" is a race the
- inviter can lose** if the link already leaked — reclaim is best-effort, and the advisory expiry
- is the actual bound. Documented as an accepted, honestly-stated limitation.
-- **Untrusted proof on claim — validate before submit (Finding 5, LOW after re-verify).** The
- `AssetLockProof` bincode decode is already bounded/panic-free; the §6 pre-decode length cap is
- the DoS mitigation (keep it). The genuinely useful part is **fail-fast UX, not a security gap**:
- cheap **local pre-submit checks** — the proof is an **Instant** proof (§5.1), the advisory
- expiry is not past, and the **voucher pubkey-hash ∈ the selected credit output**
- (`proof.output() → credit_outputs[output_index]`) — so a malformed/hostile/stale link fails with
- a clear error instead of an opaque consensus reject. The
- credit-output-pubkey binding is itself consensus-enforced, so this cannot be *bypassed* to steal;
- it only improves the error.
-- **Unauthenticated envelope (Finding 7 — documented, no v1 fix).** Nothing signs the bundle, so a
- MITM on the *link channel* can substitute the whole invite. Blast radius is limited (the
- contact only forms toward whatever inviter identity is in the link; an attacker can at most make
- the invitee contact the attacker's own identity — achievable with a normal contact request
- anyway). Reduces to "bearer-link trust = channel trust"; envelope signing wouldn't help (the
- channel is the trust root).
-- **Privacy (Finding 8, LOW).** Because id = `hash(outpoint)`, the inviter knows the invitee's
- future identity id before they claim, and that id is inviter-chosen. Noted.
-- **Malformed / hostile link:** every field size-capped before decode; a bad link fails loudly
- with no side effects.
-
----
-
-## 9. Decisions (RESOLVED — owner, 2026-07-08)
-1. **Proof type: InstantSend** (§5.1). Fast create, self-contained link; staleness covered by a
- short IS-scoped advisory expiry (claim refuses past-expiry), not an IS→CL upgrade in v1.
-2. **Contact-bootstrap: opt-in on both ends.** Inviter ticks "send a contact request back to me"
- (→ inviter info in the link); the invitee is *asked* "establish contact with \?" at
- claim and only then is a normal contactRequest sent. In v1. No auto-accept dapk (§8 Finding 1).
-3. **Inviter persistence: proper wallet-persister integration** (§4.2) — a first-class
- `invitations` table + changeset + `PersistentInvitation` SwiftData model, not a KV blob. In v1.
-4. **Link scheme:** our own self-contained versioned blob (§7).
-5. **Amount / TTL:** Rust-enforced `MAX_INVITATION_DUFFS` (default a sensible identity-reg +
- small-balance amount; confirm exact duffs during spikes) and `MAX_INVITATION_TTL` bounded to
- the **IS validity window** (default ~24h) since the proof is InstantSend.
-
----
-
-## 10. Failure modes
-- **Insufficient inviter balance to fund the lock** → create fails pre-broadcast, funds
- untouched (reservation released — existing `create_funded_asset_lock_proof` rejection path).
-- **InstantSend lock never arrives at create** → `create_funded_asset_lock_proof`'s 300s IS wait
- elapses and (for a fresh tx) it surfaces an error; the tracked lock is resumable (inviter can
- retry or reclaim). We do **not** force a CL upgrade (§5.1).
-- **Stale IS proof (claimed too late)** → the advisory expiry makes the claim refuse *before* the
- IS lock could be rejected by Platform; the inviter re-creates. (Extending the window via an
- invitee-side IS→CL upgrade is a post-v1 enhancement.)
-- **Invitee claims an already-claimed / inviter-front-run link** → Platform rejects (lock
- consumed); claim returns a clear "invitation already used" error; no identity created. (This is
- also the inviter-front-run griefing outcome, §8 Finding 6.)
-- **Malicious inviter hands a mismatched/IS/expired proof** → caught by the claim pre-submit
- checks (§4.1 step 1 / §8 Finding 5) → fail loud, no blind submit.
-- **Claim interrupted after identity created but before contact-bootstrap sent** → the identity
- exists (self-heals into the invitee's IdentityManager on next re-sync); the contact request is
- re-sendable (idempotent — the send path adopts an existing friendship). Not a data-loss path.
-- **Malformed / truncated / oversize link** → parse/size-cap error, no side effects.
-- **Invitee has no seed / can't derive identity keys** → claim fails before any network act.
-- **Voucher never claimed AND inviter loses seed (§8 Finding 9, LOW)** → L1 Dash stranded in the
- lock (asset locks are one-way). Mitigated by HD re-derivation from `funding_index` — this stays
- a generic "lost your seed" problem, not invitation-specific.
-
----
-
-## 11. Spikes (before implementation — task #11)
-1. **S1 — raw-key claim end-to-end (offline):** in a `rs-platform-wallet` integration test,
- build an asset lock at `IdentityInvitation`, derive the voucher key, and drive
- `put_to_platform_and_wait_for_response_with_private_key` against a mock/echo SDK to confirm
- the proof + raw-key + invitee-identity-signer triple registers an identity. Confirms §5.2.
-2. **S2 — seedless voucher-key export + path gate:** add `export_invitation_private_key(&path)`
- on the resolver/provider mirroring `export_auto_accept_private_key`
- (`mnemonic_resolver_core_signer.rs:353`), and prove the gate: it exports for
- `9'/coin'/5'/3'/idx'` and **rejects** `9'/coin'/5'/0'/…` (identity-auth), `…/5'/1'/…` (reg
- funding), `…/5'/2'/…` (top-up) — the Finding-2 negative test. Confirms §5.3.
-3. **S3 — create keeps the IS proof + persistence round-trip:** confirm
- `create_funded_asset_lock_proof(IdentityInvitation)` returns an **Instant** proof for a fresh
- tx (no auto-upgrade), and that an `InvitationChangeSet` round-trips through the persister
- (`created` row readable back). Confirms §5.1 + §4.2.
-4. **S4 — link envelope codec:** implement + unit-test `encode/parse_invitation_uri`
- (round-trip + malformed) — cheap, do first.
-
----
-
-## 12. Test / verification plan
-- **Rust unit:** invitation URI codec (round-trip + every malformed rejection incl. the
- pre-decode length cap); voucher blob round-trip; the **export-path-gate negative test** (§5.3 /
- S2, Finding 2 — the blocking one: exports `5'/3'`, rejects `5'/0'`,`5'/1'`,`5'/2'`,`16'`);
- create-invitation **rejects `amount > MAX_INVITATION_DUFFS` and `expiry > now+MAX_TTL`**
- (Finding 3/4); claim **pre-submit checks reject** a non-Instant proof and a voucher-pubkey ∉
- credit-output (Finding 5 — fail-fast); expired-link rejection. (Optional insurance: a fuzz test
- that `parse_invitation_uri` on arbitrary bytes never panics — not a blocker, decode is already
- bounded.)
-- **Rust integration (`rs-platform-wallet`):** the S1 offline flow as a permanent test; the
- create→export→re-derive-from-`funding_index` round-trip (recovery); reclaim-unused path.
-- **FFI:** null/oversize/bad-URI input validation; create→parse round-trip; claim marshaling
- (identity handle inserted, id out); assert the URI is not emitted to logs.
-- **Swift:** `build_ios.sh` green; wrapper unit tests for encode/decode boundaries.
-- **Testnet funded e2e (task #13):** fund an inviter wallet via the **built-in faucet**
- (Wallet → Receive → "request from testnet", `TestnetFaucetService` → `faucet.thepasta.org`) →
- register the inviter identity + DPNS name → `create_invitation` → parse the link in a **second**
- wallet with no funds → `claim_invitation` → assert the invitee identity exists on Platform and
- (if bootstrap) the contact auto-establishes after the inviter's drain. This is the acceptance
- gate. Can run headless (Rust integration against testnet) and/or two-simulator on-device.
-- **On-device (two sims):** create on sim A, claim on sim B, contact appears on both.
-- **QA contract:** the scenarios from §4.5.
-
----
-
-## 13. Commit slicing (implementation order)
-1. `crypto/invitation.rs` codec (payload struct + `encode/parse_invitation_uri` + length cap) +
- tests (S4).
-2. **Voucher-key export (v1 critical path — feasibility Finding 5):** gated
- `export_invitation_private_key` on `MnemonicResolverCoreSigner` (gate `9'/coin'/5'/3'/idx'`) +
- `ContactCryptoProvider`-style method (seedless + seed impls) + the path-gate negative test (S2).
- Without this the seedless host cannot produce a link at all.
-3. `network/invitation.rs` create (slice-2 export + keep IS proof + amount/expiry caps) + claim
- (raw-key submit wrapped in CL-height retry + Instant-proof pre-submit checks + optional
- invitee-confirmed contactRequest) helpers + unit tests (S1).
-4. **Inviter persistence (§4.2):** `invitations` migration + `InvitationChangeSet` + status sync.
-5. FFI `platform_wallet_create_invitation` (core signer only) / `_claim_invitation`
- (`establish_contact` param) + tests (marshaling mirrors `identity_registration_funded_with_signer.rs`).
-6. swift-sdk wrappers on `ManagedPlatformWallet` + `PersistentInvitation` SwiftData model (**via
- `swift-rust-ffi-engineer`**).
-7. SwiftExampleApp: create sheet (amount + "send request back" checkbox), claim sheet (with the
- "establish contact with \?" prompt), `InvitationsView` list, + `dashpay://invite`
- deep-link handler (`Info.plist` scheme + `.onOpenURL`).
-8. QA-contract rows (TEST_PLAN.md §4.10 DP-12+).
-9. Testnet e2e evidence + docs (`SPEC.md` Milestone 5 as-built, `DIP_CONFORMANCE_GAPS.md` row).
-
----
-
-## 14. Multi-agent spec-review resolutions (2026-07-08)
-Four research streams (wallet/SDK/Swift/reference) + three adversarial spec reviews
-(feasibility / security / scope). Folded:
-- **Feasibility — core mechanic CONFIRMED** (claim independence proven at `v0_methods.rs:65-78`;
- create/CL/FFI confirmed). **One blocker: seedless voucher-key export** — the resident-`Wallet`
- idea is a dead end (production wallets are `new_external_signable`); promoted to **v1 critical
- slice 2** (§5.3, §13). Should-fixes folded: bounded CL wait (§5.1/§4.1), claim submit wrapped in
- CL-height retry (§4.1), create FFI drops the spurious identity signer (§4.3).
-- **Security — no CRITICALs.** Two blockers folded: (1) the **dapk TTL contradiction** →
- auto-accept dropped, plain contactRequest bootstrap (§2); (2) **export path-gating** to
- `9'/coin'/5'/3'/idx'` with a negative test (§5.3). Hardening folded: Rust amount cap, advisory
- voucher expiry, secret/no-log URI (§8 Finding 3/4); honesty fixes (self-claim = griefing/DoS,
- reclaim = a race — §8 Finding 6). Proof-parse worry **downgraded to LOW** on re-verify (bincode
- is already bounded; the length cap is the mitigation; pre-submit checks are fail-fast UX).
-- **Reference/interop** — the production link format is dead (FDL shutdown); ship our own
- self-contained versioned envelope, preserve only on-chain semantics (§7).
-- **Scope** — scope levers threaded (single versioned blob §6; reuse over new code throughout).
-- **Owner decisions (2026-07-08, sync gate):** (1) **InstantSend** proof, not ChainLock —
- staleness handled by a short IS-scoped expiry (§5.1); (2) contact-bootstrap **opt-in on both
- ends** — inviter checkbox + invitee "establish contact?" prompt (§2, §4.1); (3) **proper
- wallet-persister** integration for invitations, not a KV blob (§4.2). All in v1.
diff --git a/docs/dashpay/DIP_CONFORMANCE_GAPS.md b/docs/dashpay/DIP_CONFORMANCE_GAPS.md
deleted file mode 100644
index 530c6bbea8c..00000000000
--- a/docs/dashpay/DIP_CONFORMANCE_GAPS.md
+++ /dev/null
@@ -1,359 +0,0 @@
-# DashPay — DIP-15 + DIP-16 conformance gaps (code-verified audit)
-
-> **Purpose.** A from-scratch re-audit of the DashPay implementation against the
-> canonical [DIP-15](https://github.com/dashpay/dips/blob/master/dip-0015.md)
-> (DashPay) and [DIP-16](https://github.com/dashpay/dips/blob/master/dip-0016.md)
-> (Headers-First SPV synchronization — DIP-15 §12 is built on it), cross-checked
-> against the **actual code** on `feat/dashpay-m1-sync-correctness` (not the
-> self-reported status in `SPEC.md`/the backlog (now dashpay/platform#4020)). The goal was to catch anything the
-> DIPs require that is **missing, stubbed, or only partially wired**, and to
-> separate genuine gaps from deliberate divergences.
->
-> **Date:** 2026-06-24. **Method:** six parallel code-reading passes (xpub/ECDH/
-> key-purpose; accountReference/multi-account/DoS/label; coreHeight/block-rescan/
-> sync-window; profile/contactInfo/DPNS; dash-spv rescan capability; full DIP-16
-> 9-step sync ordering), each citing `file:line`, plus direct verification of the
-> contested findings. SPV evidence is from the pinned `dash-spv` rev `b4779fc`
-> (`rust-dashcore`), which the platform-wallet drives via `spv/runtime.rs`.
->
-> **Headline.** The DashPay (DIP-15) core flow **fully conforms** and is in places
-> *ahead* of the reference clients. The SPV layer (DIP-16) implements the hard
-> parts for real (headers-first + checkpoints + masternode-list/quorum
-> verification + compact filters) but **deliberately diverges** from the DIP's
-> literal phasing (event-driven parallel managers, BIP157 instead of BIP37 for the
-> confirmed path, L2 decoupled from L1). Only **two** DIP-15 gaps are
-> under-tracked (one a real incoming-payment-loss risk); the rest are correctly
-> tracked as deferred (blocked on external resources) or are well-reasoned
-> divergences. The §12.6 block-rescan gap (§1.1) turns out to need only a small
-> wallet-side trigger — the rescan engine already exists in dash-spv.
-
----
-
-## 0. Conformance matrix (by DIP-15 section)
-
-| DIP-15 area | § | Verdict | Evidence |
-|---|---|---|---|
-| Encrypted xpub = 69-byte compact `fp(4)‖cc(32)‖pk(33)` → 96-byte ciphertext | 8.6 | ✅ **FULLY** | `rs-platform-encryption/src/compact_xpub.rs` (`COMPACT_XPUB_LEN=69`); send asm `network/contact_requests.rs:480-491`; SDK 96-byte assert `rs-sdk/.../contact_request.rs:311-316`; KAT `dip14.rs::compact_xpub_is_69_byte_dip15_plaintext_not_107_byte_encode` |
-| ECDH `SHA256(((y&1)|2)‖x)` | 8.3 | ✅ **FULLY** | `rs-platform-encryption/src/ecdh.rs:16-26` + hand-recomputed KAT `:56-85` |
-| `senderKeyIndex`/`recipientKeyIndex` purpose policy (liberal receive, ENCRYPTION send fallback, no permanent break on purpose mismatch) | 8.3 | ✅ **FULLY** | send sel `contact_requests.rs:846-871`; validator `crypto/validation.rs:141-251`; purpose-only ≠ broken `:90-92` + drain `:1743-1764` |
-| Friendship path `m/9'/coin'/15'/0'/owner256/cp256/index`, DIP-14 256-bit non-hardened CKD | 8.9 | ✅ **FULLY** (account 0) | `crypto/dip14.rs`; byte-identical to dashj per `INTEROP_DESK_CHECK.md` |
-| `profile` (displayName/publicMessage/avatarUrl/avatarHash/avatarFingerprint) | 9 | ✅ **FULLY** | `types/dashpay/profile.rs:85-123` — real SHA-256 hash **and** real 8-byte dHash; non-destructive update `network/profile.rs:319-378` |
-| Batched profile fetch `$ownerId in [ids]` (counterparties of new requests) | 9.10 | ✅ **FULLY** | `network/profile.rs:738-812` (`In` + required `orderBy`) |
-| `contactInfo` (ECB `encToUserId`, CBC `privateData`, `65536'/65537'`, ≥2-contacts gate, varint privateData) | 10 | ✅ **FULLY** | `crypto/contact_info.rs:45-48,232-283`; `rs-platform-encryption/src/contact_info.rs`; gate `network/contact_info.rs:548-556` |
-| `accountReference` value + version-bump rotation on re-send | 7, 8.4 | ✅ **send** / ⚪ **receive ignores (by design)** | `account_reference.rs:41-51`; version bump `contact_requests.rs:514-547` |
-| `$createdAt` incremental fetch with 10-min skew back-off | 8.8, 8.12 | ✅ **FULLY** | `SYNC_OVERLAP_MS=600_000` → `contact_requests.rs:770-776`; `StartAfter` paging `contact_request_queries.rs:54-108` |
-| `$createdAtCoreBlockHeight` populated | 8.7 | ✅ **FULLY** | server-side `document_create_transition/v0/mod.rs:253-256`; client sends `None` `rs-sdk/.../contact_request.rs:478` |
-| DPNS name↔identity resolve/search/cache | 11 | 🟡 **PARTIAL** | works (`network/dpns.rs:281-362`); QR-build doesn't fall back to on-chain name |
-| **L1 block re-scan from `min(coreHeightCreatedAt)` on new contact** | **8.7, 12.6** | ❌ **MISSING** | never read to drive a rescan; SPV exposes no rescan entry point |
-| `encryptedAccountLabel` (48–80B, padded, decrypted) | 8.5 | ✅ **FULLY** | send length-normalized in the crypto primitive (`account_label.rs`); receive decrypted + surfaced via `store_contact_account_label` (incoming-only) → `ContactDetailView` (SPEC.md Milestone 3) |
-| `acceptedAccounts` + first-request bloom gating / flood mitigation | 8.4, 10.8 | ❌ **MISSING** | codec only; unpopulated + dropped on ingest |
-| Multi-account contacts (`Account ≠ 0`) | 7.1, 8.9 | 🟡 **DEFERRED** | `account_index` hardcoded `0`; blocked on upstream |
-| QR auto-accept (`autoAcceptProof`, `m/9'/5'/16'/expiry'`, BIP21/72 URI) | 8.13 | ✅ **FULLY** (iOS-first) | `crypto/auto_accept.rs`; see `QR_AUTO_ACCEPT_SPEC.md` |
-| Invitations (asset-lock voucher + claim onboarding, DIP-13) | — | ❌ **NOT STARTED** | queued as "NEXT" in the backlog (dashpay/platform#4020) |
-
----
-
-## 1. Under-tracked gaps (the value of this audit)
-
-### 1.1 🔴 No L1 block re-scan from `coreHeightCreatedAt` on new contacts — DIP-15 §8.7 + §12.6
-
-**Status: MISSING and not mentioned anywhere in the existing docs.** This is the
-only finding with an incoming-**payment-loss** character.
-
-DIP-15 §8.7 / §12.6 require: when a wallet learns of a new contact request, it must
-**resynchronize L1 blocks from the minimum `$coreHeightCreatedAt`** across the new
-requests *after* inserting the new address spaces into its filters, so it doesn't
-miss payments sent in the device-sync-speed-skew window (a payment that landed on a
-DashPay address before that address was being watched).
-
-What the code actually does:
-- `$createdAtCoreBlockHeight` **is** captured and persisted on every request
- (`types/dashpay/contact_request.rs:38`), but is **never read** to drive a
- re-request. No "minimum across new contacts" is computed anywhere.
-- Both account-registration paths — `register_external_contact_account`
- (`network/contacts.rs:389`) and `register_contact_account` (`:140`), called from
- the G1b sweep at `network/contact_requests.rs:1630,1820` — watch **forward only**
- and aren't even passed the height.
-- A newly registered contact's addresses **do** enter the compact-filter match set
- (`monitored_script_pubkeys` enumerates `all_accounts()`), but only from the
- current scan pointer forward — nothing rewinds the pointer to backfill.
-
-Consequence: the `G1(b)` sync fix rebuilds the address *watch* on restore-from-seed,
-but does **not** backfill *history*. An incoming DashPay payment that arrived before
-the receiving account was (lazily) registered — restore-from-seed, second device, or
-the offline-accept→pay window — can be silently missed until some unrelated full
-rescan happens to cover it.
-
-**The fix is small — the rescan engine already exists.** dash-spv's `FiltersManager`
-already performs a targeted backfill rescan whenever a wallet's `synced_height` drops
-below the filter scan pointer: `tick` calls `wallets_behind(committed)`, takes the
-min stale height, runs `reset_for_rescan()` + `start_download()`, and re-downloads
-BIP157 filters from there, re-matches against the now-larger script set, and
-re-requests the matching blocks (`dash-spv .../sync/filters/sync_manager.rs:213-236`,
-`manager.rs:129-139`). So DIP-15 §12.6 is **a wiring task, not an SPV build**:
-1. **platform-wallet (the actual gap):** when the G1b sweep registers a new DashPay
- account, lower that wallet's `synced_height` to
- `min($coreHeightCreatedAt over the just-built accounts) − 1`. The height is
- already on the `ContactRequest`; the existing `FiltersManager` does the rest.
-2. **one small upstream piece (`key-wallet-manager`):** `WalletInterface::
- update_wallet_synced_height` is **forward-only by contract** — "a value below the
- current is silently ignored" (`wallet_interface.rs:127-129`). A backward rescan
- needs a new guard-bypassing method (e.g. `reset_wallet_synced_height_to(id, h)`),
- a small upstream change in the vein of rust-dashcore#813. (Optionally expose a
- thin `DashSpvClient::rescan_wallet_from(id, h)` convenience wrapper; the
- `SpvRuntime` would forward it.)
-
-Constraints to respect: the backfill floor is the checkpoint the headers were seeded
-from (`manager.rs:192`), and the BIP157 filter-headers/filters for that range must be
-re-downloadable from peers. Per DIP-15 §12.6, re-request slightly beyond the minimum
-height and avoid re-requesting the final ~10 blocks near the tip. It is a genuine
-correctness gap, but a contained one — see §6.4 for how it relates to the DIP-16
-filter layer.
-
-### 1.2 🟡 `encryptedAccountLabel` — the "DONE" padding fix is dead code
-
-**Status: PARTIAL, and it contradicts a backlog ("DONE + tests pin it") claim (now dashpay/platform#4020).**
-
-The backlog P1 item records label padding to ≥16 chars (commit `2419159bb3`) as done. In
-reality:
-- The padded helper `IdentityWallet::encrypt_account_label` + `pad_account_label`
- (`network/account_labels.rs:19,49-64`) has **zero live callers** (verified by grep;
- only its own unit tests reference it).
-- The **live** path — FFI `platform_wallet_send_contact_request_with_signer`
- (`rs-platform-wallet-ffi/src/dashpay.rs:236-269`) → `send_contact_request_with_external_signer`
- (`network/contact_requests.rs:374`) → `sdk_writer` → rs-sdk — passes the host label
- **raw**. The SDK encrypts it unpadded and hard-rejects `<48 || >80` bytes
- (`rs-sdk/.../contact_request.rs:319-330`). A **1–15-character label therefore errors
- the entire contact-request send** (16-byte plaintext block → 16 ciphertext + 16 IV =
- 32 < 48). The FFI accepts a label, so this is reachable, not theoretical.
-- The label is **never decrypted on receive**: the ingest path stores
- `encrypted_account_label` as raw bytes (`contact_requests.rs:2515`) and nothing calls
- `decrypt_account_label` (also dead code in `account_labels.rs:78-107`). The field is
- effectively write-only.
-
-A later refactor (the seedless `ContactCryptoProvider`/`sdk_writer` seam) appears to
-have orphaned the padded helper.
-
-**Resolution (2026-06-24) — send side ✅ fixed; receive surfacing 🟡 remaining.**
-The DIP-15 length normalization now lives in the single primitive
-`platform_encryption::{encrypt,decrypt}_account_label`: a short/empty label is
-space-padded to clear the 48-byte floor **and** an over-long label is truncated (on a
-char boundary) to stay under the 80-byte cap — so **no** host-supplied label can error
-the broadcast anymore (the review caught that the floor fix alone left a symmetric
-`>80` long-label failure). The dead `network/account_labels.rs` helper was deleted (it
-duplicated the convention). Red→green test
-`account_label_is_always_a_valid_48_to_80_byte_field` pins both bounds + multi-byte +
-the exact-48 boundary.
-
-**Receive-side surfacing — RESOLVED (2026-06-24, 5-lens reviewed; folded into SPEC.md
-Milestone 3).** The label is now decrypted in Rust at the two signer-bearing
-register sites (drain `RegisterExternal` Ok-branch + `accept_register_external_validated`,
-where the ECDH `shared` already lives) and stored on
-`EstablishedContact.contact_account_label`. It is **direction-specific** — derived
-strictly from the *incoming* request and projected onto the **incoming FFI row only**
-(the outgoing row's label is one *we* sent and is never surfaced), so it does **not**
-copy the symmetric `alias`/`payment_channel_broken` both-rows pattern. Decrypt
-failures / non-printable garbage coerce to `None` (cosmetic — never breaks the
-channel); rotation pre-clears the field so it never goes stale. Surfaced through
-`ContactRequestFFI.contact_account_label` → `PersistentDashpayContactRequest
-.contactAccountLabel` → a read-only "Their account" row in `ContactDetailView`.
-Backfill of pre-feature contacts deferred (dev-only; DashPay unreleased).
-
-**On-device UAT (paloma, 2026-06-25) found a SECOND, decisive bug + fixed it.**
-The receive-side surfacing above had nothing to decrypt because the **recurring
-sweep's ingest parser `parse_contact_request_doc` silently dropped
-`encryptedAccountLabel`** (it read `encryptedPublicKey` + `autoAcceptProof` but not
-the label). The send always attached the label and the decrypt was always correct —
-the label just never reached the recipient's stored request. (This audit's earlier
-"ingest works" claim cited the *sent*-request parser at `:2515`, missing that the
-*received* path uses `parse_contact_request_doc`.) **Fix:** the parser now reads
-`encryptedAccountLabel`; the sender's local bookkeeping also stores it off the
-broadcast doc; and `AddContactView` gained an optional "Account label" field so
-labels can be sent in-app. Unit tests missed the bug (they built the incoming
-request *with* the label, bypassing the parser) — now pinned by
-`parse_contact_request_doc_carries_encrypted_account_label` (red→green). **Verified
-full e2e on paloma:** send (48-byte label on-chain) → fresh sweep ingest
-(`enc=48`) → accept decrypt (`contactAccountLabel="Bob savings acct"`, incoming row
-only / outgoing null) → ContactDetail shows "Their account: Bob savings acct".
-
----
-
-## 2. Tracked-and-deferred gaps (acknowledged; blocked on external resources)
-
-These are real DIP-15 gaps, but the existing docs already record them with a correct
-blocker — not oversights.
-
-| Gap | DIP-15 § | Blocker | Doc ref |
-|---|---|---|---|
-| **True multi-account (`Account ≠ 0`)** — `account_index` hardcoded `0` at the only send site (`contact_requests.rs:476`); friendship path structurally `…/15'/0'/…`. (Key *rotation* via version-bump **is** live.) | 7.1, 8.9 | upstream `rust-dashcore#813` (honor the `index` field) | backlog dashpay/platform#4020 P1/P2 |
-| **`acceptedAccounts` + §10.8 flood mitigation** — varint codec carries the field, but publish hardcodes it empty (`network/contact_info.rs:499-506`) and `set_contact_metadata` (`managed_identity/contact_requests.rs:289-299`) **drops** it on ingest. No "first request → bloom filter, additional → require acceptance" gating. | 8.4, 10.8 | query-level DoS filter needs a registered contract change | backlog dashpay/platform#4020 Contract track |
-| **Cross-device ignore sync** — ignore is local-only; a per-sender `contactInfo` leaks the ignored target (timing correlation, R1). | 10.7 | needs an encrypted field on the `profile` contract (governance) | backlog dashpay/platform#4020 Contract track |
-| **DPNS-name on-chain fallback in QR auto-accept build** — `build_auto_accept_qr` (`rs-platform-wallet-ffi/src/dashpay.rs:801`) uses the locally-cached name; empty for imported/devnet identities. `resolve_name` exists but isn't called from the QR path. | 11 | none (small follow-up) | backlog dashpay/platform#4020 P3 |
-| **DashPay Invitations** — asset-lock voucher + claim onboarding (DIP-13 sub-feature `3'`). | — | new feature (L1 funding + identity registration + deep-link) | backlog dashpay/platform#4020 "NEXT" |
-| **Devnet/testnet e2e + full add→approve→pay XCUITest** | 11 | funded test harness | backlog dashpay/platform#4020, `SPEC.md` Part 7 |
-
----
-
-## 3. Deliberate divergences (correct decisions, not bugs)
-
-- **`accountReference` ASK28 byte order** uses the **iOS** convention
- (`be(ASK[28..32])>>4`); iOS and Android genuinely disagree, and the field is a
- sender-private one-time-pad the **recipient ignores** (`unmask_account_reference` is
- only ever called by the sender's own re-send path), so there is no on-chain interop
- break. Documented + KAT-pinned.
-- **Reject → reversible local-only `ignore`** (per-sender mute), matching Android's
- Accept/Ignore model. No on-chain artifact (R1 privacy).
-- **Retained 78/107-byte xpub `decode()` fallback** in `network/contacts.rs:447-461` —
- documented insurance for local-only legacy rows; never participates in on-wire
- encoding (send only ever emits 69 bytes; the SDK rejects non-69 before encryption).
-
-### 3.1 Reference-client (dashj / kotlin-platform) source pointers
-
-For re-checking our behavior against the canonical Android stack — `dashpay/kotlin-platform`
-(`org.dashj.platform.dashpay`, the live lib), `dashpay/dashj` (core crypto/keychains),
-and `dashpay/dash-wallet` (the app: sync, UI, DAOs), all on `master`. (`android-dashpay`
-is the **stale** predecessor, last push 2024-01 — do not diff against it.) The
-reference-side anchors that pin each cross-client comparison:
-
-| Concern | Reference-client anchor |
-|---|---|
-| `accountReference` ASK28 byte order | `BlockchainIdentity.getAccountReference` = `wrapReversed(ASK).toBigInteger().toInt() ushr 4` (= `u32_le(ASK[0..4])>>4`; we use the iOS `be(ASK[28..32])>>4` — §3 above) |
-| Friendship path (receive vs send account) | `FriendKeyChain.getContactPath` — `contact.getUserAccount()` (receive) / `getFriendAccountReference()` (send) |
-| `contactRequest` pagination (drain past 100) | `Documents.getAll` loops `startAt = last.id` while `size >= 100`; `retrieveAll` ⇒ `limit(-1)` |
-| High-water + 10-min skew overlap | `PlatformSyncService.kt:346-372`, `DashPayContactRequestDao.kt:50-54` (`MAX(timestamp)` per direction) |
-| Batched contact-profile fetch | `updateContactProfiles` → `Profiles.getList` (chunks of 100, `whereIn $ownerId`) |
-| Non-destructive profile update | `Profiles.replace` — read-modify-write (`profileData.putAll(currentProfile.toObject())`, then overlay) |
-| `encryptedAccountLabel` padding | `padAccountLabel()` — pad to ≥16 chars with spaces, always emit |
-| Recipient-key selection | kotlin = ENCRYPTION-first with AUTH/HIGH fallback |
-| Sent-tx status (live, not stored) | derived from `TransactionConfidence` |
-| tx→contact reverse (both directions) | `getFriendFromTransaction` scans sent + received pools |
-| Account/keychain self-heal | `checkDatabaseIntegrity` |
-
-**Perceptual-hash caveat — do NOT write a cross-client exact-match test on
-`avatarFingerprint`.** The dHash byte/bit layout coincidentally matches dashj, but the
-pixel pipeline differs (greyscale **average vs luma-weighted**, resize filter, 9×9 vs
-9×8), so fingerprints **will not be byte-identical cross-client**. That is inherent to
-perceptual hashing — the fingerprint is used for Hamming distance, never equality — so a
-cross-client exact-match assertion is wrong by construction.
-
----
-
-## 4. Correction to the existing docs
-
-- **`SPEC.md` G3 ("`accountReference` hardcoded to 0", deferred to M3) is STALE.**
- Code verification shows the send path computes a **real** `accountReference` and
- does **version-bump rotation** on re-send (`contact_requests.rs:514-551`,
- `account_reference.rs:41-51`). Only the *account-number* multi-account case remains
- at `0` (§2 above). The leftover comment `sdk_writer.rs:114` ("DashPay account
- reference (currently 0)") is rotted and should be corrected.
-
----
-
-## 5. Where the implementation is *ahead* of the reference clients
-
-For calibration (don't "fix" these):
-- A real `contactInfo` document type — `kotlin-platform`/`dashj` have **none**.
-- A genuine 8-byte dHash `avatarFingerprint` (commonly stubbed/zeroed elsewhere).
-- Hand-recomputed ECDH + 69-byte-xpub known-answer tests (not just doc-comment trust).
-- Stricter sync re-entrancy/shutdown discipline and a more robust
- `reconcile_incoming_payments` self-heal than dashj.
-
----
-
-## 6. DIP-16 (Headers-First SPV synchronization) conformance
-
-DIP-15 §12 requires sync to follow **DIP-16**. The SPV client lives in the
-`dash-spv` crate (rev `b4779fc`), driven by `packages/rs-platform-wallet/src/spv/`.
-
-**Two architectural facts frame every verdict:**
-1. dash-spv is **not** a literal 4-phase sequential state machine. It is an
- **event-driven coordinator** that spawns 8 independent managers (block-headers,
- filter-headers, filters, blocks, masternode, chainlock, instantsend, mempool),
- each in its own tokio task, progressing reactively off a `SyncEvent` bus
- (`dash-spv/src/sync/sync_coordinator.rs:33-62,197-250`). DIP-16's *phases* are
- realized as concurrent managers, not ordered stages.
-2. The **confirmed-tx receive path uses BIP157/158 compact filters** (pulled from
- peers, matched locally against wallet scripts) — **not** BIP37 bloom. BIP37
- `filterload` exists *only* in the optional mempool (unconfirmed-tx) manager.
- DIP-16 step 8 literally says "construct a bloom filter"; the implementation
- substitutes BIP157 for the confirmed path — a deliberate, stronger-privacy
- deviation.
-
-### 6.1 Conformance matrix (DIP-16 9-step + phasing + locator)
-
-| DIP-16 element | Verdict | Evidence (`dash-spv` unless noted) |
-|---|---|---|
-| Step 1 — chain height from **multiple** peers | ✅ IMPLEMENTED (uses `max`, not soft-consensus) | `network/pool.rs:105-151` |
-| Step 2 — headers-first from checkpoints + chain/PoW validation | ✅ IMPLEMENTED | `chain/checkpoints.rs:158-725`; `sync/block_headers/pipeline.rs:56-113`; `validation/header.rs:17-46` |
-| Step 3 — terminal masternode list + quorums | ✅ IMPLEMENTED | `sync/masternodes/sync_manager.rs:255`; `manager.rs:575` |
-| Step 4 — intermediate MN lists to verify quorums | ✅ IMPLEMENTED | `sync/masternodes/sync_manager.rs:42-147,369` |
-| Step 5 — verify quorums (real, not stubbed) | ✅ IMPLEMENTED | `sync/masternodes/manager.rs:487,577` |
-| Step 6 — retrieve identities | 🟡 PARTIAL (independent, best-effort) | platform-wallet `manager/identity_sync.rs:76,397-437`; `wallet_lifecycle.rs:421` |
-| Step 7 — retrieve platform data | ✅ IMPLEMENTED (independent timer) | platform-wallet `manager/dashpay_sync.rs:404-474` |
-| **Steps 2–7 ordered in one phase** | ⚪ NOT MODELED (intentional) | `manager/mod.rs:103-195` — no cross-coordinator gating |
-| Step 8 — compact-filter build (confirmed path) | ✅ IMPLEMENTED | `sync/filters/manager.rs:654,734,779` |
-| Step 8 — **DashPay/contact addresses in filter** | ✅ IMPLEMENTED (once receival acct exists) | `key-wallet .../wallet_info_interface.rs:302-316`; reg `network/contacts.rs:223,233` |
-| Step 8 — filter set on **all** peers | 🟡 PARTIAL (eventually-all, looped not atomic; BIP37 mempool only) | `sync/mempool/sync_manager.rs:173-193`; `network/mod.rs:174-176` |
-| Step 9 — sync-from block / wallet-birthday checkpoint | ✅ capability present; birthday auto-drive soft | `chain/checkpoints.rs:138-145`; `sync/filters/manager.rs:171-175` |
-| Block-locator shape ("last 10 + prev checkpoint + genesis") | 🟡 PARTIAL (single-hash, checkpoint-segmented) | `network/mod.rs:102-107`; `sync/block_headers/segment_state.rs:67-69` |
-| Named 4-phase state machine | 🟡 PARTIAL (generic `SyncState`, no named phases) | `sync/progress.rs:9-18` |
-
-No `todo!`/`unimplemented!`/stub markers were found in the masternode/quorum or
-header/filter sync paths — the hard cryptographic parts are real.
-
-### 6.2 DIP-16 deviations (audit findings — mostly intentional, none are dead stubs)
-
-1. **No 4-phase ordering; L2 sync decoupled from L1.** Identity/platform sync run on
- independent timers with zero gating on SPV header/masternode completion. Notably,
- platform-data **proof verification does not consume the local SPV quorum state** —
- `SpvRuntime::get_quorum_public_key` exists but no sync manager calls it; proofs go
- through the SDK/DAPI path. This is the largest DIP-16 conformance gap, but appears
- to be a deliberate UX choice (don't block L2 on full L1 sync).
-2. **Single-hash block locator** instead of the DIP's multi-hash fork-recovery
- locator. Safe under checkpoint-segmented parallel download (each segment anchor is
- a validated checkpoint/tip), but a literal non-conformance with no genesis/previous
- fallback hashes in a request.
-3. **Height aggregation is `max`, not soft-consensus** — one dishonest peer
- advertising a high `start_height` inflates the sync target. Minor, but worth a note.
-4. **BIP37 mempool filter is set per-peer in a loop**, not an atomic broadcast.
-5. **Birthday-by-timestamp start is available but not obviously auto-driven** from
- platform-wallet (`get_sync_checkpoint(creation_time)` exists; default start resumes
- from persisted `synced_height`/config height).
-6. **DashPay address coverage is conditional** — addresses are watched only *after*
- the contact's funds-bearing receival account is registered; there is no pre-emptive
- watch. This is the DIP-16-layer facet of the §1.1 gap (below).
-
-### 6.3 DIP-16 does NOT mandate the §12.6 rescan — confirmed
-
-Direct fetch of DIP-16 confirms it specifies **no** "re-request blocks from height N
-after the address set grows" mechanism. Its filter section says only that Platform-app
-address spaces "can be used" in the filter and "a client should set this filter on all
-connected peers." The rewind-on-new-address behavior is a **DIP-15 §12.6** obligation
-layered on the DIP-16 base — so §1.1 is a DIP-15 gap, not a DIP-16 one.
-
-### 6.4 The rescan engine already exists at the DIP-16 filter layer
-
-Relevant to §1.1: dash-spv's filter manager **already implements** the rescan
-machinery — `reset_for_rescan()` rolls `committed_height` back and replays when a
-wallet's `synced_height` drops below scan progress, and an in-flight `rescan_batch`
-re-scans when new gap-limit scripts appear mid-batch
-(`sync/filters/manager.rs:129-139,468-505`). It is just never *triggered* for the
-DashPay backfill case, because nothing lowers `synced_height` to the contact's
-`$coreHeightCreatedAt`. That is why §1.1's fix is a small wallet-side trigger plus one
-upstream guard-bypass method, not an SPV build.
-
----
-
-## 7. Recommended priority
-
-1. **§1.1 coreHeight block re-scan (DIP-15 §12.6)** — the only untracked
- correctness/payment-loss item. Now scoped small: a wallet-side `synced_height`
- rewind on new-contact registration + one upstream `reset_wallet_synced_height_to`
- method; the dash-spv `FiltersManager` rescan engine already does the rest.
-2. **§1.2 account-label** — ✅ DONE. Send length-normalization fixed; receive-side
- decryption + UI surfacing implemented (incoming-only) per
- SPEC.md Milestone 3. DIP-15 §8.5 now fully conforms.
-3. **DIP-16 deviations (§6.2)** — mostly intentional; if any is worth hardening it is
- #1 (consider sourcing proof-verification quorum keys from the local SPV engine) and
- #3 (height soft-consensus). Track, don't rush.
-4. Everything in §2 stays blocked on its external dependency; §3 is intentional.
diff --git a/docs/dashpay/IDENTITY_KEY_SCALAR_ELIMINATION_SPEC.md b/docs/dashpay/IDENTITY_KEY_SCALAR_ELIMINATION_SPEC.md
deleted file mode 100644
index 885c213a433..00000000000
--- a/docs/dashpay/IDENTITY_KEY_SCALAR_ELIMINATION_SPEC.md
+++ /dev/null
@@ -1,493 +0,0 @@
-# Identity-Key Scalar Elimination — derive-sign-destroy for discovered keys
-
-Status: draft, rev 2 (review must-fixes folded)
-Scope: removes the carried 32-byte ECDSA scalar
-(`IdentityKeyEntry.private_key` / `KeyWithBreadcrumb.verified_scalar`) from the
-identity-key discovery → persist → sign flow, replacing it with a
-derive-sign-destroy model in which the per-key secret only ever exists in the iOS
-Keychain (as the wallet seed), is derived on demand at sign time, and is never
-carried across the Rust→FFI→Swift boundary or stored per-key.
-
-Supersedes the earlier carried-scalar storage posture (the carry-the-verified-scalar
-fix this spec reverses). Aligns with the seed-elimination §4.9-blocker item 3 design
-and the sibling decision to stop persisting the DashPay friendship xpub and re-derive
-on load.
-
-> **Review outcome (rev 2).** Four independent reviewers (feasibility, scope,
-> security, crypto/domain) audited rev 1 against the code. The crux correctness
-> claim — pubkey-compare is byte-for-byte equivalent to
-> `validate_private_key_bytes(scalar)` — was **verified exact** for both
-> `ECDSA_SECP256K1` (33-byte compressed compare) and `ECDSA_HASH160`
-> (`ripemd160_sha256(pubkey)` vs `key.data()`; the double-hash gotcha does **not**
-> apply because we compare `key.data()`, not `entry.public_key_hash`). No key is
-> wrongly authorized and no wallet-derivable key is wrongly dropped to watch-only.
-> The must-fixes folded below are all about the **migration**, where removing the
-> carried scalar trades an *intrinsic* scalar↔pubkey binding for a *trusted-path*
-> binding — the real lockout surface. The five load-bearing corrections:
-> **(MF-1)** the backfill reads the Keychain **metadata blob** (named fields), not
-> a parse of the account-string label; **(MF-2)** the backfill **and** the sign
-> path **re-derive the pubkey at the path and compare to the row's
-> `publicKeyData`** before trusting/signing — a present-but-wrong path otherwise
-> signs silently and consensus rejects it (silent lockout); **(MF-3)** the
-> scalar-field-deletion gate is a **runtime migration stamp**, not a dev-time
-> check, and the field-deleted build still runs the Keychain-driven backfill on
-> first launch (the Keychain survives a SwiftData store rebuild); **(MF-4)** the
-> SwiftData column addition must be a verified-clean lightweight migration (or an
-> explicit `MigrationStage`), since this app has historically rebuilt the V1 store
-> from scratch; **(MF-5)** the schema delta is exactly `walletId` +
-> `identityDerivationPath`, and the FFI layout guard recomputes to exactly **184**.
-
----
-
-## 1. Problem
-
-Identity-key discovery carries a verified 32-byte ECDSA scalar **Rust → FFI →
-Swift** so the iOS Keychain stores it directly:
-
-- `discovery.rs::derive_key_breadcrumbs` derives a candidate scalar per on-chain
- key; `breadcrumb_decisions` gates each via
- `IdentityPublicKey::validate_private_key_bytes(scalar, network)` and carries the
- reproducing scalar as `KeyWithBreadcrumb.verified_scalar`
- (`changeset.rs:391`).
-- It rides `IdentityKeyEntry.private_key` (`changeset.rs:434`, `#[serde(skip)]`,
- redacting `Debug`), is copied by value into `IdentityKeyEntryFFI.private_key:[u8;32]`
- (`identity_persistence.rs:312`, with `private_key_is_some`), and Swift writes the
- 32 bytes to the Keychain via `storeCarriedIdentityKey` →
- `KeychainManager.storeIdentityPrivateKey` under account
- `identity_privkey..`.
-- At sign time the `keyType < 5` branch reads that stored scalar back out
- (`KeychainSigner.swift::lookupIdentityPrivateKey` → `ffiSign`).
-
-This is not a resident keystore — the scalar transits same-tick, is `Zeroizing`,
-never serialized, never written to SQLite, and **no Rust signing path reads it**
-(Rust signs via the external `VTableSigner`). It exists because the imported-wallet
-flow ran `createWallet` (→ discovery) *before* `storeMnemonic`, so the old
-Swift re-derive-from-mnemonic produced 23/23 watch-only keys; carrying the
-already-verified scalar removed Swift's mnemonic dependency (commit `c567981c46`).
-
-**Why change it anyway.** The carried scalar is still a raw secret crossing the FFI
-ABI and stored per-key at rest. The clean model — already proven for platform
-addresses — keeps the only secret (the seed) in the Keychain and derives each
-signing key on demand. Removing the carry yields: the raw scalar never crosses the
-FFI; one fewer class of secret at rest (no per-key scalar); Rust discovery never
-materializes the scalar at all (verify via public key).
-
-**The hard part.** This is the single most safety-critical path in the wallet: a
-wrong key-storage/resolution change **locks users out of signing**, and the change
-is only *validatable* against the iOS Keychain signer (iOS-gated). The design must
-make the cutover non-lockout **by construction**.
-
----
-
-## 2. Current vs. target architecture
-
-### Current (carried scalar)
-
-```
-discovery.rs derive candidate scalar ── validate_private_key_bytes(scalar) ──┐
- │ verified_scalar: Some
-changeset KeyWithBreadcrumb.verified_scalar ─► IdentityKeyEntry.private_key│
-FFI IdentityKeyEntryFFI.private_key[32] + private_key_is_some (by value)
-Swift store storeCarriedIdentityKey ─► Keychain item identity_privkey.. (32 raw bytes)
-Swift sign keyType<5 ─► lookupIdentityPrivateKey (read scalar back) ─► ffiSign
-```
-
-### Target (derive-sign-destroy)
-
-```
-discovery.rs derive candidate PUBLIC key ── compare to on-chain pubkey ──┐ (no scalar materialized)
- │ breadcrumb: Some, scalar: ABSENT
-changeset KeyWithBreadcrumb{ key, breadcrumb } (no verified_scalar)
-FFI IdentityKeyEntryFFI{ …, wallet_id, identity_index, key_index } (breadcrumb only — already crosses)
-Swift store persistIdentityKeys ─► PersistentPublicKey.{walletId, identityDerivationPath} (queryable columns)
-Swift sign keyType<5 ─► resolveIdentityKeyContext ─► dash_sdk_sign_with_mnemonic_resolver_and_path
- (resolve mnemonic in-callback ─► derive ─► sign ─► zeroize; only the signature returns)
-```
-
-The breadcrumb `(wallet_id, identity_index, key_index)` **already crosses the FFI**
-(`identity_persistence.rs` `wallet_id`/`identity_index`/`key_index`, gated behind
-`wallet_id_is_some` / `derivation_indices_is_some` — present whenever the entry has
-a breadcrumb, independent of the scalar). It is currently used only to build the
-Keychain account label and is then discarded. So `persistIdentityKeys` must guard
-the new-column write on `entry.derivationIndices != nil` (the snapshot already
-exposes `derivationIndices` and `walletId`).
-
----
-
-## 3. Chosen approach
-
-Mirror the **platform-address** derive-sign-destroy path, which already works
-end-to-end, and reuse its Rust primitive.
-
-### 3.1 Discovery verifies via public key (no scalar)
-
-The DIP-9 identity-auth path `m/9'/coin'/5'/0'/ECDSA'/identity_index'/key_index'`
-is fully hardened, so the candidate pubkey must still be derived from a master
-xpriv (resolved on demand inside the FFI, wiped before return — already the case
-for external-signable wallets). The change is local to discovery: compute the
-candidate **compressed public key**
-(`derive_ecdsa_identity_auth_keypair_from_master(..).public_key`) and compare to
-the on-chain key — `key.data() == pubkey` for `ECDSA_SECP256K1`,
-`ripemd160_sha256(pubkey) == key.data()` for `ECDSA_HASH160` — instead of calling
-`validate_private_key_bytes(scalar)`. **Do not populate `candidate_scalars`.**
-
-This is byte-for-byte the same decision `validate_private_key_bytes` makes
-internally; it just never needs the scalar to leave the derive function. The
-transient master resolution in discovery is **not** what we eliminate — the
-persisted/carried per-key scalar is.
-
-An uncompressed externally-registered ECDSA key (65-byte on-chain `data()`)
-correctly stays watch-only because the wallet only ever derives the **compressed**
-form, so the compare gracefully fails — *not* because Platform forbids uncompressed
-identity keys (it does not; `UncompressedPublicKeyNotAllowedError` is an asset-lock
-constraint only). Stating the real reason avoids a future "optimization" that
-assumes uncompressed identity keys can't exist on-chain.
-
-### 3.2 Sign via the existing resolver primitive (no new FFI)
-
-`dash_sdk_sign_with_mnemonic_resolver_and_path`
-(`rs-platform-wallet-ffi/src/sign_with_mnemonic_resolver.rs`) is **generic over the
-derivation path and ECDSA-only**. Every wallet-derivable identity auth key is ECDSA
-(guaranteed by the discovery verify gate), so the identity signing branch calls
-this primitive **unchanged** with the DIP-9 identity-auth path string. No new FFI
-signing entry point is required.
-
-**Sign-time binding check (MF-2).** Today's stored-scalar lookup is *intrinsically*
-correct — the scalar it returns was the one verified to reproduce that exact pubkey
-at discovery. The resolver path loses that: it routes by `wallet_id_bytes` and
-derives at `identityDerivationPath`, but never confirms the result matches the key
-being signed for. A mis-mapped resolver slot or a stale path would derive a
-*different, valid* scalar and produce a signature consensus silently rejects. So the
-identity sign path **must verify the derived compressed pubkey equals the row's
-`publicKeyData` before signing** (derive-and-compare inside the FFI, or a
-pubkey-preview call before `sign`), and fail loud on mismatch rather than emit a
-wrong-key signature. This restores the intrinsic binding the stored scalar gave for
-free.
-
-`signIdentityKeyOnDemand` and `signPlatformAddressOnDemand` differ only in their
-SwiftData lookup; the `sigBuf` setup + FFI call + error handling are identical.
-Extract a shared private `signOnDemandWithContext(walletId:path:expectedPubKey:data:)`
-so the two branches don't duplicate ~30 lines (the identity branch passes
-`expectedPubKey`, enabling the MF-2 check; the address branch passes `nil`).
-
-### 3.3 Persist the breadcrumb as queryable columns
-
-`PersistentPlatformAddress` carries `walletId: Data` + `derivationPath: String` and
-the signer reads them in `resolvePlatformAddressContext`. `PersistentPublicKey`
-carries only `privateKeyKeychainIdentifier` — no breadcrumb. Add **exactly two**
-columns: `walletId: Data?` and `identityDerivationPath: String?`. Both are required
-by the resolver FFI — `wallet_id_bytes` is a mandatory parameter (it keys the
-mnemonic-resolver callback), and the full path string is what
-`dash_sdk_sign_with_mnemonic_resolver_and_path` derives at. **Do not** add separate
-`identityIndex`/`keyIndex` columns — they are redundant with the path string (the
-inverse of `getIdentityAuthenticationPath`) and the path is authoritative.
-`persistIdentityKeys` writes the two columns, building the path with
-`KeyDerivation.getIdentityAuthenticationPath` — the same path
-`storeCarriedIdentityKey` already computes and currently throws away — and **always
-overwrites** when the FFI breadcrumb is present, so a backfilled value and a
-fresh-persister value for the same row are byte-identical (a string-format drift
-between the two would otherwise desync the stored path from what the resolver
-re-derives).
-
-**Migration safety (MF-4).** Two optional columns are the additive shape SwiftData
-lightweight migration handles — *but* this app's `DashModelContainer` runs
-`DashSchemaV1` with `stages: []` and has historically rebuilt the dev store from
-scratch on any model-hash change. Adding columns must be confirmed to
-lightweight-migrate **a real persisted production store on upgrade** (not just a
-fresh install); if SwiftData instead rebuilds the store, every
-`PersistentPublicKey` row vanishes and the §5 backfill has no rows to heal. Either
-verify the clean lightweight path on a real upgrade or bump to `DashSchemaV2` with
-an explicit additive `MigrationStage`. Because the backfill is **Keychain-driven**
-(§5) and the Keychain survives a SwiftData rebuild, a wiped row set degrades to
-re-materialization from the Keychain rather than to lockout — but the migration
-shape must still be pinned, not assumed.
-
-### 3.4 Alternatives rejected
-
-- **Keep the carried scalar (status quo).** Rejected: leaves a raw secret crossing
- the ABI and a per-key secret at rest; diverges from the platform-address model
- and the broader "derive on load, don't persist secrets" direction.
-- **Re-derive in Swift from the mnemonic at sign time (the pre-`c567981c46`
- path).** Rejected: this is exactly the anti-pattern `swift-sdk/CLAUDE.md` forbids
- (Swift running `mnemonic → seed → path → key`), and it was the original
- imported-identity bug. The resolver primitive keeps the derive inside Rust with
- only the path crossing.
-- **New identity-specific FFI signing call carrying `(identity_index, key_index)`.**
- Rejected as unnecessary now: the generic path primitive already covers every
- ECDSA identity key. (A non-ECDSA wallet-derivable identity key — none exist today
- — would be the only reason to add one.)
-
----
-
-## 4. Phased delivery
-
-**Hard ordering invariant:** the FFI/changeset scalar field is deleted **last**,
-only after every already-materialized identity key is proven to sign via the
-resolver path on a real device. Deleting it earlier — even though Rust still
-compiles — bricks the still-scalar-based Swift signer = lockout.
-
-### Phase 1 — headless-safe (Rust + FFI only; additive, removes nothing Swift reads)
-
-Gate: `cargo test -p platform-wallet` + `-p rs-platform-wallet-ffi` green;
-cross-compile `aarch64-apple-ios-sim`. No ABI change.
-
-1. Compute the pubkey-compare decision in `breadcrumb_decisions` and **assert in
- tests** it is byte-equivalent to the scalar decision (`reproduces`). This is
- *not* a second parallel derive: the candidate pubkey is already a byproduct of
- the existing keypair derivation, so the only change is what the decision logic
- compares. Production emission is unchanged — `verified_scalar: Some` still
- ships in Phase 1 because Swift still reads it; the switch to pubkey-only happens
- in Phase 2 step 6.
-2. Test the identity DIP-9 path through `dash_sdk_sign_with_mnemonic_resolver_and_path`
- (the existing happy-path test already uses `m/9'/1'/5'/0'/0'/0'/0'` — confirm it
- covers the identity case or augment it; this step may be test-only, no new code).
-3. Test that the FFI breadcrumb round-trips with `private_key_is_some == false`.
-
-Steps 1–3 have no internal ordering dependency and can land as one atomic Rust commit.
-
-### Phase 2 — iOS-gated (Swift + on-device), ordered
-
-1. Add `walletId` + `identityDerivationPath` columns to `PersistentPublicKey` (both
- optional ⇒ SwiftData lightweight migration; existing rows get `nil`; no data
- loss).
-2. Write the breadcrumb columns in `persistIdentityKeys` — **both** old (Keychain
- scalar) and new (columns) during the transition window.
-3. **Backfill migration (the lockout defense — see §5).** One-time, Keychain-driven,
- self-verifying pass populating the new columns from each `identity_privkey.*`
- item's `IdentityPrivateKeyMetadata` blob, re-deriving the pubkey at the path and
- comparing to the row's `publicKeyData` before trusting it — no network, no seed.
-4. Re-route the `keyType < 5` signer branch through `signIdentityKeyOnDemand` +
- `resolveIdentityKeyContext` (mirroring the platform-address pair), calling the
- existing resolver primitive. **Resolver-first with legacy fallback:** a row with
- no `identityDerivationPath` falls back to `lookupIdentityPrivateKey → ffiSign`;
- every fallback hit is logged (count only, no key material).
-5. **Validation gate:** transitional build on a funded testnet wallet; exercise
- signing for every identity (DPNS register, profile set, contact-request
- send+accept, payment); confirm **zero fallback hits** after backfill.
-6. **Only after the gate:** flip discovery to pubkey-only; delete
- `storeCarriedIdentityKey`, the Swift scalar copy/scrub, and the legacy signer
- path; then delete the scalar field from FFI (recompute the layout guard from
- `const _: [u8; 224]` to exactly `const _: [u8; 184]` — removing
- `private_key_is_some` at offset 184 + `private_key: [u8; 32]` + trailing padding
- drops bytes 184–223, alignment stays 8) and changeset; regenerate the header;
- rebuild.
-
-**Runtime deletion gate (MF-3) — not a dev-time gate.** Step 6 must not assume every
-device passed through the transitional build. A user can upgrade straight from the
-scalar-only build to the field-deleted build, skipping the backfill; their rows have
-`identityDerivationPath == nil` and there is no legacy signer left → lockout. So:
-(a) the field-deleted build **still runs the Keychain-driven backfill on first
-launch** (the Keychain items survive any SwiftData rebuild, so the path is
-recoverable even with no transitional run); and (b) deleting the *legacy signer
-fallback* is gated on a **persisted migration stamp** (set only after a backfill
-pass leaves zero un-pathed rows / after the scalar Keychain items are purged), so a
-binary without the stamp keeps the fallback and schedules a backfill. The fallback,
-not just the field, is the safety net — it lives until the stamp guarantees the
-resolver path covers 100 % of the live key set at runtime.
-
----
-
-## 5. Migration / back-compat — the lockout defense (new design)
-
-**Danger:** existing installs have scalars in the Keychain under
-`identity_privkey..` and `PersistentPublicKey` rows whose
-new breadcrumb columns are `nil` after the lightweight migration. If signing flips
-to resolver-only and a row has no `identityDerivationPath`, that key is unsignable
-→ the user is locked out of an already-working identity.
-
-Three layers, all required:
-
-1. **Keychain-metadata-driven, self-verifying backfill (no network, no seed)
- (MF-1, MF-2).** Each `identity_privkey.*` Keychain item carries a first-class
- `IdentityPrivateKeyMetadata` JSON blob (`kSecAttrGeneric`) with named `walletId`,
- `derivationPath`, `identityIndex`, `keyIndex`, `publicKey` fields —
- `KeychainManager.identityPrivateKeyAccount` already walks every row and decodes
- it. The one-time backfill reads `walletId` + `derivationPath` from the **blob**
- (not from a parse of the `identity_privkey..` account
- label, which is fragile and has a legacy no-`walletId` variant). It is driven by
- the **Keychain item set**, not the SwiftData row set, so it heals even if the
- SwiftData store was rebuilt (MF-4) — it can re-create the `PersistentPublicKey`
- linkage from the blob's `publicKey`. Crucially it is **self-verifying**: before
- writing `identityDerivationPath`, re-derive the compressed pubkey at that path
- (resolver / pubkey-preview FFI) and require it to equal the row's
- `publicKeyData`; on mismatch leave the column `nil` so the row falls through to
- the fallback / re-discovery rather than to a wrong-key sign. A non-zero count of
- parse-or-verify failures is a **hard blocker** on the deletion gate (it is not
- enough to test that signing works — every existing item must be accounted for).
-2. **Resolver-first with legacy fallback.** During the transition the signer tries
- the resolver path first and falls back to the stored scalar when the breadcrumb
- is absent, so a row can always sign via at least one path. Non-lockout by
- construction. (Note: the fallback covers an *absent* path, not a *present-but-
- wrong* one — MF-2's sign-time binding check is what catches the latter.)
-3. **Re-discovery heals the rest.** Any row not covered by (1) re-materializes on a
- from-0 rescan (now writing the breadcrumb columns, needing only the resolver
- mnemonic). Surface a "re-scan identities" affordance.
-
-**Population that blocks deletion (MF-3 / R5).** A wallet with a materialized scalar
-but **no readable mnemonic** (the import-flow case that motivated the carried scalar
-originally) can never reach zero resolver-fallbacks — the resolver needs the
-mnemonic. For that population the legacy scalar fallback must be **retained**, or an
-explicit mnemonic-import step required, before its scalar field/path can be removed.
-The "zero fallback hits" criterion is otherwise unachievable for exactly the wallets
-the carried scalar was introduced to serve.
-
-The legacy fallback and the scalar field are deleted **only** after the runtime gate
-(MF-3) confirms, per device, that the resolver path covers the full live key set —
-not merely after a dev-time test pass.
-
----
-
-## 6. Failure modes & risk register
-
-| ID | Risk | Mitigation |
-|----|------|------------|
-| R1 | Pubkey-verify diverges from scalar-verify → a key wrongly breadcrumbed (signable with an unauthorized key) or wrongly watch-only | Phase-1 byte-equivalence test over `ECDSA_SECP256K1` + `ECDSA_HASH160` + foreign key; assert decision set identical to `breadcrumb_decisions` |
-| R2 | Existing rows lack breadcrumb columns → resolver-only signer locks out already-materialized keys | §5: backfill migration + resolver-first-with-fallback + zero-fallback gate before deleting legacy |
-| R3 | Wrong network → wrong DIP-9 path → wrong key / sign failure | Resolve network from `PersistentWallet` exactly as `storeCarriedIdentityKey` does; unit-test the built path equals the Keychain account string |
-| R4 | ABI/layout drift on field removal → `EXC_BAD_ACCESS` in the callback | Recompute `const _: [u8; N]` + the byte-offset comment; cbindgen regen; round-trip test |
-| R5 | Resolver mnemonic missing/locked at sign time (watch-only, biometric-gated, import-only wallet with a scalar but no mnemonic) → sign fails where the stored scalar succeeded; zero-fallback gate unachievable for this population | Existing `mnemonicMissing` UX; **retain the legacy scalar fallback for the no-mnemonic population** (§5) — do not delete its scalar/path until a mnemonic-import step runs |
-| R6 | A non-ECDSA wallet-derivable identity key appears (future) → the ECDSA-only resolver rejects it | Pre-existing constraint, **not introduced by this change** (the scalar path is already ECDSA-only via `validate_private_key_bytes`); discovery only breadcrumbs ECDSA; a non-ECDSA key would need a new resolver FFI |
-| R7 | Old per-key scalars linger in the Keychain indefinitely → negates "one fewer secret at rest" | **Required (not optional)** purge of `identity_privkey.*` items, gated on the same runtime stamp; also doubles as the MF-3 migration-completed signal |
-| R8 | **Skip-version upgrade lockout** — user jumps from scalar-only to field-deleted build, skipping the backfill; rows have `identityDerivationPath == nil` and no legacy signer remains | MF-3: field-deleted build still runs the **Keychain-driven** backfill on first launch (Keychain survives a SwiftData rebuild); legacy-fallback deletion gated on a persisted migration stamp, not a dev-time check |
-| R9 | **Wrong-mnemonic / present-but-wrong-path silent signing** — resolver routes by `wallet_id_bytes` and derives a valid-but-wrong scalar; signature fails only at consensus, no local diagnostic | MF-2: derive-and-compare the pubkey to the row's `publicKeyData` before signing (and in the backfill before trusting a path); fail loud on mismatch |
-| R10 | SwiftData store rebuild on column add wipes `PersistentPublicKey` rows → backfill has nothing to heal | MF-4: verify clean lightweight migration on a real upgrade or declare a `MigrationStage`; backfill is Keychain-driven so a wiped row set degrades to re-materialization, not lockout |
-
----
-
-## 7. Test / verification plan (red→green)
-
-**Phase 1 (headless):**
-- `discovery.rs` `#[cfg(test)] mod tests` — `breadcrumb_via_pubkey_equivalence`:
- derive a multi-key identity, run both the scalar path and the new pubkey path,
- assert identical `(breadcrumb, key)` decisions and that the pubkey path carries no
- scalar; extend the existing HASH160 + non-reproducible-key tests. **Red first**
- (new path wrong), then green. This is the most important Rust correctness gate (R1).
-- `sign_with_mnemonic_resolver.rs` tests — `signs_with_dip9_identity_auth_path`
- (identity path string, verify signature). Confirms no new FFI is needed.
-- `identity_persistence.rs` tests — breadcrumb survives `from_entry` with
- `private_key_is_some == false`; (Phase-2 step 6) update the size guard to the new
- `N` and assert no scalar field.
-- `rs-platform-wallet-storage` round-trip — `IdentityKeyWire` still has no secret
- field; compiles after `private_key` removal.
-
-**Phase 2 (iOS/sim):**
-- `KeychainSignerIdentityResolveTests` — `signIdentityKeyOnDemand` resolves
- `(walletId, identityDerivationPath)` from a seeded row and signs via a mock
- resolver; `canSign` is true with breadcrumb+mnemonic, false without. **Plus the
- MF-2 binding test:** a resolver returning a *wrong* mnemonic (or a row with a
- *wrong* path) yields a **sign-failure, not a wrong-key signature** — assert the
- pre-sign pubkey compare rejects it.
-- `PersistentPublicKeyBreadcrumbMigrationTests` — seed a Keychain
- `identity_privkey.*` item (with its `IdentityPrivateKeyMetadata` blob) + a row;
- run backfill; assert columns are populated **from the blob** and that the
- backfilled path **re-derives to the row's `publicKeyData`** (MF-2 self-verify);
- assert a blob whose path does *not* re-derive to its pubkey leaves the column
- `nil`; assert a row whose backfilled value and a fresh-persister value are
- byte-identical; assert a row without a Keychain item falls back to legacy during
- the transition; assert backfill works with the **SwiftData row set empty**
- (Keychain-driven, MF-4).
-- `BackfillCoverageTests` — every existing `identity_privkey.*` item is accounted
- for; a non-zero parse-or-verify-failure count blocks the deletion gate (MF-1).
-- `persistIdentityKeys` writes the two columns from a breadcrumb-only
- (scalar-absent) entry, guarded on `derivationIndices != nil`.
-
-**On-device acceptance (the real gate):**
-- Transitional build over an existing store with already-materialized identities →
- backfill runs → exercise signing for every identity (DPNS / profile / contact
- request / payment) → **zero legacy-fallback hits** logged.
-- Fresh wipe → import funded testnet seed → discover (pubkey-verify, no scalar
- carried) → sign → success.
-- Wrong-seed rejection (`verify_seed_binds`) still holds.
-
-**Field deletion is gated:** `git grep verified_scalar` /
-`IdentityKeyEntry.private_key` empty only after the zero-fallback on-device gate
-passes. If fallbacks > 0, **do not delete** — the scalar is the safety net until the
-resolver path is proven for 100 % of the live key set.
-
----
-
-## 8. Critical files
-
-- `packages/rs-platform-wallet/src/wallet/identity/network/discovery.rs` —
- pubkey-verify in `breadcrumb_decisions` / `derive_key_breadcrumbs`; equivalence test.
-- `packages/rs-platform-wallet-ffi/src/sign_with_mnemonic_resolver.rs` — reuse the
- signing primitive; add an **optional `expected_pubkey` param** for the MF-2
- derive-and-compare-before-sign check (the address path passes none); add a
- DIP-9-path sign test + a wrong-seed-rejects test.
-- `packages/rs-platform-wallet-ffi/src/identity_persistence.rs` — (Phase 2 step 6)
- delete `private_key` / `private_key_is_some`; recompute the layout guard
- `const _: [u8; 224]` → `const _: [u8; 184]` (drops bytes 184–223; align stays 8);
- update the byte-offset comment.
-- `packages/rs-platform-wallet/src/changeset/changeset.rs` — (Phase 2 step 6) delete
- `KeyWithBreadcrumb.verified_scalar` + `IdentityKeyEntry.private_key`.
-- `packages/rs-platform-wallet/src/wallet/identity/state/managed_identity/identity_ops.rs`
- — `add_keys`: drop the scalar from the destructure + entry literal.
-- `packages/swift-sdk/Sources/SwiftDashSDK/Persistence/Models/PersistentPublicKey.swift`
- — add `walletId` + `identityDerivationPath`.
-- `packages/swift-sdk/Sources/SwiftDashSDK/Persistence/DashModelContainer.swift` —
- pin the lightweight migration / `MigrationStage` (MF-4); host the persisted
- migration stamp gating legacy-path deletion (MF-3).
-- `packages/swift-sdk/Sources/SwiftDashSDK/Security/KeychainManager.swift` —
- Keychain-driven backfill reads the `IdentityPrivateKeyMetadata` blob (`walletId`,
- `derivationPath`); required purge of `identity_privkey.*` after the gate (R7).
-- `packages/swift-sdk/Sources/SwiftDashSDK/FFI/KeychainSigner.swift` —
- `signIdentityKeyOnDemand` + `resolveIdentityKeyContext` mirroring the
- platform-address pair; fallback dispatch; delete `lookupIdentityPrivateKey` /
- `ffiSign` in step 6.
-- `packages/swift-sdk/Sources/SwiftDashSDK/PlatformWallet/PlatformWalletPersistenceHandler.swift`
- — write breadcrumb columns; delete `storeCarriedIdentityKey` + the scalar
- copy/scrub.
-
----
-
-## 9. As-built notes (what shipped vs. this spec)
-
-The additive, fallback-protected work (Phase 1 + Phase 2 steps 1–5) shipped on
-`feat/dashpay-identity-key-scalar-elimination`. Two deliberate deviations from the
-rev-2 design above, plus what is explicitly **not** done:
-
-- **Resolver FFI accepts `ECDSA_HASH160` (key_type 2), not just SECP256K1.** Full
- scalar deletion is impossible otherwise — discovery breadcrumbs both ECDSA key
- types. The MF-2 binding disambiguates by `expected_key_data` length: 33 bytes =
- compressed-pubkey equality, 20 bytes = `ripemd160_sha256(pubkey)` equality. The
- param is nullable (the address path passes none). This widens §3.2's "reuse the
- primitive unchanged."
-- **The backfill self-check is canonical-path-from-indices, not a pubkey
- re-derivation.** §5 layer 1 specified re-deriving the pubkey at the path and
- comparing to `publicKeyData`; that needs the seed, which the backfill
- deliberately avoids. Instead it rebuilds the canonical DIP-9 path from the
- metadata's `(network, identityIndex, keyIndex)` and requires it to equal the
- stored `derivationPath` (rejecting format drift), and **relies on the sign-time
- MF-2 binding as the real guard** — a present-but-wrong path yields
- `ERR_PUBKEY_MISMATCH` at sign time → a logged `IDENTITY_SIGN_FALLBACK`, never a
- wrong-key signature. A non-zero backfill failure count is still surfaced.
-- **Phase 2 step 6 — the carried scalar IS deleted; the legacy fallback signer is
- KEPT (partial by design).** `KeyWithBreadcrumb.verified_scalar`,
- `IdentityKeyEntry.private_key`, and `IdentityKeyEntryFFI.{private_key,
- private_key_is_some}` are removed (FFI layout guard recomputed `224` → `184`; the
- `from_entry` copy + free-scrub gone). Discovery now derives-verifies-**drops** the
- candidate scalar instead of emitting it — verification is still
- `validate_private_key_bytes`, the scalar just never leaves discovery. The legacy
- Keychain-scalar signer (`lookupIdentityPrivateKey` / `ffiSign`) is **retained** so
- keys already materialized on existing installs still sign: the §5 lockout defense
- is preserved *without* removing the legacy path. New keys are resolver-only.
- Merged into `feat/dashpay-m1-sync-correctness` (`fb11783706`; commits `930c100c64`
- deletion + `fa1098c081` test).
-- **The funded-testnet zero-`IDENTITY_SIGN_FALLBACK` gate (§4 step 5 / §7) was
- un-runnable and was substituted.** idb cannot actuate this app's SwiftUI
- confirmation controls (the toolbar Create/Cancel, the backup-seed "I wrote it
- down" switch), and the macOS-click fallback needs Accessibility / Automation /
- Screen-Recording TCC the tmux-hosted shell lacks — so the automated UAT could not
- even create a wallet. With explicit sign-off ("delete the field if it passes"),
- the gate was replaced by a HEADLESS Swift integration test
- (`IdentityResolverSignIntegrationTests`): it seeds a real mnemonic in
- `WalletStorage` + a consistent breadcrumb row and asserts `signIdentityKeyOnDemand`
- → resolver → on-demand derive → MF-2 binding → valid 65-byte signature (plus a
- wrong-path → `.failure`). It stays green *after* the deletion — proof the resolver
- path signs through the Swift layer with no stored scalar. Verified end to end:
- platform-wallet 314 + FFI (125/26/9) tests, Swift IdentityResolverSign 2/2 +
- IdentityKeyBreadcrumb 4/4 + 30 Identity tests, clippy clean, SwiftExampleApp sim
- BUILD SUCCEEDED. Because the legacy fallback was retained, the `DashModelContainer`
- migration-stamp (MF-3) stays deferred (it only matters once the fallback is
- removed). Still unproven: a live on-device discover→materialize→sign over an
- existing store.
diff --git a/docs/dashpay/INTEROP_DESK_CHECK.md b/docs/dashpay/INTEROP_DESK_CHECK.md
index 205b40c5f7c..4a42145d843 100644
--- a/docs/dashpay/INTEROP_DESK_CHECK.md
+++ b/docs/dashpay/INTEROP_DESK_CHECK.md
@@ -4,8 +4,8 @@
> transient `research/` directory was trimmed — older citations of
> "`research/06`" refer to this file. Kept in the shipped docs because it is
> the evidence base for the consensus-facing wire-format decisions
-> (69-byte compact xpub, key-purpose envelope, ASK28 byte order) cited by
-> `SPEC.md` and `DIP_CONFORMANCE_GAPS.md`.
+> (69-byte compact xpub, key-purpose envelope, ASK28 byte order) the wallet
+> implements.
Research date: 2026-06-10 (Milestone 1, task 5 — verify-only).
Question: do THIS stack's DashPay wire formats match the reference clients (iOS DashSync,
@@ -463,3 +463,48 @@ is unbound, and mobile recipients have no DECRYPTION key for us to select when s
(e.g. legacy 2024 AUTHENTICATION docs), degrade to a warning/skip — do **not**
permanently mark the payment channel broken, since on-chain history demonstrably
contains nonconforming-but-honest documents.
+
+## Addendum (2026-08-11): the receive side accepts legacy key purposes
+
+The alignment recommendation above is superseded on the receive side by
+[#4372](https://github.com/dashpay/platform/pull/4372). A mainnet wallet with 29
+contacts established through the legacy Android/dashj client had 27 inbound
+requests that reference the recipient's AUTHENTICATION or TRANSFER key, and the
+documents are immutable, so rejecting them left those contacts unpayable. The
+rules now are (rs-sdk `platform/dashpay/contact_request.rs`):
+
+- **Send (documents we create):** unchanged. The sender key must be ENCRYPTION
+ (`sender_key_purpose_is_valid`) and the recipient key DECRYPTION or ENCRYPTION
+ (`recipient_key_purpose_is_valid`).
+- **Receive (documents already on chain):** the recipient key may be DECRYPTION,
+ ENCRYPTION, AUTHENTICATION or TRANSFER
+ (`recipient_key_purpose_is_acceptable_on_receive`), and the sender key
+ ENCRYPTION or AUTHENTICATION (`sender_key_purpose_is_acceptable_on_receive`).
+ Any other purpose is a mismatch that is skipped and retried, never a
+ permanently broken channel.
+
+## Re-checking against the reference clients
+
+The Android sources to compare against are `dashpay/kotlin-platform`
+(`org.dashj.platform.dashpay`, the live library), `dashpay/dashj` (core crypto
+and keychains) and `dashpay/dash-wallet` (the app: sync, UI, DAOs), all on
+`master`. `android-dashpay`, cited in the source table at the top, is the stale
+predecessor (last push 2024-01); do not diff against it.
+
+| Concern | Reference-client anchor |
+|---|---|
+| `accountReference` ASK28 byte order | `BlockchainIdentity.getAccountReference` = `wrapReversed(ASK).toBigInteger().toInt() ushr 4` (= `u32_le(ASK[0..4])>>4`; we use the iOS `be(ASK[28..32])>>4`, see section (3)) |
+| Friendship path (receive vs send account) | `FriendKeyChain.getContactPath`: `contact.getUserAccount()` (receive) / `getFriendAccountReference()` (send) |
+| `contactRequest` pagination (past 100) | `Documents.getAll` loops `startAt = last.id` while `size >= 100`; `retrieveAll` means `limit(-1)` |
+| High-water + 10-minute skew overlap | `PlatformSyncService.kt` (high-water sync), `DashPayContactRequestDao.kt` (`MAX(timestamp)` per direction) |
+| Batched contact-profile fetch | `updateContactProfiles` calls `Profiles.getList` (chunks of 100, `whereIn $ownerId`) |
+| Non-destructive profile update | `Profiles.replace`: read-modify-write (`profileData.putAll(currentProfile.toObject())`, then overlay) |
+| `encryptedAccountLabel` padding | `padAccountLabel()`: pad to at least 16 chars with spaces, always emit |
+| Recipient-key selection | ENCRYPTION first, with an AUTHENTICATION/HIGH fallback |
+| Sent-tx status | derived live from `TransactionConfidence`, not stored |
+| Transaction to contact (both directions) | `getFriendFromTransaction` scans the sent and received pools |
+| Account/keychain self-heal | `checkDatabaseIntegrity` |
+
+Never assert that `avatarFingerprint` bytes match across clients: the dHash
+pixel pipelines differ, so fingerprints are compared by Hamming distance (see
+`calculate_dhash_fingerprint`).
diff --git a/docs/dashpay/KOTLIN_INVITATIONS_SPEC.md b/docs/dashpay/KOTLIN_INVITATIONS_SPEC.md
deleted file mode 100644
index 58ee9fa7d11..00000000000
--- a/docs/dashpay/KOTLIN_INVITATIONS_SPEC.md
+++ /dev/null
@@ -1,442 +0,0 @@
-# DashPay Invitations (DIP-13 3') — Kotlin/Android Port Spec
-
-Port the shipped iOS invitation feature (create + claim + reclaim + sent-invitations
-persistence, PR #4041, `docs/dashpay/DIP15_INVITATIONS_SPEC.md`) to the Kotlin SDK and
-KotlinExampleApp, on branch `feat/kotlin-dashpay-invitations` (base `v4.1-dev`).
-
-Status: **v3 — synced with owner (2026-07-22), all open questions resolved (§10);
-slices 1–5 implemented** (JVM + cargo gates green; instrumented + funded-testnet
-QA are the remaining environment-bound gates). (v2 was the post-review draft.)
-Reference implementation: `packages/swift-sdk` + SwiftExampleApp (source of truth per the
-parity doctrine in `packages/kotlin-sdk/CLAUDE.md`).
-Feature behavior spec: `docs/dashpay/DIP15_INVITATIONS_SPEC.md` §0/§0A/§0B (as-built truth).
-
----
-
-## 1. Problem
-
-The Kotlin DashPay migration (K1–K3, `KOTLIN_MIGRATION_SPEC.md`) ported all 10 DashPay
-screens and declared invitations out of scope (§8) because iOS hadn't shipped them yet.
-iOS has since shipped the full feature. Today the Kotlin stack is deliberately
-**fail-closed**: `rs-unified-sdk-jni/src/persistence.rs:174` sets
-`on_persist_invitations_fn: None`, so the backend never attests the `INVITATIONS`
-capability and `create_invitation`'s up-front
-`persistence_capabilities().contains(INVITATION_CREATION)` gate refuses to run on
-Android — rather than mint a voucher whose one-time bearer key could be re-exported
-after a restart (the defect class fixed by 55937e15c1 on iOS).
-
-Every layer is pre-seamed for this port:
-- All three invitation FFI entry points (`create`/`claim`/`parse`) plus the two reclaim
- exports already exist in `rs-platform-wallet-ffi` and are consumed by iOS through the
- same C ABI.
-- The JNI natives for resume/top-up already declare a `consumeInvitationVoucher`
- parameter (currently guard-rejected when `true`); the Kotlin *public* wrappers hardcode
- `false` and structurally gate invitation locks out (see §3.1 — they need new variants,
- not a flag flip).
-- Room schema v8 is reserved for invitations (`PARITY.md:49`; DB is at v7).
-- The handler already attests 3 of the 4 capability bits in `INVITATION_CREATION`
- (`ATOMIC_CHANGESETS 0x01 | ASSET_LOCK_FUNDING_INDICES 0x04 | WALLET_RESTORE 0x80`);
- only `INVITATIONS (0x02)` and its callback are missing.
-
-**Goal:** feature parity with iOS invitations — same flows, same persisted shape, same
-crash-safety semantics, same testTags — with **zero changes to `rs-platform-wallet` and
-`rs-platform-wallet-ffi`** (shared logic stays shared; the port is bindings + persistence
-+ UI only).
-
-## 2. What does NOT need porting (all shared Rust, already on this branch)
-
-- Link codec (`dashpay://invite?du=…&assetlocktx=…&pk=…&islock=…`, emit-strict /
- parse-lenient, applink host, WIF + byte-order leniency) — `crypto/invitation.rs`.
-- Create orchestration: amount caps (`MIN=300_000` / `MAX=5_000_000` duffs), capability
- gate, funding-index persist+flush **before broadcast** (abort on failure), invitation
- row persisted **after broadcast, before the proof wait** (hard error if it fails),
- Instant-proof requirement, path-gated voucher-key export (`9'/coin'/5'/3'/idx'`),
- key scrubbing — `network/invitation.rs`.
-- Claim orchestration: claim-by-fetch with bounded retry + byte-reversed retry,
- credit-output selection by script match, Instant/Chain proof reconstruction,
- raw-key identity registration — `network/invitation.rs`.
-- Reclaim authorization: `AssetLockFunding::FromExistingAssetLock` +
- `authorized_invitation_reclaim` (invitation-typed locks are consumable **only** with
- `consume_invitation_voucher = true`, and only into register/top-up; every generic
- path refuses them regardless of the flag) — `wallet/asset_lock/orchestration.rs:493,511`.
-- The persistence wire type `InvitationEntryFFI` (repr(C), ABI-pinned 64 B, all-POD)
- and the `on_persist_invitations_fn` callback slot + `INVITATIONS` capability bit —
- `rs-platform-wallet-ffi/src/invitation_persistence.rs`, `persistence.rs:655-664,822`.
-
-Kotlin re-implements **none** of this. Per doctrine, no JNI function stitches Rust calls
-together; each new export is a thin marshaler over exactly one existing C-ABI entry point.
-
-## 3. Design decisions
-
-1. **Reclaim: forward the flag in JNI + add outpoint-taking reclaim wrappers.**
- Two layers, two different changes:
- - *JNI*: the resume/top-up bindings (`identity.rs:496` → hardcoded `false` at `:552`,
- guard at `:524`; `credits.rs:286` → `:331`, `:311`) forward the already-declared
- `jboolean` verbatim and drop the `generic_asset_lock_recovery_allowed` call — an
- interim fail-closed measure pending this port. Rust core enforces the real policy
- independently (verified: `authorized_invitation_reclaim` requires
- invitation-typed lock ∧ flag ∧ register/top-up target; all other
- `FromExistingAssetLock` constructors hardcode `false` Rust-side), and Swift's
- wrappers forward the identical boolean with no extra guard — this reproduces the
- already-reviewed iOS trust model, not a weaker one.
- - *Kotlin wrappers*: the existing public `resumeWithExistingAssetLock` /
- `resumeTopUpWithExistingAssetLock` **cannot be reused** — they hardcode `false`
- (`IdentityRegistration.kt:158`, `IdentityCredits.kt:115`), `require()` non-invitation
- funding types, and `TrackedAssetLock.FundingType` has no `INVITATION(3)` (unknown
- types are silently dropped at `TrackedAssetLock.kt:62`). Add two **outpoint-taking
- reclaim variants** mirroring Swift (`resumeIdentityWithAssetLock:3939`,
- `resumeTopUpWithAssetLock:4128`): raw txid+vout + `consumeInvitationVoucher`, no
- funding-type gate, freely chosen `identityIndex`; each still one FFI call — no
- doctrine violation. Generic-recovery wrappers and all existing call sites stay
- exactly as they are.
- *Alternative rejected:* adding `FundingType.INVITATION` and relaxing the three
- `require()` gates on the generic-recovery wrappers — that weakens crash-recovery
- invariants shared by non-invitation flows to serve one caller.
-2. **Status transitions are client-written, exactly as on iOS.** Rust emits only
- `Created` rows (create is the sole `InvitationChangeSet` emitter today). `Claimed`
- and `Reclaimed` are written locally by the app via the reclaim-outcome classifier.
- Room is the UI's source of truth; there is **no Rust→Kotlin rehydrate** (a Room wipe
- loses list visibility only — never funds; `funding_index` re-derives the key).
-3. **Claim entry = paste + QR scan first-class, deep link additionally** (decided,
- §10.2). Add a `dashpay://invite` `VIEW` intent-filter routing to the claim sheet,
- mirroring `SwiftExampleAppApp.swift:148`, **plus an intent-filter for the legacy
- AppsFlyer host `https://invitations.dashpay.io/applink`** (the form production
- dashwallet-iOS emits; our parser already accepts it) — unverified until the domain
- serves `assetlinks.json` (dashpay/platform#4096), so it participates in the app
- chooser rather than auto-opening; upgradeable to a verified App Link when #4096
- lands. Two further deliberate deviations from iOS:
- - **Walletless parking (iOS behavior is a bug, not parity):** iOS clears
- `pendingInviteURL` before its no-wallet guard returns
- (`DashPayTabView.swift:149-153`), silently discarding the link — on Android the
- intent-filter's headline scenario *is* a fresh install tapping an invite. Kotlin
- parks the pending URI until a wallet exists (cleared only when the claim sheet is
- actually seeded) and shows "create a wallet to claim this invitation". Flag the
- drop as an upstream iOS bug to fix separately.
- - **Honest Android framing:** a custom scheme can't use App Links auto-verify; any
- app may register the same filter. Android shows a chooser on collision (per-tap
- visibility iOS doesn't have), **but** one "Always" tap for a malicious app silently
- routes every future invite link to it. Documented as an Android-specific
- persistence risk, not "the same caveat iOS documents".
- The mid-claim deferral gate (analog of `invitationClaimInFlight`) is ported as-is.
-4. **Persistence writes go through the round buffer.** On FFI hosts the durable boundary
- is `onChangesetEnd`'s single `withTransaction` replay of the per-wallet
- `ChangesetBuffer` (`PlatformWalletPersistenceHandler.kt:298-353`); `onFlush` stays the
- inherited base-class no-op (verified: Rust's `flush()` after a successful `store()`
- round is a post-commit bookkeeping notification — the round's commit result is
- honored, not advisory). Hard rule: the invitation handler **stages via `stage {}`
- into the round buffer; never writes in its own transaction and never defers past the
- round** (no `launch`/`post`) — an immediate standalone write would break round
- atomicity, a deferred one would break durability.
-5. **One PR, sliced commits** (§5 order). The feature is cohesive; the slices carry
- independent compile/test gates. (If review load demands, slice 4 can split
- screens+nav vs deep-link+classifier — optional.)
-
-## 4. Interfaces per layer
-
-### 4.1 JNI (`rs-unified-sdk-jni`) — new exports in `src/dashpay.rs`
-
-All under `support::guard`, errors via `take_pwffi_error` → `DashSDKException(code+1000)`.
-FFI structs are read via the rlib types directly (no manual offset math).
-**Secret-hygiene rule (normative): the `uri` argument is the bearer secret — it must
-never be interpolated into any exception message, log line, or debug output from
-`createInvitation`, `claimInvitation`, or `parseInvitation`** (the existing
-`"$field must be N bytes"` convention covers byte params only; no precedent exists for
-a String param that *is* the secret, so this is easy to get wrong).
-
-| Export (`Java_…_DashpayNative_…`) | Wraps | Notes |
-|---|---|---|
-| `parseInvitation(uri: String) -> String` | `platform_wallet_parse_invitation` | Returns the preview per the crate's compact-JSON convention (`structurallyValid`, `isInstant`, `hasInviter`, `inviterUsername?`); frees `inviter_username` after copy. Malformed link ⇒ `structurallyValid=false`, not an exception. |
-| `createInvitation(walletHandle: Long, amountDuffs: Long, fundingAccountIndex: Int, inviterIdentityId: ByteArray?, inviterUsername: String?, nowUnix: Int, coreSignerHandle: Long) -> String` | `platform_wallet_create_invitation` | Returns the URI (bearer secret). The out-outpoint (caller-owned POD, nothing to free) is ignored — the persistence callback records the row, as on iOS. `now_unix` from a real clock read (Rust rejects 0). |
-| `claimInvitation(walletHandle: Long, uri: String, identityIndex: Int, pubkeyRowsBlob: ByteArray, signerHandle: Long, nowUnix: Int)` | `platform_wallet_claim_invitation` | Pubkeys via the existing `decode_registration_pubkeys_blob` (`pubkey_rows.rs:320`); returns id + handle via the **`resumeIdentityWithExistingAssetLock` convention** — `IdentityRegistrationNativeResult` (`[BJ)V`) + `ManagedIdentityHandleGuard` (`identity.rs:569-576, :57-76`). (Not `registerIdentityWithFunding`, which destroys the handle and returns only the id.) |
-
-Plus the reclaim-flag forwarding change in `identity.rs` / `credits.rs` (§3.1).
-
-### 4.2 JNI persistence bridge (`src/persistence.rs`)
-
-- New trampoline `tramp_persist_invitations`, modeled on `tramp_persist_asset_locks`
- (`:1233`): loop the `InvitationEntryFFI` upsert slice and the `[u8;36]` removal slice,
- calling **per-row** bridge methods (the bridge has no array-of-struct convention):
- `onPersistInvitationUpsert(walletId, outPoint: ByteArray(36), fundingIndex: Int,
- amountDuffs: Long, expiryUnix: Int, createdAtSecs: Int, hasInviter: Boolean,
- status: Int)` and `onPersistInvitationRemoval(walletId, outPoint: ByteArray(36))`.
- Any nonzero Kotlin return fails the round so `create_invitation` surfaces
- funded-but-unrecorded instead of silently losing the row.
-- Set `on_persist_invitations_fn: Some(tramp_persist_invitations)` (replacing the `None`
- at `:174` and its fail-closed comment).
-- ⚠ Lockstep rule: the Rust `call_method` descriptors and the Kotlin bridge signatures
- must land in the same commit — a mismatch is a runtime failure, not a compile error.
- **Descriptor coverage is tested, not eyeballed (adversarial M4):** an instrumented
- test resolves every `NativePersistenceBridge` (name, signature) pair via a test-only
- JNI export (`GetMethodID` over the loaded class — covering all slots, not just the
- new ones), so a descriptor typo fails CI instead of failing the first funded create.
-- **Address-pool silent-skip fix (adversarial M1, security-critical):**
- `onPersistAccountAddressPoolEntry` currently early-returns when the parent account row
- is missing (`fetchAccount(...) ?: return@stage`,
- `PlatformWalletPersistenceHandler.kt:509`). For the invitation flow that skip is a
- **lie to the pre-broadcast durability gate**: Rust treats the round's success as
- "funding index durably recorded" and broadcasts; on restart `next_unused` resets and
- the already-exported bearer key is re-exported — the 55937e15c1 bug class with no
- crash needed. Fix in slice 1: for asset-lock funding account types, a missing parent
- account row **fails the round** (nonzero) or upserts the row — never silently skips.
- Add the upgrade-path test: a v7-era wallet with no invitation-account Room row →
- first `createInvitation` must abort **before broadcast**, not succeed non-durably.
- (Audit whether Swift's handler shares the skip; if so, flag upstream.)
-
-### 4.3 Kotlin SDK
-
-- `ffi/DashpayNative.kt`: three new `external fun`s matching §4.1.
-- `ffi/NativePersistenceBridge.kt`: the two per-row slots from §4.2
- (`open fun … : Int = 0`).
-- `persistence/PlatformWalletPersistenceHandler.kt`:
- - implement both slots — stage upsert/delete of `InvitationEntity` rows via `stage {}`
- keyed by `outPointHex` (the upsert-key ↔ removal-key seam, pinned by test as on
- iOS). **Upserts are partial**: only the FFI-fed columns are written on conflict —
- client-written columns (`statusRaw`, `reclaimInFlight`) are preserved, so a future
- Rust re-emit of an existing outpoint can't reset local status;
- - add `CAPABILITY_INVITATIONS: Long = 0x02` and OR it into
- `persistenceCapabilitiesBits()` **in the same commit that wires the full path**
- (bit + path are inseparable; attesting early re-opens the fail-closed hole).
-- New wrapper surface (idiom of `IdentityRegistration.kt` / `IdentityCredits.kt`, suspend
- on `Dispatchers.IO`, KDoc citing the Swift source):
- - `createInvitation(amountDuffs, fundingAccountIndex, inviterIdentityId: ByteArray?, inviterUsername: String?): String` (Swift `ManagedPlatformWallet.createInvitation:2062`)
- - `claimInvitation(uri, identityIndex, identityPubkeys, signer): ByteArray /* identityId */` (Swift `:2145`) — adopts then releases the managed handle per the resume idiom (the claimed identity is already folded into the identity manager + persisted Rust-side); key set = the existing `RegistrationKeys` **6-key** layout (4 base + DashPay enc/dec at keyIds 4–5), pre-persisted via the same path registration uses.
- - `parseInvitation(uri): InvitationPreview` (Swift `:2211`)
- - **New reclaim variants (§3.1):** `reclaimInvitationAsNewIdentity(outPointTxid: ByteArray(32), outPointVout: Int, identityIndex, identityPubkeys /* 4-key set — iOS reclaim-register uses authKeyCount=4, no DashPay pair */, signer)` and `reclaimInvitationAsTopUp(identityId: ByteArray(32), outPointTxid, outPointVout): ULong /* new balance */` — both passing `consumeInvitationVoucher = true`; the only call sites that ever do.
-
-### 4.4 Room (`persistence/`) — DB v7 → v8
-
-`InvitationEntity` — field-exact port of `PersistentInvitation.swift` (and of
-`InvitationEntryFFI` for the callback-fed fields):
-
-| Column | Type | Notes |
-|---|---|---|
-| `outPointHex` | String, `@Unique` | `:` via the same encode as the asset-lock entity — the upsert/removal join key |
-| `rawOutPoint` | ByteArray(36) | raw `txid_le ‖ vout_le`; reclaim rebuilds the outpoint without re-parsing hex |
-| `walletId` | ByteArray, indexed | |
-| `fundingIndexRaw` | Int | display metadata; the key is re-derived Rust-side, **no secret column** |
-| `amountDuffs` | Long | |
-| `expiryUnix` | Int | inviter-side display only (not on the wire) |
-| `createdAtSecs` | Int | |
-| `hasInviter` | Boolean | |
-| `statusRaw` | Int | **0=Created, 1=Claimed, 2=Reclaimed** — pinned Rust-side by `status_to_u8`; discriminants must match byte-for-byte; client-written after create |
-| `reclaimInFlight` | Boolean, default false | crash-forensics marker, §4.6 — never a concurrency guard |
-| `createdAt` / `updatedAt` | Long | |
-
-Additive migration `MIGRATION_7_8` + exported schema `8.json` + migration test against
-`7.json` (instrumented tier, following `DashDatabaseMigrationTest`). DAO exposes a
-`Flow` sorted by `createdAtSecs` desc for the list screen.
-
-### 4.5 KotlinExampleApp UI (Compose, `app/…/ui/dashpay/`)
-
-One screen/sheet per Swift file; testTags = iOS accessibility identifiers verbatim;
-screens driven by Room Flows + snapshot data — never retained native handles.
-**Coroutine-scope rule (all three network flows):** create/claim/reclaim run in an
-app-/container-scope (not `rememberCoroutineScope`, which cancels on leaving
-composition), with `withContext(NonCancellable)` around the
-marker-write → consume → status-write sequence — a mid-flow dismissal must not strand
-a half-done reclaim (mirrors the `performDashPaySend` double-send guard precedent).
-
-| Swift | Compose target | Notes |
-|---|---|---|
-| `InvitationsView.swift` | `InvitationsScreen` | Room Flow over all invitations filtered to loaded wallets (multi-wallet aware — each row reclaims via its own `walletId`); rows show amount, short outpoint, contact-request badge, expiry, status badge (Created/Claimed/Reclaimed); create entry hidden only when no wallet (an active identity is NOT required). Tags `dashpay.invitations.{list,create,reclaim}`, entry `dashpay.openSentInvitations` on `DashPayTabScreen`. |
-| `CreateInvitationSheet.swift` | `CreateInvitationSheet` | amount field default **0.03 DASH**, UI range [0.003, 0.05] mirrored for display (Rust enforces); "Send a contact request back to me" toggle (default on, disabled without a username — inviter id+username passed only when opted in); result = QR (ZXing, in-memory bitmap only) + **share as text** (no image export → no `FileProvider` temp file holding the secret) + copy per the clipboard rules in §6; UI single-flight (submit disabled while creating). Tags `dashpay.invite.create.{amount,sendBack,submit,share,copy,done}`. |
-| `ClaimInvitationSheet.swift` | `ClaimInvitationSheet` | URI via paste, the existing `QrScanner` route (`savedStateHandle` result), or a parked deep link; `parseInvitation` preview gated only on `structurallyValid` (amount shows "—"); claim wallet selection pins the iOS rule: the active identity's wallet, else the first loaded wallet, entry disabled when none (`DashPayTabView.swift:131-134`); identity index = next unused (reuse the existing registration index logic); claim pre-persists the 6-key `RegistrationKeys` set then calls `claimInvitation`; on success, if `hasInviter && inviterUsername != null` → "Add \?" prompt → DPNS resolve → `sendContactRequest` (both already ported); works with **no active identity** (fresh invitee); back/dismiss gated while claiming. Tags `dashpay.invite.claim.{uriField,submit}`. |
-| `ReclaimInvitationSheet.swift` | `ReclaimInvitationSheet` | reachable only from `statusRaw == 0` rows; segmented target Top-up existing (identity picker) / Register new (**4-key set**); calls the new reclaim wrappers (§4.3); **in-memory `isReclaiming` single-flight gating submit AND dismissal** (Swift `:37,92-109,168-181`) — the persisted marker is crash forensics, never the concurrency guard (an unguarded recomposition off the Room Flow re-emit could double-consume and let the loser's classifier overwrite Reclaimed with Claimed); marker + classifier per §4.6. Tags `dashpay.invite.reclaim.{target,identityPicker,submit}`. |
-
-Navigation: new routes in `Routes.kt` + `AppNavHost.kt`; entry points on
-`DashPayTabScreen` ("Sent invitations" + "Claim invitation", ids as on iOS). Deep link
-per §3.3: `dashpay`/`invite` `VIEW` intent-filter on `MainActivity` → pending-invite
-state → parked until a wallet exists → claim sheet, with the claim-in-flight deferral.
-
-### 4.6 Reclaim crash-safety: marker + classifier (verbatim port)
-
-- **Marker discipline:** capture `hadPriorReclaimInFlight`; persist
- `reclaimInFlight = true` (the Room write **must succeed** — abort the reclaim if it
- doesn't) only **immediately before** the on-chain consume; the register arm pre-persists
- its identity keys **before** setting the marker. On observed success: `statusRaw = 2`,
- clear the marker, save.
-- **Classifier:** a pure `internal fun classifyReclaimFailure(error, hadPriorReclaimInFlight)`
- in the app layer, arms identical to Swift (`ReclaimInvitationSheet.swift:406`):
- 1. typed `DashSdkError.PlatformWallet.AssetLockAlreadyConsumed` (the mapped class for
- FFI code 24, `DashSdkError.kt:246` — the local tombstone written only after our own
- successful consume; match the type, not a numeric code) → **Reclaimed**
- (`statusRaw = 2`);
- 2. message contains `"already completely used"` (consensus 10504, exact canonical
- phrase, lowercased-contains — the typed FFI code for this remains the known
- follow-up) → **Claimed** if no prior marker (provably a foreign claim, neutral
- "already claimed" copy, claimant never named); **ambiguous** if the marker was set
- (resolves to the conservative terminal `Claimed` + ambiguity message, never an
- inferred `Reclaimed`);
- 3. message contains `"is not tracked"` with the marker set → explicit ambiguity error,
- state unchanged;
- 4. else generic error; clear a stale marker only when `!hadPrior && isNotTracked`.
-
-## 5. Work plan (commit slices)
-
-1. **Persistence spine (the unblocker):** `InvitationEntity` + DAO + `MIGRATION_7_8` +
- `8.json`; per-row bridge slots + handler impl (partial upsert); JNI trampoline; flip
- `on_persist_invitations_fn` to `Some`; attest `CAPABILITY_INVITATIONS`; **the
- address-pool silent-skip fix + upgrade-path test (§4.2)** — all one commit
- (capability bit and path are inseparable). Gate: migration test (instrumented) +
- handler mapping tests + the descriptor-resolution instrumented test +
- `cargo check -p rs-unified-sdk-jni`.
-2. **Parse + create + claim bindings:** three JNI exports + `external fun`s + SDK
- wrappers + `InvitationPreview` type. Gate: `./build_android.sh --verify` +
- symbol-load smoke.
-3. **Reclaim:** JNI flag forwarding (guard dropped in exactly the two bindings) + the
- two new outpoint-taking Kotlin reclaim wrappers. Gate: compile + tests pinning that
- every generic-recovery path still passes `false` and the generic wrappers still
- reject invitation locks.
-4. **App UI:** four screens + routes + DashPay-tab entry points + deep-link filter with
- walletless parking + classifier + marker discipline + single-flight/scope rules.
- Gate: `:app:assembleDebug` + classifier/UI tests.
-5. **QA + parity bookkeeping:** PARITY.md rows, emulator QA runs (§7), doc updates.
-
-## 6. Security invariants preserved (review checklist)
-
-- **Voucher-key-reuse defense (55937e15c1):** honest capability attestation + durable
- round commits + **no silent skips anywhere in the funding-index persist chain**
- (§4.2 M1 fix). `CAPABILITY_INVITATIONS` is attested only in the commit that wires the
- full path; the handler stages into the round buffer (§3.4); a persist path that cannot
- complete must fail the round, never no-op.
-- **Durability residual (documented, accepted):** Room runs WAL +
- `synchronous=NORMAL` — safe across process death (the relevant Android failure mode),
- but a hard **power loss** can roll back the most recent committed transaction. This is
- pre-existing for every already-attested capability bit and the same class of residual
- iOS carries; documented rather than silently assumed. (Optional hardening —
- `synchronous=FULL` for the wallet DB — is Open Question 4.)
-- **Bearer-secret hygiene:** the URI embeds the voucher WIF. Never logged (no
- logcat/android_logger, no exception messages carrying the URI — §4.1 rule — no
- crash-report breadcrumbs), never persisted. Clipboard: Android has **no**
- local-only or auto-expiring clipboard primitive (unlike iOS's
- `localOnly + expirationDate:+60s`), and `ClipDescription.EXTRA_IS_SENSITIVE` is
- API 33+ with minSdk 29 — so: set the flag when `SDK_INT >= 33`, and actively
- compare-and-clear the clipboard after ~60 s (matching the iOS window); the missing
- device-scoping half is an accepted, documented platform gap. Share = text-only
- (no secret-bearing temp image files). QR rendered from an in-memory bitmap
- (`util/QrCode.kt` precedent, no file/cache writes).
-- **Backup:** the example app's manifest already sets `android:allowBackup="false"`
- (Room DB has no secret column regardless). Note for integrators: this is an app-level
- property the SDK cannot enforce — production apps must set it themselves.
-- **Path gate untouched:** zero changes to `rs-platform-wallet` / `rs-platform-wallet-ffi`
- / `rs-sdk-ffi`, so the `9'/coin'/5'/3'/idx'` export gate and the amount caps stay
- exactly as reviewed on iOS.
-- **`consume_invitation_voucher` discipline:** `true` appears at exactly two call sites
- (the two reclaim wrappers, reached only from the reclaim sheet); every generic
- resume/top-up path stays `false` and Rust refuses invitation locks there regardless.
-- **Single-flight everywhere funds move:** create sheet (submit disabled while
- creating; Rust's per-wallet build-persist mutex is the backstop) **and** reclaim
- sheet (`isReclaiming` gating submit + dismissal — §4.5); claim sheet gates
- back/dismiss while claiming.
-
-## 7. Test / verification plan
-
-- **JVM unit:** classifier matrix (port `ReclaimInvitationClassifierTests` — all arms,
- incl. marker/no-marker ambiguity split and stale-marker clearing);
- `InvitationEntity` upsert/removal key-seam test (outPointHex form drift, mirror of
- `InvitationPersistenceTests`); handler mapping incl. removal → DAO delete and the
- **partial-upsert preserves `statusRaw`/`reclaimInFlight`** pin; status discriminant
- pin (0/1/2); generic wrappers still reject invitation locks.
-- **Rust:** `cargo check -p rs-unified-sdk-jni` (+ clippy).
-- **Instrumented (CI emulator):** Room `MIGRATION_7_8` against `7.json`;
- `FfiSmokeTest`-style symbol load for the new externs; the **descriptor-resolution
- test over every bridge slot** (§4.2); a capability-bits assertion that the handler
- satisfies `INVITATION_CREATION`; the **upgrade-path test** (missing invitation-account
- row → create aborts before broadcast).
-- **Testnet-gated (`-Ptestnet=true`) / emulator QA** (`emulator-control` skill; faucet is
- rate-limited → self-fund): mirror the iOS rows DP-12..DP-19 —
- create (funded, row lands in Room + list), claim (second wallet, no funds → funded
- identity; optional contact bootstrap), malformed/reused/wrong-network rejection (fail
- loud, no side effects), sent-list persistence + upsert-in-place, reclaim-as-top-up
- (balance rises, row → Reclaimed), reclaim-as-register (4-key set), already-consumed
- race (second consume → deterministic "already completely used" → neutral Claimed
- copy). The funded two-wallet race remains manual, as on iOS.
-- **Acceptance gate:** the funded create→claim e2e on the emulator against testnet.
-
-### Funded e2e evidence (2026-07-23, arm64 emulator + testnet)
-
-- Instrumented tier: 32/32 green (Room `MIGRATION_7_8` + full v1→v8 chain,
- `persistenceBridgeDescriptorsAllResolve`, FFI smoke; 3 testnet-gated skips).
-- Create (DP-12/16): 0.03 DASH voucher funded + InstantSend-locked, legacy link
- emitted, row landed in Room + Sent list as `Created` (outpoint `f9ef7f5b…:0`).
-- Claim (DP-13): link pasted → valid preview → new identity `292ebab4…`
- registered with 2 818 643 580 credits, funded solely by the voucher.
-- Already-consumed (DP-19 classifier, live): reclaiming the claimed voucher hit
- consensus 10504 → neutral "This invitation was already claimed.", row →
- `Claimed`, reclaim affordance gone.
-- Interrupted create: a second create timed out waiting for its InstantSend
- lock — the funded row (`732bf38c…:0`) was already persisted and reclaimable,
- proving the persist-before-proof-wait ordering on Android.
-- Reclaim-as-top-up (DP-17): that voucher consumed into identity `7023bed1…`,
- balance 49 818 637 700 → 52 736 112 114 credits, row → `Reclaimed`.
-- Malformed link (DP-15): garbage URI → "Invalid invitation link.", claim
- disabled, no side effects.
-- Not exercised here: the two-wallet contact-bootstrap (DP-14 — inviter had no
- DPNS username, so the opt-in toggle was correctly disabled) and
- reclaim-as-register (DP-18 — same FFI + wrapper as the seam-tested claim
- path); both remain manual QA, matching iOS.
-
-## 8. Failure modes
-
-- **Capability over-attestation / silent persist skips** → voucher-key reuse (the
- pre-port bug class; adversarial M1 shows a no-crash variant via the address-pool
- skip). Guard: bit + path in one commit; the M1 fail-the-round fix; instrumented
- capability + upgrade-path assertions.
-- **Handler write outside the round buffer** → broken round atomicity or lost-on-crash
- rows reported as persisted. Guard: §3.4 rule + mapping tests exercise `stage {}`.
-- **JNI descriptor mismatch** → every invitation persist fails at runtime — after real
- funds broadcast, if untested. Guard: lockstep rule + the descriptor-resolution
- instrumented test **before** any funded QA.
-- **Crash between broadcast and the invitation-row round** → a funded voucher with no
- Room row: invisible in Sent invitations, unreclaimable from the UI (reclaim is
- row-driven), funds stranded pending manual recovery. Inherent to the shared
- persist-after-broadcast ordering (iOS carries the same window); the
- funded-but-unrecorded error copy must surface the **outpoint** so support/manual
- recovery is possible.
-- **Process death mid-reclaim** → marker semantics resolve it (our-tombstone →
- Reclaimed; foreign claim → Claimed; genuinely ambiguous → conservative Claimed +
- message). Double-tap/recomposition races are excluded by the in-memory single-flight
- (§4.5), not by the marker.
-- **URI leakage via logs/exceptions/clipboard/share files** → bearer theft. Guard:
- §4.1 exception rule + §6 clipboard/share rules.
-- **ChainLock fallback at create** → Rust rejects the link, lock stays reclaimable;
- surface the error copy as iOS does.
-- **Deep link with no wallet** → parked, not dropped (§3.3); "create a wallet to claim"
- copy.
-- **QR/paste garbage** → `structurallyValid=false` preview, claim button disabled, no
- side effects.
-
-## 9. Out of scope
-
-- Typed FFI code for consensus 10504 already-consumed (shared iOS/Android follow-up).
-- A typed funded-failure result carrying the recovery outpoint from
- `create_invitation` (adversarial-review follow-up): on the rare
- funded-but-row-persist-failed path the outpoint doesn't cross the JNI boundary
- (the FFI fills out-params only on success — changing that is a shared-Rust
- change). Recovery exists meanwhile via the Rust-side tracked-lock list
- (diagnostics surface); the common interrupted-create case persists the row
- and is reclaimable from the UI (funded-e2e verified).
-- Driving the Rust pre-broadcast abort from a JVM test (the broadcast boundary
- is Rust-side; the Kotlin tier pins the round-failure half —
- `invitationPoolEntryWithoutWalletFailsTheRound` — and Rust's own
- `create_invitation_requires_durable_persistence` pins the abort).
-- Any `rs-platform-wallet` / `rs-platform-wallet-ffi` change (incl. Rust-emitted
- Claimed/Reclaimed status changesets — latent on iOS too).
-- Fixing the iOS walletless deep-link drop (flagged upstream, §3.3).
-- AppsFlyer/OneLink or App-Links-style verified deep-link transport (tracked separately
- for iOS as well).
-- Contested-name claim tier (deferred on iOS).
-
-## 10. Decisions (RESOLVED — owner sync, 2026-07-22)
-
-1. **Reclaim JNI guard drop: approved.** `generic_asset_lock_recovery_allowed` is
- removed from exactly the two forwarding bindings; Rust core's
- `authorized_invitation_reclaim` remains the (independently unit-tested) enforcement —
- the same single-gate trust model shipped on iOS.
-2. **Deep link: both filters.** `dashpay://invite` custom scheme + the legacy AppsFlyer
- `https://invitations.dashpay.io/applink` host (unverified chooser participation until
- #4096 serves `assetlinks.json`), with walletless parking (§3.3).
-3. **PR strategy: one PR, five sliced commits** (§5).
-4. **WAL durability: accept + document** the `synchronous=NORMAL` power-loss residual
- (§6) — platform-wide status quo, same class as iOS; `synchronous=FULL` would tax
- every wallet write and belongs to a separate measured change if ever.
diff --git a/docs/dashpay/KOTLIN_MIGRATION_FOLLOWUPS_SPEC.md b/docs/dashpay/KOTLIN_MIGRATION_FOLLOWUPS_SPEC.md
deleted file mode 100644
index db993c19c1e..00000000000
--- a/docs/dashpay/KOTLIN_MIGRATION_FOLLOWUPS_SPEC.md
+++ /dev/null
@@ -1,219 +0,0 @@
-# Kotlin DashPay Migration — Follow-up Fixes Spec
-
-**Historical status:** Items A, B, and C are resolved in the consolidated
-migration; the completed work is recorded in
-[`KOTLIN_MIGRATION_LEFTOVERS.md`](KOTLIN_MIGRATION_LEFTOVERS.md). Item D remains
-deferred. The problem/approach sections below preserve the reviewed rationale
-and pre-implementation baseline rather than describing current open work.
-
-Scope: three DashPay follow-ups folded into the consolidated migration PR
-(`feat/kotlin-sdk-dashpay-migration`, stacked on the base Kotlin SDK PR
-`feat/kotlin-sdk-and-example-app`). A fourth item (durable contact-crypto
-queue persistence) is **deferred** with rationale in §D.
-
-All three items were verified **still open at the base PR HEAD** (`1fd86fbae5`)
-and **not** addressed by the parent PR — none duplicates parent work.
-
----
-
-## A. Platform-address signing: eliminate the JVM-String seed exposure
-
-### Problem
-`KeystoreSigner.signPlatformAddressOnDemand` retrieves the wallet mnemonic as a
-`java.lang.String` (`WalletStorage.retrieveMnemonic`) and passes it across JNI to
-`SignerNative.signWithMnemonicAndPath(mnemonic: String, …)`. An immutable `String`
-cannot be scrubbed; the phrase sits on the JVM heap until GC (if ever),
-recoverable from a heap dump. This is the exact anti-pattern the K2 resolver path
-eliminated with `resolveMnemonicInto`, and it is explicitly flagged as the tracked
-follow-up in three places (`KeystoreSigner.kt:151-157`, `SignerNative.kt:36-38`,
-`signer.rs:436-437`). Every on-demand platform-address signature re-exposes the seed.
-
-### Approach (out-buffer discipline, input direction)
-Pass the phrase as a **scrubbable `ByteArray`** the caller owns and zeroes, so no
-`String` of the seed ever exists on the signing path.
-
-- **Rust JNI** (`packages/rs-unified-sdk-jni/src/signer.rs`): replace
- `Java_…_SignerNative_signWithMnemonicAndPath` with
- `Java_…_SignerNative_signWithMnemonicAndPathInto`; the mnemonic argument becomes
- `JByteArray`. Decode the non-secret args (path, payload) **first** and the
- mnemonic **last**. **Do NOT use `convert_byte_array` + `reserve_exact`** — that
- pattern orphans an unscrubbed plaintext heap copy: `convert_byte_array` returns a
- `cap==len==N` `Vec`, and `CString::new`'s NUL append then forces a growth realloc
- that frees the original buffer unscrubbed (real on Android's Scudo allocator).
- Instead keep the phrase in a `Zeroizing>` **end-to-end** — never launder
- it through a `CString` (whose `into_boxed_slice` shrink-to-fit can silently realloc
- and free the plaintext unscrubbed, and which sits outside any zeroize guard across
- the FFI call):
- ```rust
- let n = env.get_array_length(&mnemonic)? as usize; // reject Err
- let mut plain: Zeroizing> = Zeroizing::new(Vec::with_capacity(n + 1));
- plain.resize(n, 0); // len=n, cap=n+1
- let dst = unsafe { slice::from_raw_parts_mut(plain.as_mut_ptr().cast::(), n) };
- env.get_byte_array_region(&mnemonic, 0, dst)?; // reject Err
- if plain.contains(&0) { throw; return } // reject interior NUL
- plain.push(0); // NUL-terminate in place, no realloc
- // pass plain.as_ptr().cast::() to the FFI; drop(plain) scrubs the sole
- // copy after signing — and on any panic unwind, since it never leaves Zeroizing.
- ```
-- **Kotlin FFI decl** (`ffi/SignerNative.kt`): replace the `external fun` with
- `signWithMnemonicAndPathInto(mnemonicUtf8: ByteArray, derivationPath: String,
- network: Int, data: ByteArray): ByteArray?`.
-- **Caller** (`security/KeystoreSigner.kt`): switch
- `storage.retrieveMnemonic(...)` → `storage.retrieveMnemonicUtf8(...)` (returns
- `ByteArray?`, already exists) and wrap the sign in
- `try { … } finally { mnemonicUtf8.fill(0) }`. Delete the KNOWN-residual comment
- block (151-157) — it is now resolved.
-
-The old String function has a **single caller** (verified), so it is removed
-outright; no String seed path remains. The derived key still never crosses JNI —
-Rust derives, signs, and scrubs internally, unchanged.
-
-### Failure modes
-- Interior NUL in the phrase bytes → reject before the FFI call; dropping the
- `Zeroizing>` scrubs the phrase copy.
-- Empty byte array → a NUL-only buffer reaches the inner FFI, whose mnemonic parse
- fails with an invalid-mnemonic error. There is **no size cap** (a large array just
- allocates and then fails the parse); the caller's `retrieveMnemonicUtf8` only ever
- returns real phrase bytes or null, so this is a defense-in-depth path.
-- Caller forgetting to scrub → mitigated by the `finally` block; the array is
- caller-owned per `retrieveMnemonicUtf8`'s contract. The Kotlin `null`-check must
- precede the `try` (nothing fallible between the retrieve and entering the `try`).
-- Note: the JNI symbol name (`Java_…_signWithMnemonicAndPathInto`) must match the
- Kotlin `external fun` exactly — a mismatch is a runtime `UnsatisfiedLinkError`, not
- a compile error, and the JVM helper test won't catch it (pinned by `cargo check` +
- an instrumented/on-device sign smoke).
-
-### Test plan (red→green)
-`KeystoreSigner` cannot be constructed on the JVM tier (its constructor eagerly
-calls native `createSigner`), and `WalletStorage` is a final class with no mock
-framework in the module — so a seam *on the class* is untestable. Instead extract
-a **pure `internal` scrub-and-sign helper** that owns the discipline:
-
-```kotlin
-internal inline fun signWithScrubbedMnemonic(
- mnemonicUtf8: ByteArray,
- derivationPath: String,
- network: Int,
- data: ByteArray,
- sign: (ByteArray, String, Int, ByteArray) -> ByteArray?,
-): ByteArray? = try { sign(mnemonicUtf8, derivationPath, network, data) }
- finally { mnemonicUtf8.fill(0) }
-```
-
-`KeystoreSigner.signPlatformAddressOnDemand` calls it with
-`sign = SignerNative::signWithMnemonicAndPathInto`. JVM test
-(`SignerMnemonicScrubTest`): call the helper with a fake `sign` that records a
-**copy** of the bytes it received and returns a dummy signature, then assert the
-input array is all-zero afterward, and that the dummy signature is returned
-(scrub happens after the sign, not before). Red→green is demonstrated by toggling
-only the `finally { fill(0) }` — without it the array retains the phrase (red),
-with it the array is zeroed (green). This pins the scrub invariant honestly (the
-String-path removal itself is a structural guarantee, verified by the code + the
-absence of any `retrieveMnemonic`/String-sign call on the signing path).
-Plus `cargo check -p rs-unified-sdk-jni` for the Rust side.
-
----
-
-## B. Payment-sheet dispose-mid-send: pin the double-send guard
-
-### Problem
-`SendDashPayPaymentSheet.send()` broadcasts a payment and then records the txid +
-triggers the durability refresh (`onSent` → `refreshDashPayPayments`). The K3 fix
-wraps the broadcast + bookkeeping in `withContext(NonCancellable)` (plus a Compose
-dismissal gate in `ContactDetailScreen`) so a dispose-mid-send cannot skip the
-durability refresh — which would otherwise invite a **double-send** on retry (the
-JNI broadcast is uncancellable; the coin leaves the wallet regardless). This guard
-has **no regression coverage**; a future refactor could drop `NonCancellable`
-silently.
-
-### Approach (extract the send body + inject the sender)
-`send()` is a local function inside a `@Composable`, uncallable from `runTest`.
-**Extract the coroutine body** (`SendDashPayPaymentSheet.kt:90-120`) into a
-non-composable `internal suspend fun performDashPaySend(...)` that takes a minimal
-`fun interface PaymentSender { suspend fun send(...): ByteArray? }` plus the
-`onSent`/`onClose`/status callbacks. The composable calls it inside its existing
-`scope.launch { … }`, so runtime behavior (including the launching Job) is
-identical; the production `PaymentSender` closes over `wallet`/`manager` and calls
-`w.dashpay.sendPayment(...)`. `performDashPaySend` keeps the `withContext(
-NonCancellable) { sender.send(...); onSent() }` wrapper — the guard under test.
-
-**Critical extraction constraints** (else the test is invalid):
-- `performDashPaySend` is a plain `suspend fun` inheriting the caller's Job — it
- must NOT introduce a new `CoroutineScope`/`coroutineScope { }`, which would
- change the cancellation semantics being tested.
-- The `CancellationException` rethrow stays; the best-effort tail
- (`kickDashPaySync`/`delay`/`onClose`) stays OUTSIDE the `NonCancellable` block.
-
-### Test plan (red→green)
-JVM `runTest` (`PerformDashPaySendDoubleSendGuardTest`): launch
-`performDashPaySend` in a child `Job` with a fake `PaymentSender` that records the
-broadcast **before** suspending on a test gate (models "broadcast completed,
-bookkeeping pending" — the real hazard, not "cancelled before broadcast"):
-1. launch the send; await the recorded broadcast entry;
-2. cancel the child Job (simulate dispose-mid-send);
-3. release the gate; join.
-Assert **`onSent` fired exactly once** (count 0 pre-fix vs 1 post-fix — the
-decisive assertion; the `NonCancellable` block completed despite cancellation).
-Against the pre-fix code (no `NonCancellable`), cancellation observed on resume
-after the broadcast skips `onSent` → count 0 → red. Assert on `onSent` count, NOT
-`sendPayment` count (which is 1 either way).
-
-The Compose **dismissal gate** (secondary, defense-in-depth) is left to the
-existing instrumented UI tier; it is not the double-send regression and is not
-deterministically JVM-testable without Robolectric (not configured).
-
----
-
-## C. DataContractRef: add the NativeCleaner GC backstop
-
-### Problem
-`DataContractRef` (`queries/PlatformQueries.kt:621-633`) destroys its native handle
-inline in `close()` but registers **no** `NativeCleaner`, so a leaked (never-closed,
-never-`use{}`) ref leaks the native contract handle forever. Every other owned
-handle — `Sdk`, `ManagedPlatformWallet`, and the K3 `ContactRequestRef`/
-`EstablishedContactRef` — has the GC backstop. Consistency gap.
-
-### Approach
-Match the established idiom exactly: `import NativeCleaner`; register
-`private val cleanable = NativeCleaner.register(this, HandleCleanup(handleRef))`;
-`close()` → `cleanable.clean()`; standalone inner
-`private class HandleCleanup(handleRef: AtomicLong) : Runnable` whose `run()` does
-`handleRef.getAndSet(0)` then `QueriesNative.dataContractDestroy(h)` iff non-zero
-(destroys exactly once, whichever of close/GC fires first). `AtomicLong handleRef`
-and `value` are unchanged; no call sites change.
-
-### Test plan
-`close()` idempotency + registration are the testable surface; the native destroy
-and GC-timing of the phantom backstop are not deterministically unit-testable. Rely
-on parity with the already-shipped `ContactRequestRef` pattern + compile; add a
-`NativeCleaner`-level idempotency assertion only if a fake action seam already
-exists. (Noted as best-effort per the untestable-path carve-out.)
-
----
-
-## D. Durable contact-crypto queue persistence — DEFERRED (own PR)
-
-The DashPay deferred contact-crypto queue (`PlatformWalletChangeSet.
-pending_contact_crypto_added/_cleared`) is not durable across process death on FFI
-hosts. Research (see below) shows only the **write** half is cleanly addable; the
-**restore** half is blocked upstream:
-
-- The in-repo SQLite persister already writes the queue, but its reader is
- `#[cfg(test)]`-only "because production load restore is blocked upstream
- (`LOAD_UNIMPLEMENTED: ClientStartState::wallets`)".
-- `rs-platform-wallet/src/wallet/apply.rs:111-116` drops the queue fields on the
- changeset-replay path; restore is meant to flow through the wallet **start-state**
- path, which isn't wired for this field.
-
-Adding the FFI `on_persist_pending_contact_crypto_fn` slot + JNI trampoline + a
-`DashDatabase` v3→v4 Room migration + Swift SwiftData model would persist rows that
-**nothing reads back** — a large, irreversible, cross-cutting surface (including a
-schema migration) for **zero cross-restart durability** until the upstream
-start-state restore exists. This is worse than the current honest deferral: the
-recurring signerless sweep already re-enqueues the work after restart, so the only
-exposure is delayed (not lost) contact-crypto work between sweeps.
-
-**Decision:** keep as a documented leftover; implement as a dedicated PR once the
-`rs-platform-wallet` start-state restore path lands. Full data-path map (producers,
-SQLite schema, FFI/JNI/Kotlin/Swift layers, the add-a-persisted-field recipe) is
-recorded for that future PR.
diff --git a/docs/dashpay/KOTLIN_MIGRATION_LEFTOVERS.md b/docs/dashpay/KOTLIN_MIGRATION_LEFTOVERS.md
deleted file mode 100644
index 7113bb97c58..00000000000
--- a/docs/dashpay/KOTLIN_MIGRATION_LEFTOVERS.md
+++ /dev/null
@@ -1,98 +0,0 @@
-# Kotlin DashPay Migration — Known Leftovers & Follow-ups
-
-Durable record of what is and isn't done as of the consolidated DashPay migration
-PR (K1 + K2 + K3 + the follow-up fixes below). Kept so nothing is silently lost
-when the four stacked PRs collapse into one.
-
-## Resolved in this PR (follow-up fixes)
-
-- **Seed hygiene on the platform-address signing path** — the mnemonic now crosses
- JNI as a scrubbable UTF-8 `byte[]` (`signWithMnemonicAndPathInto`) that the caller
- zeroes after use, never an un-scrubbable `java.lang.String`. Rust copies it
- directly into a pre-sized `Zeroizing>`, NUL-terminates that buffer in
- place, and never routes the phrase through `CString`. Pins:
- `SignerMnemonicScrubTest` (JVM, red→green).
-- **Payment dispose-mid-send double-send guard** — the send flow was extracted to
- `performDashPaySend` + a `PaymentSender` seam so the `withContext(NonCancellable)`
- guard (broadcast + durability bookkeeping stay atomic against a mid-send teardown)
- has a deterministic regression test: `PerformDashPaySendDoubleSendGuardTest` (JVM,
- red→green).
-- **`DataContractRef` GC backstop** — now registers a `NativeCleaner` like every
- other owned handle, so a leaked (never-closed) ref no longer leaks the native
- contract handle.
-- **`signWithMnemonicAndPathInto` instrumented sign smoke** —
- `FfiSmokeTest.mnemonicAndPathSignerSymbolLoadsAndSigns` loads the exact JNI
- symbol on-device, derives from a valid BIP-39 vector, and returns a compact
- recoverable signature. The direct test caller scrubs its JVM-owned mnemonic
- array in `finally`; JNI scrubs only its Rust-owned copy.
-
-## Deferred to dedicated follow-up PRs
-
-- **Durable contact-crypto queue persistence (item D).** The DashPay deferred
- contact-crypto queue (`PlatformWalletChangeSet.pending_contact_crypto_added/
- _cleared`) is not durable across process death on FFI hosts. Only the *write* half
- is cleanly addable; the *restore* half is blocked upstream — the in-repo SQLite
- persister's reader is `#[cfg(test)]`-only "because production load restore is
- blocked upstream (`LOAD_UNIMPLEMENTED: ClientStartState::wallets`)", and
- `rs-platform-wallet/src/wallet/apply.rs` drops the queue fields on the
- changeset-replay path. Adding the FFI slot + JNI trampoline + a `DashDatabase`
- v3→v4 Room migration + Swift model would persist rows nothing reads back — a large,
- irreversible surface (incl. a schema migration) for zero durability until the
- upstream start-state restore lands. The recurring signerless sweep already
- re-enqueues the work after restart, so the exposure is *delayed*, not *lost*
- contact-crypto work between sweeps. Do it as its own PR once the start-state
- restore exists. Full data-path map + add-a-persisted-field recipe recorded during
- the follow-ups research.
-
-- **PARITY: 94 ported / 5 partial / 0 deferred views**
- (`packages/kotlin-sdk/PARITY.md`). All 10 DashPay screens remain fully ported.
- The five partials are `CreateIdentityView`, `IdentityDetailView`,
- `TransitionDetailView`, `WalletMemoryExplorerView`, and `CoreContentView`.
- All 23 transition-catalog definitions now execute; `TransitionDetailView` is
- partial only because `identityUpdate`'s add-keys sub-path remains on the
- dedicated AddIdentityKey flow rather than the catalog form. Each partial row
- names its concrete remaining FFI, persistence, UI/catalog-adaptation, device,
- or restart gate.
-
-- **`SigningKeyUnavailable` MESSAGE_MARKER fallback removal.** The signer's
- "missing key" failure now travels as a typed completion code (rs-sdk-ffi
- `DashSDKSignerErrorCode::SigningKeyUnavailable` → platform-wallet code 31 →
- `DashSdkError.PlatformWallet.SigningKeyUnavailable`). The message-marker
- sniff on the catch-all codes is retained ONLY for the #4191 merge-order
- transition and for conversion paths that lose the machine prefix — NOT for
- mixed old-native/new-Kotlin builds, which the completion JNI arity change
- (3→4 args) makes unsupported outright; delete it (and `MESSAGE_MARKER`'s
- matcher role) in the next minor release. Accepted residual until rs-dpp grows a typed variant: the
- Rust-internal segment rides the `signer_error:key_unavailable: ` prefix
- through `ProtocolError::Generic` (typed at both ABI edges, one Rust-owned
- constant bridging the string segment).
-
-- **On-device `KeyPermanentlyInvalidatedException` coverage.** The
- invalidation recovery (generation-checked alias deletion + re-derive via
- forced repair) is pinned at the unit tier through the fake Keystore seam;
- a REAL KPIE requires biometric re-enrollment mid-test, which CI's emulator
- cannot do — same residual #4172 accepted. Exercise manually per the device
- test plan when touching the invalidation path.
-
-## Environment-bound (cannot be code-fixed here)
-
-- **End-to-end send→accept→pay testnet UAT** — device/testnet-bound; not runnable in
- CI. Must be exercised on a real testnet wallet before relying on the full DashPay
- flow.
-- **Live-network paths** (`searchDpnsNames`, `dashPaySyncNow`, the DashPay write
- paths) are exercised only under the `-Ptestnet=true` instrumented tier.
-
-## Behavioral notes to carry (from the base Kotlin SDK PR)
-
-- `rs-sdk-ffi` / `rs-sdk-trusted-context-provider` use rustls + webpki roots instead
- of the platform TLS stack (OpenSSL doesn't exist on Android). This also changes the
- iOS trust roots — no API change, but worth an iOS-side look.
-
-## Review findings — all resolved upstream
-
-Every P0/P1/P2 from the base-PR reviews (invalid Cargo `--features` arg, JNI
-local-frame leaks, negative-amount/index/selector validation across credits/tokens/
-funding/identity, `sendDashPayPayment` guard) was verified **already fixed** at the
-base PR HEAD before this consolidation — none was outstanding. The only carried
-review finding was the contact-crypto durability suggestion, addressed as item D
-above.
diff --git a/docs/dashpay/KOTLIN_MIGRATION_SPEC.md b/docs/dashpay/KOTLIN_MIGRATION_SPEC.md
deleted file mode 100644
index 784ff8897d8..00000000000
--- a/docs/dashpay/KOTLIN_MIGRATION_SPEC.md
+++ /dev/null
@@ -1,441 +0,0 @@
-# DashPay — Kotlin/Android Migration Spec
-
-**Historical baseline:** This document preserves the reviewed pre-migration plan
-and its then-current 88/90 inventory. It is not the current open-work tracker:
-subsequent implementation changed the contact-crypto persistence boundary,
-shipped invitation support on iOS, and closed the cited transition FFI gaps. See
-[`packages/kotlin-sdk/PARITY_SUMMARY.md`](../../packages/kotlin-sdk/PARITY_SUMMARY.md) and
-[`docs/sdk/sdk-parity-manifest.json`](../sdk/sdk-parity-manifest.json) for current
-capability status.
-
-Port the complete DashPay feature (as shipped for iOS in PR #3841, merged to
-`v4.1-dev` 2026-07-06) to the Kotlin/Android SDK and KotlinExampleApp
-(PR #3999, branch `feat/kotlin-sdk-and-example-app`).
-
-Status: **v2 — post five-lens review** (feasibility, scope, security,
-adversarial, domain-fit); all must-fixes folded in.
-Reference implementation: `packages/swift-sdk` + SwiftExampleApp.
-Feature spec: `docs/dashpay/SPEC.md` (+ companion specs in `docs/dashpay/`).
-
----
-
-## 1. Problem
-
-The Kotlin SDK (PR #3999) is a one-for-one Android port of SwiftExampleApp,
-snapshotted **before** PR #3841 landed. #3841 completed DashPay on iOS:
-
-- Replaced the utilitarian `FriendsView` (which the Kotlin `FriendsScreen`
- mirrors — now a port of a **deleted** Swift view) with a first-class
- **DashPay tab**: 10 views, ~3.8k LOC (`Views/DashPay/`).
-- Added a recurring **DashPay background sync** service
- (`platform_wallet_manager_dashpay_sync_*`, 7 FFI fns).
-- Added **payment history**, **cached contact profiles**, **contactInfo**
- (alias/note/hidden, cross-device), **ignore tombstones**, **seedless
- unlock + deferred contact-crypto drain**, and **DIP-15 auto-accept QR**.
-- Net: 23 new C exports in `rs-platform-wallet-ffi`; 6 old exports removed
- (incl. the `*reject_contact_request` pair — reject became ignore).
-
-Today the Kotlin stack has: 3 of 5 DashPay Room entities (field-exact with
-SwiftData), the send/accept/ignore/sync contact-request pipeline bridged
-(17 JNI exports in `tokens.rs`), and one compressed `FriendsScreen`. It
-lacks: profile/contactInfo writes, payments, the contact-profile cache, the
-sync service, QR, seedless unlock, wallet-scoped DPNS search, and the
-DashPay tab. `PARITY.md` still claims 88/90 ported against the stale
-pre-#3841 view list.
-
-**Compatibility baseline:** the existing 17 exports were reconciled with
-the #3841 Rust in commit `2298a2059f` (reject→ignore, `coreSignerHandle`
-threading, contacts-vtable ignored-sender deltas + contactInfo metadata
-fields, Room v1→v2). That reconciliation was verified by compile + 3
-Robolectric tests only — no runtime exercise of the Android
-send/accept/ignore path against a network has been recorded since the
-merge. K1 therefore opens with a runtime revalidation gate (§4).
-
-**Goal:** feature parity with iOS DashPay per the parity doctrine
-(`kotlin-sdk/CLAUDE.md`): cite Swift sources in KDoc, reuse iOS
-accessibility identifiers as Compose `testTag`s, keep all orchestration in
-Rust.
-
-## 2. What does NOT need porting
-
-All DashPay *logic* is shared Rust, already compiled into this branch and
-consumed by iOS through the same C ABI:
-
-- Contact-request crypto (ECDH, AES-256-CBC, DIP-15 69-byte compact xpub),
- DIP-14/15 derivation, accountReference — `rs-platform-wallet` +
- `platform-encryption` + `key-wallet`.
-- Accept → reciprocal request → `register_external_contact_account`
- (friendship address derivation) — automatic inside Rust.
-- The recurring sync sweep (`manager/dashpay_sync.rs`), seed binding
- verification, deferred contact-crypto queue, auto-accept QR
- build/parse/proof.
-- The bundled DashPay contract.
-
-Kotlin re-implements **none** of this. Per doctrine, no JNI function may
-stitch Rust calls together — any composite gap found during the port goes
-into `rs-platform-wallet-ffi` as its own reviewed change (none are
-currently known to be needed; iOS ships on the existing exports).
-
-**Deliberately not bridged** (zero non-README callers in the Swift app —
-the app reads this data from persisted rows, not handles; verified
-function-by-function during review):
-
-- The 8 `contact_request_*` field getters + `contact_request_create`.
-- The 14 `established_contact_*` accessors +
- `managed_identity_get_established_contact` /
- `_get_sent_contact_request` — `ContactDetailView` reads alias/note/
- hidden/paymentChannelBroken off `PersistentDashpayContactRequest` rows
- and writes via `set_dashpay_contact_info_with_signer`.
-- `managed_identity_is_contact_established`,
- `platform_wallet_pubkey_hash_from_private_key` (no app consumers),
- and the `managed_identity`-level send/accept/ignore variants (Kotlin
- uses the `platform_wallet_*` composites, as iOS does).
-
-Consequence: **no new handle-wrapper types.** The existing
-`ContactRequestRef` / `EstablishedContactRef` (`tokens/Dashpay.kt`) stay
-as-is for the accept path; new screens are driven by Room Flows and
-snapshot data classes. Rule: never retain a native handle in Compose
-composition — snapshot fields at the JNI boundary (a Cleaner can free a
-handle mid-read otherwise).
-
-## 3. Approach
-
-Three milestones, re-cut after review so that **every bridged function
-lands in the milestone that first consumes and tests it** (the original
-bottom-up "bridge everything first" cut concentrated all marshaling
-defects at final UAT — rejected).
-
-### Alternatives rejected
-
-- **Re-orchestrating DashPay flows in Kotlin**: forbidden by doctrine;
- Swift ships proof that single-call composites suffice.
-- **Keeping `FriendsScreen` and bolting features onto it**: its Swift
- counterpart was deleted; keeping a dead view's port violates parity.
-- **uniffi / JNA instead of hand-rolled JNI**: the crate has 160
- hand-rolled exports with established support patterns (`support::guard`,
- handle-as-jlong, GlobalRef callbacks); mixing binding generators adds
- toolchain cost for no capability gain.
-- **Bridging the full ~47-function FFI sweep**: 24 of them have no
- consumer anywhere in the reference app (see §2); bridging them would
- add dead surface + a speculative handle-wrapper abstraction.
-
-## 4. Work plan
-
-New JNI exports go in a new `rs-unified-sdk-jni/src/dashpay.rs` +
-`ffi/DashpayNative.kt`; the 17 existing DashPay exports stay in
-`tokens.rs` (moving them is a follow-up refactor). Array-free FFI
-counterparts (`dashpay_payment_array_free`, `dpns_search_results_free`,
-`platform_wallet_manager_free_account_balances`) are consumed Rust-side
-inside the JNI wrappers, as the existing exports do.
-
-⚠ Lockstep rule: extending the identity-entry persist callback changes the
-`onPersistIdentityUpsert` JNI method descriptor (hand-written string,
-`persistence.rs:866-893`) — the Rust descriptor and the Kotlin
-`NativePersistenceBridge` signature must change in the same commit or
-every identity persist fails at runtime.
-
-### Milestone K1 — Persistence completion + the read surface it can prove
-
-**Entry gate:** one runtime revalidation of the existing 17-export
-pipeline — the FriendsScreen send→accept→ignore flow against testnet
-(`-Ptestnet=true`), or its instrumented equivalent — before any new
-bridging. Also the 15-minute PARITY.md interim fix: drop the stale
-`FriendsView.swift` row claims, mark the DashPay section
-"in migration — see KOTLIN_MIGRATION_SPEC.md".
-
-**Bridge (6 exports):**
-
-| Group | Functions |
-|---|---|
-| Payments | `managed_identity_get_dashpay_payments` |
-| Profile reads | `platform_wallet_get_contact_profile`, `managed_identity_get_dashpay_profile`, `managed_identity_get_dashpay_sync_state` |
-| DPNS search | `platform_wallet_search_dpns_names` (wallet-scoped; the existing `QueriesNative.dpnsSearch` wraps the *SDK-scoped* `dash_sdk_dpns_search` — a different call path; AddContactScreen must use the wallet-scoped one for parity) |
-| Account balances | `platform_wallet_manager_get_account_balances` (DashPayTabView's per-account balance display; not DashPay-prefixed, easy to miss) |
-
-**Persistence** (mirrors `PersistentDashpayContactProfile.swift` /
-`PersistentDashpayPayment.swift`):
-
-- New Room entities `DashpayContactProfileEntity`
- (`(networkRaw, ownerIdentityId, contactIdentityId)` unique, `checkedAtMs`
- backoff) and `DashpayPaymentEntity`
- (`(networkRaw, ownerIdentityId, txid)` unique) + DAOs + Flows;
- `DashDatabase` v2→v3 additive migration, exported schema `3.json`.
-- **Persist direction — contact profiles:** extend identity-entry
- marshaling to carry `IdentityEntryFFI.contact_profiles` (slot exists,
- currently skipped at `persistence.rs:825-895`). **Tombstone semantics
- are load-bearing:** the projection emits `is_present == false` rows that
- mean DELETE the persisted row (`identity_persistence.rs:600-608`) — an
- upsert-only implementation compiles, passes upsert tests, and leaves
- stale contact names/avatars forever. Required: `is_present=false` → DAO
- delete (with test) + `checkedAtMs` round-trip fidelity test. (Note: the
- outer doc comment at `identity_persistence.rs:146` contradicts the
- projection code and should be corrected in passing.)
-- **Persist direction — payments:** pull-based, mirroring iOS exactly.
- **Invariant:** payment rows reach Room *only* via the
- `refreshDashPayPayments` equivalent (FFI read → Room upsert);
- `dashpay_sync_now` reconciles payments **in-memory without persisting**
- (identity persist skips payments, `identity_persistence.rs:37-39`).
- Android process death is aggressive, so K3's send flow must call
- refresh-after-send, and the K1 test suite pins the invariant.
-- **Restore direction:** populate the null-stubbed `contact_profiles` /
- `payments` arrays in `rs-unified-sdk-jni/src/persistence.rs`
- (staging comment :1476-1480, stubs :1848-1853, free-path :2049-2051),
- mirroring the Swift restore blocks
- (`PlatformWalletPersistenceHandler.swift` ~4918, ~4978).
-
-**Gate (re-tiered after review):** the persist→wipe→restore→re-read
-round-trip runs as an **instrumented test** (extend
-`sdk/src/androidTest/.../WalletManagerRoundTripTest.kt`, which already
-drives the real native lib on the CI emulator) — payload includes payments
-with memo/direction/status, a contact-profile `is_present=false` tombstone,
-and ignored senders; the new K1 getters read back restore-injected fixtures
-and assert field equality. JVM/Robolectric tests cover handler↔Room mapping
-only (they never load the native lib, so they cannot see marshaling bugs —
-this was the original spec's top-listed risk guarded by the wrong tier).
-
-### Milestone K2 — Sync service, seedless unlock, writes
-
-**Pre-req (security must-fix): mnemonic-handling discipline.** The
-existing Kotlin resolver materializes the mnemonic as an immutable JVM
-`String` (`WalletStorage.retrieveMnemonic` does `decodeToString()` and
-scrubs only the ByteArray; `MnemonicResolverAndPersister.kt:36-42` returns
-the String to native). iOS never creates a string-shaped copy: it keeps
-XOR-masked UTF-8 bytes (`MaskedMnemonicUTF8`) and writes into Rust's
-out-buffer, scrubbing on every access. K2 multiplies resolver calls (the
-drain runs it once per queued entry), so **before** wiring the automatic
-drain: port the out-buffer + masked-bytes discipline to `WalletStorage.kt`
-/ `MnemonicResolverAndPersister.kt` / `mnemonic.rs` (whose "same residual
-exposure as iOS" comment is false and must be corrected), and zeroize the
-`mnemonic_str`/`mnemonic_c` copies in `signer.rs:342-422` (currently only
-`sig_buf` is scrubbed).
-
-**Bridge (13 exports):** the 7 `platform_wallet_manager_dashpay_sync_*`
-fns; the seedless trio `platform_wallet_verify_seed_binds_to_wallet`,
-`platform_wallet_pending_contact_crypto_count`,
-`platform_wallet_drain_pending_contact_crypto` (takes **both** a
-`SignerHandle` and a `MnemonicResolverHandle` —
-`rs-platform-wallet-ffi/src/dashpay.rs:757-761`);
-`platform_wallet_create_or_update_dashpay_profile_with_signer`,
-`platform_wallet_set_dashpay_contact_info_with_signer`;
-`dash_sdk_resolver_supports_key_type` (rs-sdk-ffi; consumed by the
-production signer — Swift `KeychainSigner.swift`).
-
-**`DashpaySyncService`** (`sdk/services/`, mirroring
-`PlatformWalletManagerDashPaySync.swift`):
-
-- Owned by the `PlatformWalletManager` instance; started when platform
- wallets are present (after load / on the rebind path), stopped and
- disposed on the `WalletManagerStore` manager swap. **Not gated on
- process lifecycle** — iOS keeps the sweep running while backgrounded
- (the OS suspends the process; Android freezes/kills similarly under
- modern app-standby), and this matches the manager-owned ownership
- doctrine. (The v1 spec's "mirrors scenePhase" claim was wrong — Swift
- drives start/stop off wallet-presence/rebind `.onChange`, not
- scenePhase. No `lifecycle-process` dependency needed.)
-- `isSyncing` / `lastSync` / `pendingAccountBuilds` exposed as `StateFlow`,
- updated by a **1 Hz poll with change-gated assignment and stale-key
- pruning** — pinned now, not "verify later": that is exactly how iOS does
- it (`PlatformWalletManager.swift:1140-1186`; the comment at :1133-1139
- documents why naive re-assignment burned CPU). Natural home: the
- `SpvProgressPublisher` pattern.
-
-**Seedless unlock — invocation topology (domain must-fix; the API method
-alone is not the feature):**
-
-- `unlockWalletFromKeystore(walletId)` = scoped verify → drain, and is
- called **automatically, per restored wallet, inside
- `loadPersistedWallets`**, best-effort/never-throwing (mirrors
- `PlatformWalletManager.swift:477-506`; Kotlin's
- `PlatformWalletManager.kt:457` currently documents the absence). This is
- load-bearing: the deferred contact-crypto queue is **in-memory by
- design** (no persisted table on either platform — do not "fix" that) and
- recovery is self-healing only if every launch runs
- load → unlock (verify+drain) → sweep. Banner-triggered-only unlock would
- leave contacts never finishing establishment after process restart.
-- **Seed-mismatch contract:** Rust `SeedMismatch` surfaces as
- `ErrorInvalidParameter` (`rs-platform-wallet-ffi/src/dashpay.rs:960-965`);
- like Swift, Kotlin disambiguates *only* by scoping the catch to the
- verify call (the JNI error code arrives as
- `code + PWFFI_CODE_OFFSET`). Publish per-wallet
- `draining` / `seedMismatch` / `pendingAccountBuilds` as `StateFlow`
- (Swift: `dashPayUnlockStatus`).
-- **Re-entrancy guard:** a second unlock while `draining == true` returns
- immediately (Swift :622-624) — load-time auto-drain and a user banner
- tap must not double-run the ECDH work.
-- **Biometric interaction (Android-only failure mode):** Kotlin's
- identity-key Keystore alias is auth-gated (30 s validity +
- `BiometricGate`) — *stricter than iOS*, whose identity keys are not
- auth-gated at all. The reciprocal-accept signing inside a background
- drain can therefore throw on an expired auth window with no Activity to
- re-prompt. Required behavior: catch, leave the entry queued (the sweep
- self-heals), reflect it in the unlock status; instrumented test for the
- auth-expired path.
-- **Breadcrumb backfill — explicitly not ported.** Swift's unlock also
- schedules `scheduleBackfillIdentityKeyBreadcrumbs`, an iOS-legacy
- Keychain healing step for pre-breadcrumb installs. The Kotlin SDK is new
- — every identity key it has ever created is breadcrumbed at creation —
- so there is nothing to heal. Recorded here so its absence is a decision,
- not an oversight.
-
-**Gate:** unit tests + instrumented sync-service lifecycle tests
-(start/stop/isRunning; manager-swap disposal; double-start), unlock state
-machine with a wrong-seed fixture (seedMismatch path) and the
-auth-expired-drain path; write paths against testnet behind
-`-Ptestnet=true`.
-
-### Milestone K3 — DashPay tab UI + parity bookkeeping
-
-**Bridge (2 exports):** `platform_wallet_build_auto_accept_qr`,
-`platform_wallet_send_contact_request_from_qr` (first consumed here).
-
-Navigation restructure (mirrors Swift `ContentView.swift`):
-
-- `RootTab`: `SYNC, WALLETS, IDENTITIES, DASHPAY, SETTINGS` — **Contracts
- tab is demoted into Settings** (`ContractsHome` becomes an entry in the
- Settings screen's Platform section, as on iOS).
-- Retire `FriendsScreen` + its route; entry points repoint at the DashPay
- tab/contact flows.
-- Reset the DashPay tab's identity-picker selection on network switch —
- retained Compose state would otherwise query a wallet absent on the new
- network-locked manager.
-
-Port the 10 Swift views (Compose screens under `ui/dashpay/`, one file per
-Swift file, testTags = iOS accessibility identifiers, screens driven by
-Room Flows / snapshot data classes — never by retained native handles):
-
-| Swift (`Views/DashPay/`) | Compose target | Notes |
-|---|---|---|
-| DashPayTabView (909) | `DashPayTabScreen` | identity picker, per-account balances, pull-to-refresh → `syncNow`, unlock banner (reads unlock-status Flow), sub-sheet navigation |
-| ContactsView (299) | `ContactsScreen` | Room Flow over established contacts (both-direction join) |
-| ContactRequestsView (391) | `ContactRequestsScreen` | incoming accept/ignore + outgoing pending |
-| AddContactView (497) | `AddContactScreen` | wallet-scoped DPNS prefix search (300 ms debounce), raw id, QR entry |
-| ContactDetailView (561) | `ContactDetailScreen` | payment history (refresh→Room), alias/note editors **surfacing the `ContactInfoPublishOutcome`** — `DeferredUntilTwoContacts` means local-only until ≥2 established contacts and the UI must say so (parity), `SkippedWatchOnly` likewise; hide; send-payment entry |
-| SendDashPayPaymentSheet (386) | `SendDashPayPaymentSheet` | amount/memo → `sendDashPayPayment` → txid, then **refresh-after-send** (payments-durability invariant, §K1) |
-| DashPayProfileView (188) | `DashPayProfileScreen` | own profile display/edit + auto-accept QR render (ZXing — already a dependency) |
-| IgnoredContactsView (181) | `IgnoredContactsScreen` | unignore |
-| HiddenContactsView (240) | `HiddenContactsScreen` | unhide |
-| DashPayContactMeta (183) | `DashPayContactMeta.kt` | meta store (UserDefaults → SharedPreferences/DataStore; plaintext is parity — iOS documents UserDefaults as "the honest backing" for device-local data), display-name precedence, avatar composable |
-
-QR scan reuses the already-ported `QrScannerScreen`; the auto-accept
-scan-to-send path calls `sendContactRequestFromQR`.
-
-**New dependency:** Coil (`coil-compose`) for avatar loading — not
-currently in `libs.versions.toml`; flagged here because it is the plan's
-only new third-party dependency. (ZXing is already present.)
-
-Parity bookkeeping: full `PARITY.md` DashPay rewrite — a `Views/DashPay/`
-section with one row per view, corrected totals (interim stale-claims fix
-already landed in K1).
-
-**Gate:** `:app:assembleDebug` + Compose UI tests mirroring
-`DashPayTabUITests.swift`; manual UAT next to the iOS simulator per
-`QA_TESTCASES_SPEC.md` flows, including the end-to-end
-send→accept→pay testnet run.
-
-## 5. Interfaces & data flow (summary)
-
-```
-Compose UI ── StateFlow/Room Flow ── PlatformWalletManager / Dashpay.kt
- │ │ (thin, marshal-only)
- │ DashpayNative.kt (external fun)
- │ │ JNI
- ▼ rs-unified-sdk-jni/src/dashpay.rs
- Room (Dashpay* entities) │ rlib call
- ▲ rs-platform-wallet-ffi (C ABI)
- │ persistence callbacks │
- └── NativePersistenceBridge ◄── rs-platform-wallet (all logic)
-```
-
-- Writes require two callback handles: identity signer (`signer.rs`) and
- mnemonic resolver (`mnemonic.rs`) — both exist; DashPay adds no new
- callback *types*, only new call sites. Handles are kept strongly
- referenced for the duration of each call (GC hazard).
-- Contact profiles ride the identity persist/restore callback path
- (with tombstone-delete semantics); payments are pull-persisted and
- array-restored. The deferred contact-crypto queue is deliberately
- not persisted (in-memory + sweep self-heal, both platforms).
-- Cold-start contract: `loadPersistedWallets` → per-wallet best-effort
- unlock (verify → drain) → sync service start → recurring sweep.
-- Threading: FFI calls on `Dispatchers.IO`; Rust→Kotlin callbacks attach
- as JVM daemon threads; persistence-handler read-modify-writes run in
- Room transactions (callback threads race UI-triggered refreshes
- otherwise).
-
-## 6. Failure modes / risks
-
-- **Restore-path marshaling bugs** corrupt Rust wallet state on load.
- Guard: the K1 instrumented round-trip (real native lib) — JVM tests
- cannot see this code.
-- **Contact-profile staleness:** upsert-only persist misses tombstones →
- stale names/avatars forever. Guard: `is_present=false` delete test (K1).
-- **Payment loss on process death:** `syncNow` does not persist payments.
- Guard: refresh-after-send + kill/relaunch test (K1/K3).
-- **Never-draining wallets:** unlock not wired into load → contacts stuck
- pending after every restart. Guard: unlock topology spec (§K2) +
- restore→unlock instrumented test.
-- **Background drain vs biometric gate:** auth-expired signing during
- drain must requeue, not fail silently. Guard: auth-expired test (K2).
-- **GC vs callback lifetime** during long drains: strong refs on
- signer/resolver bridges per call site; drain stress test.
-- **Sync-service leak across network switch:** manager-owned lifecycle,
- disposed on `WalletManagerStore` swap; double-start test. UI-side:
- identity-picker reset on network change.
-- **JNI descriptor lockstep** on the identity persist callback (§4 note):
- Rust descriptor + Kotlin signature in one commit.
-- **Room migration:** additive-only v3; migration test against `2.json`.
-- **Build env:** exFAT gotcha (`build_android.sh` sparse image), NDK r28+,
- 16 KB alignment — K1 ends with `./build_android.sh --verify` passing.
-
-## 7. Test plan
-
-1. **Instrumented (`connectedDebugAndroidTest`, CI emulator)** — the
- load-bearing tier: extend `WalletManagerRoundTripTest` with the DashPay
- persist→wipe→restore→re-read round-trip (payments incl. memo/direction/
- status, contact-profile tombstone, ignored senders); K1 getters read
- restore-injected fixtures; K2 sync-service lifecycle, unlock
- state machine (wrong-seed → seedMismatch; auth-expired drain).
-2. **JVM unit (`:sdk:testDebugUnitTest`)** — handler↔Room mapping only
- (explicitly *not* the marshaling tier): contact-profile
- upsert/delete/backoff, payment row mapping, Room v2→v3 migration.
-3. **Compose UI tests** — port `DashPayTabUITests.swift` flows using the
- shared testTags (tab presence, add-contact form, requests accept path
- with a fake bridge).
-4. **Testnet opt-in (`-Ptestnet=true`)** — K1 entry gate: existing
- FriendsScreen send→accept→ignore revalidation. K3 exit: end-to-end
- send→accept→pay between two fixture identities, mirroring iOS UAT.
-5. **CI** — existing `kotlin-sdk-build.yml` runs tiers 1–3.
-
-## 8. Out of scope
-
-- Invitations (SPEC.md Milestone 5) — not implemented on iOS either.
-- The 24 unconsumed FFI functions listed in §2 (and any new handle-wrapper
- types for them).
-- `managed_identity_get_contested_dpns_names` — its consumers
- (SelectMainName / WalletMemoryExplorer / IdentityDetail) are outside
- `Views/DashPay/`; it belongs to the existing non-DashPay PARITY-partial
- bucket, not this migration.
-- Identity-key breadcrumb backfill (justified in §K2 — no pre-breadcrumb
- Android installs can exist).
-- Migrating the 17 pre-existing DashPay JNI exports out of `tokens.rs`.
-- The 5 non-DashPay `TransitionDetailView` FFI gaps and other PARITY
- "partial" items.
-- Any change to Rust crates other than `rs-unified-sdk-jni` (a genuine
- composite gap, if found, becomes its own reviewed `rs-platform-wallet-ffi`
- change).
-
-## 9. Decisions taken in this spec (previously open)
-
-- **One PR per milestone** — the milestones carry independent gates by
- design; review units should match.
-- **Sync lifecycle: manager-owned, not process-lifecycle-gated** (§K2).
-- **Poll (1 Hz, change-gated), not events, for sync/unlock status** (§K2).
-- **Coil added** as the single new dependency (§K3).
-
-## 10. Open questions (for Ivan)
-
-1. **Branch/PR strategy:** land the K-milestones as stacked PRs on top of
- `feat/kotlin-sdk-and-example-app` (PR #3999 is already ~50k insertions),
- or fold into #3999? Recommendation: **stacked PRs**.
-2. **Tab restructure confirmation:** mirroring Swift means demoting the
- Contracts tab into Settings on Android too. Confirm parity wins over
- Android-specific navigation taste.
diff --git a/docs/dashpay/MULTI_ACCOUNT_SPEC.md b/docs/dashpay/MULTI_ACCOUNT_SPEC.md
deleted file mode 100644
index dfdf657a79e..00000000000
--- a/docs/dashpay/MULTI_ACCOUNT_SPEC.md
+++ /dev/null
@@ -1,330 +0,0 @@
-# DashPay simultaneous multi-account contacts — implementation spec
-
-> **Problem.** DIP-15 lets a contact expose **multiple DashPay accounts** at once —
-> each a separate `contactRequest` with a distinct `accountReference` (DIP-15 §8.4,
-> §8.9, §10.8). Our contact-state layer collapses everything to **one channel per
-> counterparty** (`BTreeMap`, a single-channel `EstablishedContact`),
-> and the rotation machinery actively *supersedes* a contact's prior request rather
-> than letting accounts coexist. So we can neither represent nor pay across a
-> contact's multiple accounts, we always send our own account `0`, and we drop
-> `contactInfo.acceptedAccounts` on ingest.
->
-> **Status.** REVIEWED (4 lenses, 2026-06-24) — **KEEP DEFERRED** (see *Review
-> outcome* below: a foundational blocker B-1 + reopened DoS + abuse surface, and no
-> requirement). This feature was **deliberately
-> deferred** by the team as *"conditional, not a requirement"* (backlog
-> dashpay/platform#4020 multi-account item; `DIP_CONFORMANCE_GAPS.md` §2). There is **no current product
-> requirement** forcing simultaneous multi-account. This spec exists so the work is
-> *scoped and reviewed* and can be implemented when a requirement appears — and so
-> the decision to keep deferring is an informed one. **Do not implement before this
-> spec is reviewed and a requirement exists.**
->
-> **Source.** Scope map from the 2026-06-24 blast-radius audit (all file:line below
-> verified against `feat/dashpay-m1-sync-correctness`, pinned rust-dashcore `b4779fc`).
-
----
-
-## Review outcome (2026-06-24, 4 lenses) — **KEEP DEFERRED**
-
-A four-lens review (DIP-15 domain-fit, state-machine feasibility, scope/go-no-go,
-security/abuse), each grounded against the code, converged: **the spec is an
-accurate scope map, but the feature must NOT be built as designed, and there is no
-product requirement driving it. Keep it deferred.** The reviews also corrected
-several claims in §0–§4 below (annotated inline as ⚠**REV**). If a requirement ever
-appears, **the first deliverable is a focused "channel identity under an opaque
-`accountReference`" design note (resolving B-1) — not code.**
-
-### Blocking findings (must be resolved in a revision before any code)
-
-- **B-1 — Channel identity is unsolvable from the wire (the foundational blocker).**
- The design keys channels by the raw `accountReference`, but DIP-15's
- `accountReference = (version<<28) | (ASK28 ^ account)` is a sender-private one-time
- pad: the version nibble is cleartext, but `ASK28 = HMAC(sender_secret, compact_xpub)`
- is uninvertible by the recipient, **and a rotation ships a new xpub**, so the
- low-28-bit value is *uncorrelated* across a rotation. Result: "rotation of added
- channel B" is **information-theoretically indistinguishable** from "a brand-new
- account." So channels cannot be keyed by `accountReference` and still collapse
- rotations. Channel identity must be **out-of-band** (user-assigned at accept time;
- every later rotation re-prompts "which channel does this replace?"). This is
- permanent UX, not a TODO — and it gates B-2/B-3 below. (`account_reference.rs:41-66`.)
-- **B-2 — Keying the collapse by `accountReference` re-opens the PR #3841 sweep
- thrash.** Immutable on-chain docs never disappear; a rotated sender leaves both
- old+new docs returning every sweep. `newest_received_per_sender` collapses
- per-sender *because rotation mutates the reference*; keying the collapse by the
- reference produces two survivors that flip-flop the stored channel forever — the
- exact regression #3841 fixed. A fixpoint exists only with a rotation-stable key →
- loops back to B-1. (`contact_requests.rs:811-829,1087-1117,3103-3105`.)
-- **B-3 — The "local channel index" corrupts the receiving derivation path.**
- `DashpayReceivingFunds.index` is a **hardened path component**
- (`account_type.rs:489`), not just a map key — it selects the BIP32 path the
- counterparty derives against. A fabricated local index desyncs our *advertised*
- receiving addresses from our *watched* ones → incoming payments to that channel
- become invisible. The receiving index must be our **real** DashPay account number
- (the one masked into the published `accountReference`); only the *external* account
- may use a namespace. The send-side real-account thread (§2.3) is the only correct
- mechanism. (`key-wallet account_type.rs:472-526`.)
-- **B-4 — `BTreeMap` silently overwrites on collision.** Keying
- by a non-unique 28-bit value means two channels masking equal silently shadow each
- other (fund misdirection). The on-chain unique index `($ownerId, toUserId,
- accountReference)` (`dashpay.schema.json:148-163`) bounds this **per-sender** (a
- sender can't broadcast two colliding docs), but the spec must *state and rely on*
- that invariant and **reject-on-collision** (insert returning `Some` = loud error),
- never overwrite.
-- **B-5 — No per-sender flood cap; the re-key converts a flood into pending-queue
- exhaustion.** Today `incoming_contact_requests` is one slot per sender + collapse →
- a flood is structurally absorbed. The re-key to `(counterparty, accountReference)`
- makes each new reference a pending triage prompt (a permanent doc returning every
- sweep). Needs a `MAX_PENDING_ADDITIONAL_ACCOUNTS_PER_SENDER` (mirror
- `MAX_AUTO_ACCEPT_QUEUED_PER_OWNER`, but per-(owner,sender)); over-cap → **silently
- drop, not enqueue**; wire the gate to the existing `ignored_senders` block.
-- **B-6 — "Add account" is a phishing / confused-deputy surface.** A *malicious
- established contact* can send an add-request whose xpub points at an
- attacker-controlled address space (the crypto binds the channel to the contact's
- identity, not to the contact being honest). "Add account" must carry the same
- trust gravity as accepting a brand-new contact (surface the derived first address;
- no one-tap inline accept). The spec frames the gate as anti-flood only and omits
- payment redirection.
-
-### Corrections to the body (factual)
-- ⚠**REV §2.2 / Open Q2:** the version nibble is readable but does **not** correlate a
- rotation to a specific channel (B-1). Don't claim "consult the version nibble"
- resolves rotation-vs-new — it doesn't.
-- ⚠**REV §2.1:** strike the "local channel index" for the receiving account (B-3).
-- ⚠**REV §2.2:** `acceptedAccounts` per DIP-15 §10.4 stores **only non-version-0**
- references — never write channel-0 into it.
-- ⚠**REV §4.4:** same-sender collisions are *blocked on-chain* by the unique index;
- state this invariant (it's the saving grace) and reject-on-collision (B-4).
-- ⚠**REV §4.3:** migration is essentially free — there is **no in-repo SQLite schema**,
- `DashMigrationPlan.stages == []` (dev stores recreate from scratch), and contacts
- rebuild from chain (metadata rides `contactInfo`). The spec over-worries; the real
- plan is "let the store rebuild," with a "wipe local → re-sync reconstructs" test.
-- ⚠**REV §3:** T2 (`accepted_accounts` round-trip) is **not** independently valuable —
- it writes a field nothing reads (inert). Fold it into T1; do **not** ship standalone.
- T1 itself must split into ≥4 PRs (struct re-key / collapse-inversion / user-gate /
- account-index thread), each with its own #3841-style fixpoint test.
-- **Conformant lower-cost fallback (R1):** DIP-15 §8.4 allows *"either disregard all
- future contact requests ... or preferably ask the user."* Silently disregarding
- additional requests (≈ today's collapse) is **also conformant** and avoids the
- entire B-2/B-3/B-5/B-6 surface — the cheapest path if multi-account is ever wanted
- only nominally.
-
-### Verdict & recommendation
-**KEEP DEFERRED.** Upstream (#813) is unblocked, but the feature has a foundational
-information-theoretic blocker (B-1), re-opens a fixed DoS (B-2), and adds real abuse
-surface (B-4/B-5/B-6) — for **no current requirement**. The review *prevented building
-the wrong thing*, which is the point of the pipeline. **Next step only if a
-requirement appears:** a B-1 channel-identity design note, then re-spec around it.
-
----
-
-## 0. What "multi-account" means here (and what it does NOT)
-
-Two distinct things share the "different `accountReference` from a known sender"
-shape and must not be conflated:
-
-- **Rotation (LIVE today):** the sender rotated the payment xpub for the *same*
- logical account; the new request **supersedes** the old (DIP-15 §8.10 immutability
- → rotate via a new request). `apply_rotated_incoming_request`
- (`state/managed_identity/contact_requests.rs:337-407`) replaces `incoming_request`
- in place, tears down the stale external account, rebuilds from the new xpub. The
- sync sweep's `newest_received_per_sender` (`network/contact_requests.rs:811-829`)
- **discards all-but-newest per sender** — the comment (`:752-765`) calls this "the
- idempotency keystone."
-- **Simultaneous multi-account (THIS spec):** the sender exposes *additional* live
- accounts that must **coexist** as separate channels (DIP-15 §8.4 "Recipients either
- ignore subsequent requests or prompt users to select destination accounts";
- §10.8 "additional contact requests require user acceptance; upon approval the new
- account reference joins `acceptedAccounts`").
-
-These are **antithetical** — rotation's whole purpose is to *prevent* two live
-channels per sender. Multi-account must *invert* that for **accepted** additional
-accounts while keeping supersede for genuine rotations. The disambiguation is the
-crux of this spec (§2.2).
-
-**Out of scope:** the §10.8 *query-level* flood mitigation ("only the first request
-to the bloom filter; filter blocked senders server-side") needs a registered
-`dashpay` contract change and stays blocked (Contract track). This spec covers the
-**client-side** multi-account model only.
-
----
-
-## 1. Research — current state (verified)
-
-### 1.1 What's already multi-account-ready
-- **Upstream derivation (#813, merged, in `b4779fc`):** `AccountType::derivation_path()`
- for `DashpayReceivingFunds`/`DashpayExternalAccount` uses
- `ChildNumber::from_hardened_idx(*account_index)` (`key-wallet/.../account_type.rs:472-531`)
- — the friendship path honors a non-zero account.
-- **Account collections** are keyed by `DashpayAccountKey { index, user_identity_id,
- friend_identity_id }` (`key-wallet/.../account_collection.rs:25-29`) — the
- account/UTXO layer already supports multiple accounts per (user, friend).
-- **Provider/register signatures already take an account index:**
- `receiving_xpub_for(…, account_index, …)`, `account_reference(…, account_index,
- version)`, `register_contact_account(…, account_index, …)` (`network/contacts.rs:140`),
- `register_external_contact_account` derives `DashpayAccountKey { index }`.
- The `accountReference` masking already folds `account_index` into the low 28 bits
- correctly (`network/contact_requests.rs:514-551`).
-
-### 1.2 The bottleneck — contact state collapses to `Identifier`
-`ManagedIdentity` (`state/managed_identity/mod.rs:62-85`):
-- `established_contacts: BTreeMap`
-- `sent_contact_requests: BTreeMap`
-- `incoming_contact_requests: BTreeMap`
-
-`EstablishedContact` (`types/dashpay/established_contact.rs:14-51`) holds **exactly
-one** `outgoing_request` + **one** `incoming_request`. It carries a dead
-`accepted_accounts: Vec` (`:34`) + `add/remove_accepted_account` (`:138-146`)
-with **zero production callers**.
-
-### 1.3 Hardcoded account `0` on send/build (≈6 sites)
-`network/contact_requests.rs:476` (`let account_index: u32 = 0;`), `contacts.rs:397`,
-the build sweep `DashpayAccountKey { index: 0 }` (`contact_requests.rs:1398`), the
-register-receiving builds (`:1614`, accept `:2259`).
-
-### 1.4 `accepted_accounts` is lossy
-- Codec round-trips it (`crypto/contact_info.rs:133,238-281`, test `:346-366`). ✅
-- Publish hardcodes empty (`network/contact_info.rs:499-506`, "isn't populated yet").
-- `set_contact_metadata` (`state/managed_identity/contact_requests.rs:279-313`)
- copies only `alias/note/display_hidden` — **drops `metadata.accepted_accounts`**.
-- Not marshalled to FFI/Swift anywhere.
-
-### 1.5 The recipient-ignores-`accountReference` asymmetry
-DIP-15 makes `accountReference` a sender-private one-time pad the recipient **cannot
-reliably un-mask** (the 4-way convention split; `DIP_CONFORMANCE_GAPS.md` §3). So the
-recipient **cannot** recover the sender's real account number from the wire. It can
-only treat the **raw `accountReference` u32** as an opaque channel discriminator, and
-derive the actual addresses from the **decrypted xpub** (which is account-correct).
-→ multi-account channels must be keyed by the **raw `accountReference`**, not an
-unmasked account number.
-
----
-
-## 2. Chosen approach
-
-### 2.1 Re-key contact state by `(counterparty, accountReference)`
-Replace the single-channel model with a per-contact set of channels keyed by the raw
-`accountReference`:
-- `EstablishedContact` becomes multi-channel: a `BTreeMap` where `ContactChannel` holds the `{outgoing_request,
- incoming_request, payment_channel_broken}` that are today flat on
- `EstablishedContact`. Metadata (`alias`, `note`, `is_hidden`, `accepted_accounts`)
- stays **per-contact** (one alias for the person, not per channel).
-- `incoming_contact_requests` / `sent_contact_requests` re-key to
- `(counterparty, accountReference)`.
-- Account registration already keys by `DashpayAccountKey { index }`; the channel's
- account index comes from the **decrypted-xpub-derived** account, but since we can't
- unmask, we allocate a **local channel index** per accepted accountReference and use
- it as the `DashpayAccountKey.index` (the xpub is account-correct regardless; the
- index only namespaces our local account collection).
-
-### 2.2 Disambiguate rotation (supersede) vs new account (coexist) — by USER GATE
-We cannot tell from the wire whether a new `accountReference` is a rotation or a new
-account (§1.5). DIP-15 §8.4/§10.8 resolves this with a **user gate**:
-- The **first** request from a sender → auto-established (channel 0), as today.
-- A **subsequent** request with a new `accountReference` from an established contact →
- surfaced as a **pending additional-account request**, NOT auto-applied. The current
- auto-`apply_rotated_incoming_request` supersede is **replaced** by: enqueue as
- pending; the user chooses **"replace addresses" (rotation)** or **"add account"
- (coexist)**.
- - "Replace" → supersede (today's behavior, the channel's request is swapped).
- - "Add" → the `accountReference` joins `accepted_accounts`, a new coexisting channel
- is built, and the receival/external accounts are registered under a fresh local
- index.
-- `accepted_accounts` is the **persistent record of which additional references the
- user accepted** — so the gate is sticky across sweeps/restarts (an accepted ref is
- never re-prompted; an un-accepted one is dropped per §10.8, not bloom-filtered).
-
-This **inverts the idempotency keystone** (`newest_received_per_sender` collapse) for
-accepted references: the sweep must keep every *accepted* `accountReference`'s newest
-doc, and collapse only *within* an accountReference (rotation of that channel). That
-is the load-bearing, highest-risk change (§4.1).
-
-### 2.3 Send side — thread a real account (gated behind a UI affordance)
-Thread an `account: u32` param from the send FFI through the ≈6 hardcoded sites. The
-example app gains an optional "send from account N" affordance; default stays `0`.
-**Not a standalone change** — only meaningful once §2.1 state can hold the result.
-
-### Alternatives rejected
-| Approach | Why rejected |
-|---|---|
-| Unmask `accountReference` to recover the account number, key by that | Recipient can't reliably un-mask (4-way convention split, §1.5). |
-| Auto-accept every new `accountReference` as a new account | Violates §10.8 flood mitigation; an attacker floods accounts. |
-| Keep single-channel, just stop dropping `accepted_accounts` (Slice A) | Inert today (nothing produces a non-empty value); preserves a field nothing writes — YAGNI. |
-| Reuse rotation as the foundation | Rotation *prevents* coexistence by design (§0); it's scaffolding to bypass, not build on. |
-
----
-
-## 3. Layered change map (task split)
-
-| Layer | Change | Rough size |
-|---|---|---|
-| **T1 — Rust contact state** | multi-channel `EstablishedContact`; re-key the 3 maps to `(counterparty, accountRef)`; per-contact metadata; invert the sweep collapse to per-accountRef; user-gate additional accounts; populate `accepted_accounts` | large, the core |
-| **T2 — `accepted_accounts` round-trip** | `set_contact_metadata` copies it; publish reads it; (independently shippable as the data-layer floor of T1) | ~15-30 LOC |
-| **T3 — Changeset/persistence** | accountRef in `SentContactRequestKey`/`ReceivedContactRequestKey` + `established` map key; carry `accepted_accounts` | medium |
-| **T4 — FFI** | `account_index`/`accepted_accounts` on `ContactRequestFFI` + persist callbacks; +1 send param; pending-additional-account surface | medium |
-| **T5 — Swift/SwiftData** | accountRef in `PersistentDashpayContactRequest` unique key; per-account grouping in ContactsView/ContactRequestsView/ContactDetailView/AddContactView/SendDashPayPaymentSheet; "add account vs replace" prompt; send-from-account picker | large, UI-heavy |
-| **T6 — Tests** | unit (re-key, coexist, user-gate, accepted_accounts round-trip, rotation-still-supersedes-within-a-channel); `dp_*` e2e multi-account send/receive (devnet) | medium |
-
-The persistence (T3) + Swift (T5) layers need a **migration** for existing
-single-channel rows (map the lone channel to `accountReference` of its stored
-request).
-
----
-
-## 4. Failure modes & risks (for reviewers to stress)
-
-1. **Inverting the idempotency keystone (T1, highest risk).** `newest_received_per_sender`
- collapse and `apply_rotated_incoming_request` supersede are the mechanism that keeps
- the recurring sweep from thrashing. Splitting "collapse per sender" into "collapse
- per (sender, accountRef), keep all accepted refs" must not reintroduce the
- multi-doc sweep thrash that PR #3841 fixed (the `newest_received_per_sender`
- comment at `:752-765`). Needs the same red→green pinning as the original fix.
-2. **Rotation vs add ambiguity.** If the user picks "replace" we must supersede the
- *right* channel; if "add" we must not later mistake the rotation of an added channel
- for yet another new account. Channels keyed by raw `accountReference` make a
- *rotation within a channel* indistinguishable from a *new account* unless the
- version nibble is consulted — but the recipient ignores `accountReference`. Resolve:
- does "rotation of an added account" even occur, and how is it keyed?
-3. **Migration.** Existing persisted single-channel contacts (SQLite + SwiftData) must
- map to the new keyed shape without losing alias/note/hidden/broken state or
- double-counting payments.
-4. **The `accountReference == 0` collision.** Today everything is accountRef `0`-ish;
- re-keying must handle the legacy `0` channel and a genuinely-new `0`-masked account
- (collisions are possible — `accountReference` uniqueness isn't guaranteed, DIP-15 §7).
-5. **UI blow-up.** A contact rendering as N rows vs one row with N accounts; the send
- sheet picking an account; the "add vs replace" prompt. Scope creep risk.
-6. **No requirement = speculative surface.** Building this without a driving use case
- risks shipping inert complexity (Rule 2). The spec must end with a go/no-go.
-
----
-
-## 5. Verification plan
-
-- **T2 (unit):** `set_contact_metadata` preserves `accepted_accounts`; publish emits the
- contact's accepted set; round-trip through the codec. (TDD red→green.)
-- **T1 (unit):** an established contact accepts a second `accountReference` → two live
- channels; a rotation of channel 0 supersedes channel 0 only; the sweep does not thrash
- across two recurring passes (mirror the PR #3841 idempotency pin); an un-accepted
- additional request stays pending and is not watched.
-- **Migration (unit):** a persisted single-channel contact loads as a one-channel
- multi-account contact with metadata intact.
-- **Integration (`dp_*` e2e, devnet-gated):** send from a non-zero account; receive +
- accept a contact's second account; pay across both.
-
----
-
-## 6. Open questions for review (resolve before any coding)
-
-1. **Go/no-go:** is there an actual requirement for simultaneous multi-account, or does
- this stay deferred? (The spec's existence shouldn't force the build.)
-2. **Rotation-within-an-added-channel (§4.2):** does it occur in practice, and how is a
- channel keyed if not by raw `accountReference`? (Possibly `(accountReference &
- 0x0FFFFFFF)` ignoring the version nibble — but the recipient can't unmask… revisit.)
-3. **Metadata granularity:** confirm `alias/note/is_hidden` are per-contact (per person)
- and only `accepted_accounts` + `payment_channel_broken` are per-channel.
-4. **UI model:** one contact row with N accounts (recommended) vs N rows. Send sheet
- default account.
-5. **Could T2 (accepted_accounts non-lossy) ship now** as a tiny data-preservation fix
- ahead of the rest, or does shipping an inert field invite confusion? (Lean: ship with
- T1, not standalone.)
-6. **Migration safety** for existing devnet/testnet contacts.
diff --git a/docs/dashpay/PENDING_CONTACT_CRYPTO_RELOCATION_SPEC.md b/docs/dashpay/PENDING_CONTACT_CRYPTO_RELOCATION_SPEC.md
deleted file mode 100644
index 4ff5929ce53..00000000000
--- a/docs/dashpay/PENDING_CONTACT_CRYPTO_RELOCATION_SPEC.md
+++ /dev/null
@@ -1,188 +0,0 @@
-# Relocate the deferred-crypto queue from the wallet to the identity
-
-**Status:** implemented (historical reviewed spec). The in-memory relocation is
-complete; durable cold-load restoration remains deferred as described in §7.
-**Scope:** `packages/rs-platform-wallet` (+ a one-line doc-comment in `rs-platform-wallet-storage`).
-Rust-only. FFI signatures, Swift, and whole-struct serialization are **unchanged**.
-
-## 1. Problem
-
-`pending_contact_crypto: Vec` lives on the **wallet-level** struct
-`PlatformWalletInfo` (`wallet/platform_wallet.rs:57`), a sibling of `identity_manager`. Every
-*other* DashPay artifact already lives **per-identity** on `ManagedIdentity`
-(`state/managed_identity/mod.rs`): `established_contacts`, `sent_contact_requests`,
-`incoming_contact_requests`, `dashpay_rescan_triggered`, `auto_accept_verify_failed`,
-`dashpay_payments`. The queue is the lone exception, and each `PendingContactCrypto` entry carries
-`owner_identity_id` — manually re-storing the exact container key that is *implicit* for the
-others. Identity-network code (`IdentityWallet`) reaches *up* into wallet state to touch it.
-
-This is **cleanup, not a bug fix**. The queue is functionally correct where it is; the value is
-consistency/maintainability. The risk is a routing regression on a signer-gated DashPay path (a
-mis-routed drain → a contact account never gets built → a DashPay payment silently can't resolve
-its external account). So it is speced, reviewed, isolated (its own commit/PR), and tested.
-
-## 2. Current architecture (verified in research + review)
-
-- **Type** (`changeset/changeset.rs:1075`): `PendingContactCrypto { owner_identity_id, contact_id,
- op: PendingContactCryptoOp, enqueued_at_ms }`. Dedup key `PendingContactCryptoKey =
- (owner_identity_id, contact_id, kind)`; `upsert_pending_contact_crypto` keeps ≤1 entry per key.
-- **Receiver**: methods are on `IdentityWallet` (bound to a `wallet_id`,
- NOT one identity); reaches the queue via `wm.get_wallet_info(&self.wallet_id)`.
-- **`IdentityManager` has TWO buckets** (`state/manager/mod.rs:69-83`):
- `wallet_identities: BTreeMap>` and
- `out_of_wallet_identities: BTreeMap`. Today's flat wallet-level queue
- is **bucket-agnostic**. This is the crux of the refactor's one real trap — see §3 D4 / §5 R1.
-- **Enqueue** — exactly THREE production sites, each already holding `&mut PlatformWalletInfo` and
- the owner id: `enqueue_pending_auto_accepts` (`contact_requests.rs:1553`),
- `enqueue_deferred_contact_crypto` (`:1734`), `enqueue_contact_info_decrypt`
- (`contact_info.rs:385`). Each pairs the in-memory `upsert_pending_contact_crypto(&mut
- info.pending_contact_crypto, e)` with a changeset `pending_contact_crypto_added: vec![e]` and a
- `persister.store(...)`. (`payments.rs:2757` is a **test**, not a production enqueue.)
-- **Drain** (`drain_pending_contact_crypto`, `contact_requests.rs:1781`): read-lock → clone the flat
- queue → **drop lock** → async match over the owned snapshot (each arm routes every side-effect by
- `entry.owner_identity_id`; the loop body never touches the queue) → write-lock → single
- `retain_drained_by_snapshot(&mut info.pending_contact_crypto, &cleared)`.
- `drain_auto_accepts` (`:2160`) is the signer-gated sibling for `AutoAccept` ops; its removal block
- also marks `auto_accept_verify_failed` per owner.
-- **Count** (`pending_contact_crypto_count`, `:1763`): `count_account_build_ops` over the flat queue
- (excludes `ContactInfoDecrypt`). Backs the "waiting for unlock" UI banner.
-- **Op ownership asymmetry (subtle, load-bearing for R1):** `RegisterReceiving` /
- `RegisterExternal` are owned-only (`build_contact_accounts` gates on `identity_index.is_some()`,
- `contact_requests.rs:1661`); `ContactInfoDecrypt` is owned-only (`contact_info.rs` iterates only
- `wallet_identities`). But `AutoAccept` is **NOT** gated — `enqueue_pending_auto_accepts` runs for
- every identity in the sweep's `all_identities()` loop (both buckets), so an `AutoAccept` op can
- legitimately land on an **out-of-wallet** identity's queue.
-- **Changeset** (`PlatformWalletChangeSet.pending_contact_crypto_{added,cleared}`,
- `changeset.rs:1189`): flat top-level Vecs; entries carry the owner.
-- **Apply is a no-op for the queue** (`wallet/apply.rs:115`): the in-memory queue is mutated
- *directly* at the enqueue/drain sites; the changeset deltas are for persistence, not in-memory replay.
-- **The queue IS durably persisted** — via the `rs-platform-wallet-storage` SQLite backend: table
- `pending_contact_crypto` keyed `(wallet_id, owner_identity_id, contact_id, kind)`
- (`migrations/V001__initial.rs:87`), live writer `apply_pending_contact_crypto`
- (`sqlite/schema/pending_contact_crypto.rs:49`) driven from `apply_changeset_to_tx`
- (`sqlite/persister.rs:1063`), reader `all_pending_contact_crypto` (`:108`), round-trip test
- (`:161`). The **FFI/SwiftData** backend has no callback for it, so on iOS it is not durably
- persisted — but that is one backend, not "the field is vestigial." **The changeset fields are
- load-bearing; nothing here gets deleted.** Because the SQLite writer keys on
- `(wallet_id, owner_identity_id, contact_id, kind)`, moving the *in-memory* field per-identity
- changes **zero** SQLite writes (the owner stays on every row — D2/D5).
-- **Not restored on cold load** (`manager/load.rs:102-114`): starts `Vec::new()`; the sweep
- re-enqueues. A restore path is half-wired but blocked upstream — see R6.
-- **FFI** (`ffi/dashpay.rs:733, 798`): `platform_wallet_drain_pending_contact_crypto` /
- `_count` take a **wallet** handle and call `wallet.identity().()`. Called from Swift
- (`PlatformWalletManager.swift:640,968`). D4 keeps the `IdentityWallet` method signatures →
- **no FFI or Swift change**.
-- **Accessors** (`state/manager/accessors.rs`): `managed_identity(&Identifier)` /
- `managed_identity_mut(&Identifier)` (`:70,75`) already resolve across **both** buckets via
- `location_index`. Enumerators `all_identities() -> Vec<&Identity>` and `identity_ids() ->
- Vec` exist, but **there is no iterator yielding `&ManagedIdentity`** — one is added
- (D3).
-
-## 3. Design
-
-Move the in-memory Vec to `ManagedIdentity`, keyed by the owning identity. Keep
-persistence/apply/FFI shapes unchanged to bound the blast radius.
-
-- **D1 — Field placement.** Add `pending_contact_crypto: Vec` to
- `ManagedIdentity`; remove it from `PlatformWalletInfo`. Init `Vec::new()` in `ManagedIdentity::new`
- + `new_out_of_wallet` (next to `established_contacts`); drop the 5 `PlatformWalletInfo` init sites.
-- **D2 — Keep `PendingContactCrypto` unchanged (keep `owner_identity_id`).** It is the drain's
- routing key (each op's side-effects derive from it) AND the SQLite key column; keeping it holds the
- type, dedup key, changeset, and their tests stable. The in-memory redundancy (owner == container)
- is benign. *Dropping it is out of scope* (§7).
-- **D3 — Access by identity.**
- - Enqueue + per-owner removal: existing `managed_identity_mut(&owner)` (spans both buckets).
- - Drain-snapshot + count: **add** `IdentityManager::managed_identities(&self) -> impl Iterator- ` chaining `out_of_wallet_identities.values()` with
- `wallet_identities.values().flat_map(|m| m.values())`. Iterate **both buckets** (R1).
-- **D4 — Drain/count: flat snapshot → unchanged async loop → per-owner-grouped removal. Both
- buckets. Same signatures, same wallet-wide semantics.** Concretely (do NOT write an outer
- per-identity loop — it borrows `&mut ManagedIdentity`/holds the lock across `.await` and will not
- compile):
- 1. **Snapshot:** under a read guard, gather every resident identity's queue into one flat owned
- `Vec` (`managed_identities().flat_map(|m| m.pending_contact_crypto.iter()
- .cloned())`), then drop the guard. `count` sums `count_account_build_ops` per identity the same
- way.
- 2. **Async loop:** unchanged — it already keys every lookup/side-effect off
- `entry.owner_identity_id` and touches the queue nowhere.
- 3. **Removal:** under a write guard, group `cleared_snapshots` by `owner_identity_id` and, per
- owner, `retain_drained_by_snapshot(&mut managed_identity_mut(&owner).pending_contact_crypto,
- &subset)`. Fully synchronous under one guard — nothing crosses `.await`.
- `retain_drained_by_snapshot`'s value-equality (which includes the owner) transfers unchanged.
- For `drain_auto_accepts`, the same per-owner hop also carries the `auto_accept_verify_failed`
- mark (already per-owner today).
- - *Send-drain scope (Q1 resolved → wallet-wide):* keep `payments.rs:575`
- `self.drain_pending_contact_crypto` draining every resident identity, not just the sender. It is
- safe and useful — the Keychain provider is wallet-**seed**-scoped, so one identity's send
- correctly finishes other identities' pending builds as a free, correct side effect; no
- cross-identity dependency exists (accounts are keyed by both ids). Narrowing to sender-only is a
- one-line snapshot filter with only a mild latency/UX argument — deferred.
-- **D5 — Changeset + apply + FFI unchanged.** Keep `pending_contact_crypto_{added,cleared}` flat
- (entries carry owner), keep `apply.rs` ignoring them, keep the FFI signatures. Only the *in-memory*
- field + its access sites move. The SQLite writer is unaffected (owner-keyed).
-- **D6 — `ManagedIdentity` field is persistence-inert. Do NOT add it to `IdentityEntry::from_managed`**
- (`changeset/changeset.rs:332`), which explicitly enumerates the persisted per-identity fields.
- Leaving it out keeps the queue in-memory-only per identity (like `established_contacts` /
- `dashpay_rescan_triggered`), so it is NOT double-persisted (once via the flat changeset delta,
- never via a snapshot). This preserves D5.
-
-## 4. Alternatives rejected
-
-- **Per-identity changeset routing:** unnecessary — apply ignores the queue deltas and the SQLite
- writer already keys by owner. Adds churn + a migration question for zero benefit.
-- **Drop `owner_identity_id`:** forces the changeset/SQLite key to carry the owner another way and
- rewrites the dedup key + tests. Higher risk, separable, deferred (§7).
-- **Move only to `IdentityManager`:** leaves the queue one flat list one struct deeper — still not
- keyed by identity. Doesn't achieve the goal.
-- **Defer / TODO:** legitimate (a reviewer's call, given this path just absorbed the scalar
- elimination). Decision: proceed now as an isolated, tested, reviewed change so it's bisectable.
-
-## 5. Failure modes & risk register
-
-| ID | Risk | Mitigation |
-|----|------|------------|
-| R1 | **[Critical]** Drain/count iterate only the owned bucket → an `AutoAccept` op on an *out-of-wallet* identity is silently never drained/counted (auto-accept never fires; banner under-counts). This reproduces the exact silent signer-gated regression this refactor fears. | D3/D4 iterate **both** buckets (`managed_identities()`). Test: an out-of-wallet identity holding an `AutoAccept` entry is still counted and drained. |
-| R2 | Drain restructure holds a `&mut ManagedIdentity` or the wallet-manager guard across `.await` → won't compile / deadlock (the register fns re-acquire the non-reentrant manager lock). | D4: flat owned snapshot → drop guard → async loop → re-lock → synchronous per-owner removal. Never a live `values_mut()` borrow across `.await`; snapshot ids/entries into owned Vecs, re-lookup per owner (mirrors the sweep). |
-| R3 | Enqueue routes to a wrong/absent identity. | Owner is already in hand at all 3 sites; add `managed_identity_mut(owner)` before upsert. `None` (identity removed in the narrow collect-guard→write-guard window) → log + drop (benign; the identity is gone). Test: enqueue lands on the owner's queue and nowhere else. |
-| R4 | `count`/`drain` totals drift (miss an identity). | Same signatures + wallet-wide semantics over both buckets; test with 2 identities each holding entries asserts the aggregate equals the sum. |
-| R5 | Auto-accept verify-failure marking regresses. | Marking is already per-owner (`managed_identity_mut(owner).mark_...`); folds into the same removal hop. Keep its test. |
-| R6 | Future cold-load restore drops entries (a persisted row whose owner identity isn't applied yet). | The move introduces an ordering constraint the flat queue didn't have: any future restore must apply identities **before** fanning each persisted row out to its owner's queue. Documented here + in the storage doc-comment so whoever finishes the (currently blocked) restore doesn't reintroduce the drop. Not active today (nothing restores). |
-| R7 | Identity removal now GC's its queue (dies with the `ManagedIdentity`). | Behavior change vs the flat wallet Vec (entries used to outlive owner residence). **Accepted** — it's orphan cleanup; a transient remove/re-add loses queued ops that the sweep re-enqueues on re-add. Noted, no code needed. |
-| R8 | Identity removed between drain snapshot and removal → its keys aren't retained-off/cleared. | Net-identical to today: the op's target is gone and `apply` ignores the `cleared` delta anyway. Noted. |
-
-## 6. Change list (critical files)
-
-- `wallet/platform_wallet.rs` — remove the field + doc.
-- `wallet/identity/state/managed_identity/mod.rs` — add the field + doc.
-- `wallet/identity/state/managed_identity/identity_ops.rs` — init in `new` + `new_out_of_wallet`; **do not** touch `from_managed`.
-- `wallet/identity/state/manager/accessors.rs` — add `managed_identities()` iterator (both buckets).
-- `wallet/identity/network/contact_requests.rs` — 2 enqueues (`:1553`, `:1734`); `drain_pending_contact_crypto` (snapshot both buckets, per-owner removal); `pending_contact_crypto_count` (sum both buckets); `drain_auto_accepts`; `empty_info` test helper (`:3254`); the drain/count/auto-accept tests.
-- `wallet/identity/network/contact_info.rs` — enqueue (`:385`).
-- `wallet/identity/network/payments.rs` — send drain unchanged (`:575`); **re-seed** the drain tests (`:2757`, `:2811`) with real registered identities (out-of-wallet for the `identity_index==None` case).
-- `manager/load.rs:114`, `manager/wallet_lifecycle.rs:249`, `wallet/apply.rs:420`, `wallet/platform_wallet_traits.rs:43,56` — drop the `PlatformWalletInfo` init sites.
-- `rs-platform-wallet-storage/src/sqlite/schema/pending_contact_crypto.rs:100-106` — update the doc-comment that names `PlatformWalletInfo.pending_contact_crypto` as the restore target (now per-identity fan-out by `owner_identity_id`; see R6).
-- `ffi/dashpay.rs` — unchanged; verify it still compiles.
-
-## 7. Explicitly out of scope
-
-- Dropping `owner_identity_id` from `PendingContactCrypto`.
-- Deleting/altering the `pending_contact_crypto_{added,cleared}` changeset fields — they are
- persisted by the SQLite backend (§2). NOT vestigial.
-- Wiring the cold-load restore (blocked upstream); this change only leaves it a correct ordering note.
-- Any Swift / FFI-signature change.
-
-## 8. Test / verification plan
-
-- **R1 (the one that matters):** an out-of-wallet identity holding an `AutoAccept` entry is counted
- by `pending_contact_crypto_count` and processed by the drain — asserts both buckets are iterated.
-- **R3:** enqueue lands on the owner identity's queue and no other identity's.
-- **R4:** 2 resident identities each holding queue entries → aggregate count + drain == sum.
-- **R2:** re-uses `retain_drained_by_snapshot`'s existing value-equality test shape, per-owner.
-- **R5:** auto-accept verify-failure marks the right identity.
-- Cold-load: a freshly-loaded identity has an empty queue; the sweep re-enqueues (behavior unchanged).
-- Keep green (re-seeded where noted): `send_payment_runs_pending_contact_crypto_drain`,
- `drain_completes_register_receiving_and_clears_queue`,
- `drain_leaves_register_external_it_cannot_complete`,
- `account_build_count_excludes_contact_info_decrypt`, the changeset merge/dedup tests.
-- `cargo test -p platform-wallet -p platform-wallet-ffi`; `cargo clippy … --all-targets` clean;
- `build_ios.sh --target sim` BUILD SUCCEEDED (FFI unchanged → Swift unaffected).
diff --git a/docs/dashpay/QA_TESTCASES_SPEC.md b/docs/dashpay/QA_TESTCASES_SPEC.md
deleted file mode 100644
index 55d0456ee12..00000000000
--- a/docs/dashpay/QA_TESTCASES_SPEC.md
+++ /dev/null
@@ -1,225 +0,0 @@
-# DashPay (DIP-15 / DIP-16) QA test-case expansion — SPEC
-
-**Status:** reviewed (4-lens multi-agent pass folded in)
-**Target file:** `packages/swift-sdk/SwiftExampleApp/TEST_PLAN.md` §4.10 (+ §5, §6, §1)
-**Base:** branched off `feat/dashpay-m1-sync-correctness` (PR #3841,
-"fix(platform-wallet)!: complete dashpay") — the DashPay views, `docs/dashpay/`,
-and the features these rows describe live there, not yet in `v3.1-dev`.
-**PR target:** `v3.1-dev`, to merge **after** #3841 lands (pure-docs diff;
-rebased so it shows only the TEST_PLAN.md / spec changes).
-**Renders in:** [`dashpay/qa-dashboard-site`](https://github.com/dashpay/qa-dashboard-site)
-once the sibling seed task re-seeds the `dash-qa` contract from the updated plan.
-
----
-
-## 1. Problem
-
-`TEST_PLAN.md` §4.10 (DashPay) had **6 coarse rows** (`DP-01..06` + cross-ref
-`MW-03`) at "feature exists" granularity, and their entry points were **stale**:
-they cited `FriendsView` / `AddFriendView`, which #3841 replaced with a dedicated
-DashPay tab (`DashPayTabView`, `AddContactView`, `ContactsView`,
-`ContactRequestsView`, `ContactDetailView`, `DashPayProfileView`,
-`IgnoredContactsView`, `SendDashPayPaymentSheet`).
-
-#3841 implements substantial **DIP-15** surface the catalog did not exercise
-(per the branch's audit `docs/dashpay/DIP_CONFORMANCE_GAPS.md`):
-`encryptedAccountLabel` send+receive, QR auto-accept (build + paste-to-add),
-on-chain `contactInfo` publish, and the §12.6 incoming-payment backfill rescan.
-**DIP-16** is the SPV sync layer underneath (covered by `CORE-07`; its
-DashPay-specific facet is the single backfill row `DP-10`).
-
-## 2. Goal & non-goals
-
-**Goal:** correct the stale `DP-01..06` entry points and add rows for the
-**user-observable, simulator-drivable** DIP-15/16 DashPay flows #3841 ships.
-
-**Non-goals (out of scope):**
-- **DIP-15 crypto internals** (ECDH, 69-byte compact xpub, `accountReference`
- masking, avatar hash/dHash) — already Rust known-answer tests; not app rows.
-- **Gap / absence rows** (`🚫`/`➖`) for unimplemented features (multi-account
- `Account≠0`, `acceptedAccounts` flood mitigation, invitations). Drivable only.
-- **A DIP-16 section.** SPV sync is `CORE-07`; the DashPay facet is `DP-10`.
-- **New table columns.** Keep the uniform 6-col `ID | Action | Layer | Tier |
- Status | Entry point & test notes`; cite DIP §s inline. (The dashboard
- normaliser reads no section field; a 7th column is dropped on seed.)
-- **A `tags` column.** Tag assignment for the v5 contract is the seed tool's job
- (not in these repos). Open question §7.
-
-## 3. Corrections to existing rows (`DP-01..06`)
-
-Entry points updated to the #3841 DashPay tab; FFI-symbol naming (matches the
-existing §4.10 convention). Merged sub-flows folded in as notes:
-- **DPNS-add path** → a note on `DP-01` (precedent: `ID-04`/`MW-01` list input
- methods in one row; DPNS resolution itself is `DPNS-03`/`DPNS-07`).
-- **Payment-channel-broken** state → a note on `DP-03` (precedent: `ID-12`/`DOC-07`
- attach gating state to the action row).
-- **Avatar** (url + Rust-computed hash/fingerprint) → a note on `DP-04`.
-- `DP-06` **Reject → Ignore** rename (the branch made reject a reversible local mute).
-
-## 4. New rows (drivable DIP-15/16 flows)
-
-| ID | Tier | DIP-15 § | Behavior |
-|---|---|---|---|
-| DP-07 | Common | §8.5 | Attach `encryptedAccountLabel` on send; counterparty sees "Their account" (decrypted, incoming-row only). |
-| DP-08 | Thorough | §8.13 | QR auto-accept: build "Add me" QR; add via pasted URI → auto-accepted without manual accept. Paste-drivable; camera = Manual variant. |
-| DP-09 | Thorough | §10 | Publish encrypted on-chain `contactInfo`; ≥2-contact gate → `.published` / `.deferredUntilTwoContacts` / `.skippedWatchOnly`. |
-| DP-10 | Manual | §8.7/§12.6 | Incoming-payment backfill rescan (no UI trigger; `reconcile_dashpay_rescan` rewinds SPV `synced_height`). Env-limited; the §12.6 payment-loss regression pin. |
-
-`DP-10` note: the branch's `DIP_CONFORMANCE_GAPS.md` §1.1 still marks this MISSING,
-but that audit predates the implementing commit `18483e4232`
-(`reconcile_dashpay_rescan`, wired in `manager/dashpay_sync.rs`, 4 unit tests) —
-so Status=✅ is correct.
-
-## 5. Cross-cutting edits (applied)
-
-- **§6 index** — DashPay: `DP-01..06, MW-03` → `DP-01..10, MW-03`.
-- **§5 by-tier** — Common `31→32`, Thorough `35→37`, Manual `1→2` (`CORE-08, DP-10`).
-- **§5 by-layer (automatable; Manual EXCLUDED)** — Platform `~72→~75`; **Cross
- unchanged** (DP-10 is Manual).
-- **§1 worked example** — "list the manual tests" → `CORE-08, DP-10`.
-
-## 6. Final row set
-
-6 corrections (`DP-01..06`) **+ 4 new** (`DP-07` account label, `DP-08` QR
-auto-accept, `DP-09` on-chain `contactInfo`, `DP-10` backfill rescan).
-
-## 7. Open questions
-
-1. **Tags** — does the seed tool assign v5 tags (e.g. `dip15`, `sync`) from
- Domain/§6, or should the plan encode them? Needs the seed tool (not in repos).
-2. **DP-07 a11y id** — the "Their account" block in `ContactDetailView` has no
- `accessibilityIdentifier`, so DP-07 asserts on visible text. A 1-line app
- change would make it cleanly automatable — a trivial follow-up, deliberately
- kept out of this docs-only PR.
-
-## 8. Verification plan
-
-1. **Render check** — IDs match `^DP-\d+$`; tier/category present so the dashboard
- matrix charts them (the normaliser only hard-requires `testId`).
-2. **Drive each new row** with the `simulator-control` skill on a booted sim; two
- on-device wallets where a counterparty is needed (`DP-07`/`DP-08`, cf. `MW-03`).
- Verify against **persisted SwiftData state**, not UI alone (§1 pass criteria).
- `DP-10` is Manual → skip-and-flag in automation.
-3. **No code change** — pure TEST_PLAN.md edit. The seed task re-seeds `dash-qa`;
- the dashboard renders.
-
-## 9. Review provenance
-
-Four independent review lenses (DIP domain-fit, scope/simplicity, automatability/
-entry-point accuracy, catalog conventions) ran against the draft. Key folds:
-- Added `DP-09` (on-chain `contactInfo`) — the draft wrongly excluded it as
- "local-only / no UI"; it is a drivable DIP-15 §10 publish.
-- Merged the draft's separate DPNS / avatar / channel-broken rows into notes on
- `DP-01` / `DP-04` / `DP-03` (catalog precedent; lean set).
-- Dropped a 7th `DIP-15 §` column (normaliser ignores it; breaks the 6-col shape).
-- Confirmed `DP-10`'s rescan is implemented + wired; corrected the `wallet.pass`
- SF-symbol-vs-a11y-id confusion in `DP-07`.
-
-## 10. Runtime verification (simulator)
-
-Driven on a booted iOS simulator (iPhone 17) against a live devnet build with real
-DashPay fixtures (wallet "SimB", 1 identity, 2 contacts, 5 requests), via the
-`simulator-control` skill. Read-only structural pass — navigated to each row's
-entry point and confirmed the cited screens/controls exist; **no broadcasts fired**.
-
-Confirmed live:
-- `DP-01` — `AddContactView` mode picker + resolved-recipient preview + **Send Request**.
-- `DP-03` — `ContactDetailView` `dashpay.detail.sendDash`.
-- `DP-04` — `DashPayProfileView` **Edit** (→ editor) + avatar.
-- `DP-05` — DashPay tab: `ContactsView` (search, contacts, segment, profile header).
-- `DP-07` (send) — `dashpay.addContact.accountLabel` renders once a recipient resolves.
-- `DP-08` — build: `dashpay.profile.qrURI` emits a real `dash:?du=…&dapk=…` URI + QR
- image; add: `AddViaQRSheet` `dashpay.qr.uriField`.
-- `DP-09` — the **Alias / Note / Hide** editor calls `saveContactInfo` →
- `setDashPayContactInfo`; the in-app footer confirms the ≥2-contact encrypted-publish
- gate. **Refined the row** accordingly — the original "distinct from a local note"
- wording was wrong (the same editor caches locally *and* publishes on-chain).
-
-Code-confirmed but not rendered this pass (no fixture): `DP-02` / `DP-06`
-(`dashpay.request.accept` / `.ignore` — need an *incoming* pending request);
-`DP-07` receive-side "Their account" (only shows when a contact sent a label);
-`DP-10` (no UI by design — automatic in DashPay sync). Live broadcast execution and
-the two-wallet loops (`DP-07`/`DP-08`) are the next step, gated on credits + a
-counterparty identity.
-
-A full code re-audit of every row (4 parallel passes) confirmed 8/10 rows + all the
-§5/§6/§1 count edits accurate, and corrected 5 row-wording inaccuracies: DP-01 (open
-button id `dashpay.addContact` vs the in-sheet mode toggle), DP-02/DP-05
-(`EstablishedContact` is a Rust/FFI handle, **not** a SwiftData model — the tab views
-are backed by `PersistentDashpayContactRequest`), DP-03 (channel-broken is any
-permanent channel failure, not only key rotation), and DP-08 (TTL is exactly 3600s).
-
-## 11. Implementation observations (for #3841 — surfaced during verification, NOT addressed here)
-
-These are defects/smells in the DashPay *implementation* found while auditing the
-plan. They are out of scope for this docs PR; recorded for the #3841 author.
-
-1. **Stale doc-comment** — `ContactRequestsView.swift:5-8` says incoming rows carry
- "**Accept / Reject**", but the button is **Ignore** (reject was replaced by the
- reversible local mute). Same file `:35-37` carries an internal `§6.4` spec-gate
- ref (rots; against the timeless-comment convention).
-2. **Multi-wallet mis-attribution risk** — `DashPayProfileEditorView` falls back to
- `walletManager.firstWallet` when `walletId` is nil (`IdentityDetailView.swift:1316`);
- in a multi-wallet setup a profile update could submit under the wrong identity.
- Already acknowledged in an in-code comment as needing tightening.
-3. **Handle-leak smell** — `acceptContactRequest`'s returned `EstablishedContact`
- (FFI handle wrapper) is discarded with `_ =` at both call sites
- (`ContactRequestsView.swift:228`, `AddContactView.swift:487`); leaks per accept
- unless the wrapper frees the handle in `deinit` (worth confirming a `deinit`).
-4. **QR clock edge** — `build_auto_accept_qr` derives expiry from
- `SystemTime::now()…unwrap_or(0)`; a pre-1970 / badly-skewed clock yields an
- already-expired QR. Harmless on a real device.
-5. **No collision handling in `AddViaQRSheet`** — pasting a URI from someone who
- already sent *you* a request broadcasts a duplicate outgoing request rather than
- offering "Accept instead" (`AddContactView` handles this; the QR path does not).
-
-By-design / cosmetic (no action expected): avatar hash does not change if the image
-bytes are swapped behind the same URL; a corrupt/hostile incoming account label
-unpads to garbage and is coerced to `None` (shows no "Their account" — relevant to
-DP-07 negative testing); `setDashPayContactInfo` maps unknown future outcome bytes to
-`.published` on the Swift side; a stale memo doc-comment in `SendDashPayPaymentSheet`
-(DashPay payments always pass `memo: nil`); a dead `_ = bytes` local + a redundant
-`?? nil` duplicated across four profile-cache reads.
-
-## 12. Live end-to-end run (freshly-built binary)
-
-Built `build_ios.sh --target sim` from `feat/dashpay-m1-sync-correctness` HEAD
-(`47d9044b5a`), installed on two iOS simulators, and drove the flows on-chain
-against devnet (two funded identities per side): **Eve** (SimB) ↔ **Alice / Bob /
-Dolly(7A8E)** (SimA), each ~25–30B credits. Verified against SwiftData ground
-truth (and on-chain for the payment).
-
-| Row | Result (fresh build) | Evidence |
-|---|---|---|
-| DP-01 send | ✅ | labeled contact request broadcast (sheet dismissed, no error); also the DP-02 reciprocal send |
-| DP-02 accept | ✅ | 7A8E accepted Eve → reciprocal `7A8E→Eve` row created (established) |
-| DP-03 payment | ✅ one direction | Eve→Alice **0.001 DASH** real L1 tx (input spent, change `74,899,477`, fee `226` duffs, txid `850433507c88…560e`) — **after starting Core SPV**. ⚠️ only the forward direction was driven; see the bidirectional gap below |
-| DP-04 profile | ✅ | publicMessage updated on-chain → SwiftData (`QA fresh-build 16:10`) |
-| DP-05 view | ✅ | contacts / requests / profile rendered throughout |
-| DP-06 ignore | ✅ | registered a fresh identity (asset-lock funded, ChainLock proof) → sent Eve a request → Eve **ignored** it (→ ignored-senders) → **un-ignored** (reversed). Local-only mute |
-| DP-07 label | ✅ fresh first-contact | Bob→**EveN** (fresh pair) labeled send; EveN accepted → decrypted "Their account" = the sent label. Confirms decrypt-on-accept end-to-end |
-| DP-08 QR | ✅ fresh first-contact | Alice built `dash:?du=…&dapk=…`; **EveN** pasted + `sendContactRequestFromQR`; Alice **auto-accepted** (reciprocal, no manual Accept) — *after unlocking Alice's wallet* (signer-backed drain; see note) |
-| DP-09 contactInfo | ✅ on-chain | log: `Published contactInfo document identity=Eve contact=Alice` — the `.published` outcome, not just local persist |
-| DP-10 backfill rescan | ✅ mechanism (logs) | the §12.6 rescan fired live: `DashPay rescan: lowered SPV synced_height … floor=51112` → `dash_spv…filters: synced_height 51112 fell below committed_height 52175, restarting scan`. No UI trigger (Manual tier); the full restore-from-seed payment-recovery remains a device exercise |
-
-**10/10 flows verified live on the fresh build** — DP-01..09 driven on-chain
-(SwiftData + chain; DP-07/DP-08 via a freshly-registered unconnected identity to
-get clean first-contact pairs), DP-09's on-chain publish + DP-10's backfill-rescan
-both confirmed in the Rust logs.
-
-**Gap (DP-03 bidirectional):** only the **forward** payment (Eve→Alice) was driven.
-The reverse (Alice→Eve) is symmetric by design — once established, each party derives
-the other's payment address from the exchanged xpubs — but it was **not** verified
-live (SimA's app context had flipped to a separate testnet wallet set). DP-03 now
-explicitly requires verifying **both** directions; the reverse remains to be driven.
-
-**Finding (DP-08):** the QR auto-accept *reciprocal* is signer-backed, so it only
-fires once the recipient's wallet is **unlocked** (the "N contacts waiting to finish
-setup → Unlock" drain). The request and auto-accept proof reach the recipient
-immediately, but the established reciprocal lands after unlock — so "auto-accept" is
-not fully hands-off. Worth surfacing in DIP-15 §8.13 expectations.
-
-Two plan corrections came out of the run: **DP-03** now records the Core-SPV
-precondition (a DashPay payment is an L1 broadcast — fails "SPV Client not started"
-if SPV is stopped); **DP-07** now states the label decrypts **on accept**, not on
-ingest.
diff --git a/docs/dashpay/QR_AUTO_ACCEPT_SPEC.md b/docs/dashpay/QR_AUTO_ACCEPT_SPEC.md
deleted file mode 100644
index 23b84e11764..00000000000
--- a/docs/dashpay/QR_AUTO_ACCEPT_SPEC.md
+++ /dev/null
@@ -1,270 +0,0 @@
-# DashPay QR Auto-Accept (DIP-15) — Implementation Spec
-
-Decision (2026-06-24): build the DIP-15 `autoAcceptProof` QR flow, **faithful to the
-DIP-15 wire formats** so we are a correct reference implementation. Research (incl. the
-finding that no reference client implements this today, so it is iOS-first / convention-
-setting) informed this spec. Invitations (DIP-13) are queued next.
-
-> **Status:** IMPLEMENTED (2026-06-24) across Rust + FFI + Swift; `build_ios.sh` green,
-> platform-wallet 299 + ffi 117 tests green. REVIEWED (4-lens: DIP-fidelity / security /
-> feasibility / scope) and revised — §10. The first draft's §4 was materially wrong
-> (verify can't use `&Wallet` in the seedless drain; the drain lacks the identity signer;
-> the sweep parser drops the proof) — all fixed. **Owner decisions:** TTL = 1h fixed;
-> auto-accept = always automatic; whole feature in one pass; DIP-literal HD-derived owner
-> key (scoped raw-key export). **On-device:** My-QR UI + DPNS-name guard verified; the full
-> QR-generate→scan→auto-accept loop is pending a DPNS-named *local* identity (the available
-> devnet wallets have on-chain names not cached in `PersistentIdentity.dpnsName`). Follow-up
-> (P3): resolve the owner's DPNS name on-chain in `build_auto_accept_qr` when the local
-> field is empty.
-
-## 1. Problem & goal
-
-DashPay contact establishment is two manual taps. DIP-15 defines an optional
-`autoAcceptProof` so a party can pre-authorize automatic acceptance — the canonical use
-case is a **merchant / in-person QR**: show a QR, the scanner sends a contact request that
-the owner's client auto-accepts with no manual tap. The proof crypto exists and is
-unit-tested (`auto_accept.rs`) but is **dormant** — nothing generates, verifies, or acts
-on it. Goal: wire the full three-role flow, end to end (Rust + FFI + Swift + on-device).
-
-### Non-goals
-- Not Invitations (DIP-13 `dashpay://invite` + AssetLock onboarding) — separate, queued next.
-- No Android interop today (no reference client verifies the proof); iOS-first. We still
- follow DIP-15 byte layouts so a future client can interop.
-- No new on-chain artifact beyond the already-defined optional `autoAcceptProof` field.
-- No `di=` identity-id URI fallback in v1 (DIP uses `du`; require a DPNS name — §9).
-- No TTL picker, no opt-in toggle (always automatic) in v1 (§9).
-
-## 2. The DIP-15 model — three roles
-
-1. **Owner (QR shower, "Bob").** Derives an auto-accept key at `m/9'/5'/16'/expiry'`,
- embeds the **private key + expiry** in a QR (`dash:?du=&dapk=`),
- shows it. (`expiry = now + 1h`.)
-2. **Scanner ("Carol").** Scans, resolves `du`→Bob's identity, decodes `dapk`→(private key,
- expiry), derives her friendship `accountReference` to Bob, **signs `Carol.$ownerId ‖
- Bob.toUserId ‖ accountReference` with the handed key**, and sends a contactRequest to
- Bob carrying that proof.
-3. **Owner receives + auto-accepts.** Bob's client (at a signer-present drain) verifies the
- proof against **his own** re-derived auto-accept **public** key and, if valid and
- unexpired, **auto-accepts** (sends the reciprocal) with no manual tap.
-
-Why the scanner signs (not the owner): the signed message includes the scanner's
-`$ownerId`, unknown at QR-create time — so the owner delegates signing via the (expiry-
-bounded) private key. This per-sender binding means a leaked proof can't be replayed by a
-*different* sender.
-
-## 3. DIP-15 wire formats — normative-for-us
-
-These are wire-faithful to DIP-15 (fidelity review: byte-for-byte match). Where DIP-15 is
-silent, the value below is **normative for our implementation** — a future interop client
-MUST match it or verification silently fails.
-
-**Auto-accept key blob** (`dapk` value), 38 bytes for ECDSA:
-
-| field | size | value |
-|---|---|---|
-| key type | 1 | `0x00` (ECDSA_SECP256K1) |
-| timestamp/expiry (= derivation index) | 4 | u32, **big-endian** *(DIP-silent → normative)* |
-| key size | 1 | `0x20` (32) |
-| key | 32 | secp256k1 **private** key |
-
-**Proof blob** (`autoAcceptProof` field), 70 bytes for ECDSA, 38–102 range:
-
-| field | size | value |
-|---|---|---|
-| key type | 1 | `0x00` |
-| key index (= expiry, same value as the blob) | 4 | u32, **big-endian** |
-| signature size | 1 | `0x40` (64) |
-| signature | 64 | compact ECDSA |
-
-**Signed message** *(DIP names the fields; hashing/encoding DIP-silent → normative)*:
-`SHA256($ownerId(32) ‖ toUserId(32) ‖ accountReference(4, little-endian))`, where
-`$ownerId` = the contactRequest **sender (scanner)**, `toUserId` = the QR **owner**, and
-`accountReference` is the **raw masked u32** the contactRequest carries (`version<<28 |
-masked_index`). Matches the existing `auto_accept.rs::build_message_hash`. **Security pin
-(§6):** the verifier MUST bind `$ownerId` to the **consensus-authenticated document owner
-id** (`doc.owner_id()`), never a self-reported field.
-
-**Derivation path**: `m / 9' / 5'(mainnet, else 1') / 16' / expiry'`, all hardened; `expiry`
-≤ 2^31−1 (hardened-index bound, ~year 2038 — reject at encode time). Matches code.
-
-**URI**: `dash:?du=&dapk=` (contact-only). Matches the
-DIP-15 example. No `di=` fallback in v1.
-
-## 4. Seedless integration (the corrected crux)
-
-Our wallets are `ExternalSignable` — no seed in Rust; key material is reachable only via
-the Keychain resolver/provider. The background sweep is **signerless**. Both verify
-(needs the owner's auto-accept key) and auto-accept (sends a signed state transition) need
-key material, so **neither runs in the sweep** — they ride the deferred-crypto queue +
-the signer-present drain. The first draft got the mechanics wrong; corrected:
-
-### 4.1 Sweep (signerless) — read the proof, enqueue, bounded
-- **FIX (feasibility #2):** `parse_contact_request_doc` must read
- `props.get("autoAcceptProof")` into the parsed `ContactRequest` (today it's hard-coded
- `None`, so the proof is dropped before the queue). Mirror the outgoing reader.
-- After `add_incoming_contact_request`, if the request carries an `autoAcceptProof` that
- passes a cheap **structural pre-check** (length 38–102, key-type `0x00`), enqueue
- `PendingContactCryptoOp::AutoAccept { sender_id }` (dedup key `(owner, sender, AutoAccept)`).
-- **DoS bound (security #4):** cap queued `AutoAccept` ops per owner (constant, e.g. 64);
- beyond the cap, skip enqueue (the request is still manually acceptable — nothing lost).
- Log the drop (no silent cap).
-
-### 4.2 Drain (signer present) — needs BOTH signers
-- **FIX (feasibility #3 / scope M1):** the drain needs the identity `Signer`
- (to send the reciprocal) **and** the `ContactCryptoProvider`. Thread a `signer` into
- `drain_pending_contact_crypto` and add a `signer_handle` to the drain FFI (matching the
- send/accept FFIs). Existing arms ignore it (additive bound). **Note:** the drain FFI is
- the same one `unlockWalletFromKeychain` calls (needs-unlock work) — that call site now
- passes the Swift `KeychainSigner` too.
-- Per `AutoAccept` entry, in order:
- 1. **Local verify FIRST, before any network fetch** (security #4 — anti-DoS): build the
- path `m/9'/coin'/16'/expiry'` (expiry from the proof header), derive the owner's
- auto-accept **public** key via `provider.receiving_xpub(&path).public_key` (**FIX
- feasibility #4** — verify needs only the pubkey; no `&Wallet`), then
- `verify_auto_accept_proof_with_pubkey(pubkey, proof, sender_id = request.sender_id
- (= doc.owner_id), recipient_id = self_identity, account_ref = request.account_reference)`.
- 2. **Expiry check** against the **same** timestamp that keyed verification
- (`now > expiry → reject`).
- 3. If valid + unexpired → `accept_contact_request_with_external_signer(request, signer,
- provider)` (sends the reciprocal; idempotent — adopts if already reciprocated).
-- **Verdict mapping (security #3):** invalid signature / wrong params / expired /
- out-of-range index (the `Err` from path derivation) ⇒ **permanent: clear the entry**
- (the request falls back to a normal manual-acceptable pending request). Signer/network
- unavailable ⇒ **transient: leave queued** for the next drain. Never `mark_channel_broken`
- (there's no channel yet).
-
-Consequence: auto-accept completes at the owner's next signer-present moment (unlock or any
-DashPay action), not instantly in the background. Consistent with the seedless model.
-
-## 5. Interface / data flow per layer
-
-### 5.1 Rust — `auto_accept.rs` (extend; keep existing tested fns)
-- KEEP `derive_auto_accept_private_key(wallet, network, expiry)` (owner, QR-create).
-- ADD `encode_auto_accept_key_blob(secret_key, expiry) -> Vec` /
- `decode_auto_accept_key_blob(&[u8]) -> Result<(SecretKey, u32)>` (38-byte `dapk`).
-- ADD `sign_auto_accept_proof(secret_key, scanner_id, owner_id, account_ref, expiry) -> Vec`
- — scanner signs with the **handed** key. Message bytes = `scanner_id ‖ owner_id ‖
- account_ref(LE)` (the existing `build_message_hash`); a doc-comment ties the param names
- to DIP roles (**scope M2** — the current `generate` models the owner as `sender_id`,
- the opposite; don't invert at wiring).
-- ADD `verify_auto_accept_proof_with_pubkey(pubkey, proof, scanner_id, owner_id, account_ref) -> bool`
- — pure, no wallet (the drain's verify path).
-- ADD `auto_accept_proof_expiry(proof) -> Option` and fold the expiry check into the
- acceptance entry point — **do not** expose a public bare `verify` that returns `true`
- for an expired proof (**security #2** foot-gun). Keep `verify_auto_accept_proof(wallet,…)`
- for owner-side tests only.
-- ADD a URI codec `encode_dashpay_contact_uri(username, key_blob)` /
- `parse_dashpay_contact_uri(&str) -> Result<(username, key_blob)>` (pure, testable).
-- Refactor `generate_auto_accept_proof` to `derive + sign` (test/convenience).
-- Remove the stale `// TODO: Where and how we use these helpers?` and fix the docstring
- that references a now-real `verify_auto_accept_proof_with_pubkey` (**scope N3**).
-
-### 5.2 Rust — changeset.rs + contact_requests.rs (flow)
-- `PendingContactCryptoOp::AutoAccept { sender_id }` + `PendingContactCryptoKind::AutoAccept`
- — 9 sites (feasibility #1 change-list): enum, kind, `kind()`, storage `KIND_LABELS`,
- `kind_db_label`, the `kind_labels_match_enum` test, the drain's exhaustive `match`, and
- `count_account_build_ops` (decide inclusion — **yes**, so the needs-unlock banner counts
- pending auto-accepts; reword the banner copy, **scope S3**).
-- `parse_contact_request_doc` reads `autoAcceptProof` (§4.1).
-- `sync_contact_requests` enqueues `AutoAccept` (bounded) when a proof is present.
-- `drain_pending_contact_crypto` gains the `signer` param + the `AutoAccept` arm (§4.2).
-- Scanner send reuses `send_contact_request_with_external_signer(..., auto_accept_proof)`
- (already threaded). **scope M3:** the scanner must derive its `accountReference` first
- (in-signer, masked over the friendship xpub) and sign the proof over that **exact** value
- before broadcast — test that the signed `accountReference` equals the document's.
-- DPNS resolve: `IdentityWallet::resolve_name(&str) -> Option` (feasibility #5;
- not `search_names`).
-
-### 5.3 FFI (rs-platform-wallet-ffi)
-- `platform_wallet_build_auto_accept_qr(wallet, identity_id, out_uri…)` — owner: resolve
- the wallet's DPNS name (error if none), `expiry = now + 3600`, derive the key, build the
- `dash:?du=…&dapk=…` URI; return it. (Single Rust entry — no decisions in Swift.) `now` is
- passed in from Swift (FFI can't read the clock deterministically) or read via a host hook.
-- `platform_wallet_send_contact_request_from_qr(wallet, signer, core_signer, uri, out…)` —
- scanner: parse URI → resolve `du` → decode `dapk` → (derive accountRef, sign proof) → send
- the contactRequest with the proof. One call.
-- `platform_wallet_drain_pending_contact_crypto` gains `signer_handle: *mut SignerHandle`
- (the identity signer) alongside the existing `core_signer_handle` (§4.2).
-
-### 5.4 Swift (SwiftExampleApp)
-- **My QR** (net-new, `DashPayProfileView`): a "Show my QR" affordance rendering the URI
- from `build_auto_accept_qr` via the existing `generateQRCode` helper.
-- **Scan** (net-new entry in the DashPay tab toolbar): present `QRScannerView`; add a new
- parse branch + result type (`ScannedContact{username, keyBlob}`) to `QRPayloadParser`
- (the existing `ScannedPayment` path doesn't fit a no-address URI — **scope N2**), route to
- `platform_wallet_send_contact_request_from_qr`.
-- **Drain call-site:** `unlockWalletFromKeychain` now passes the `KeychainSigner` to the
- drain FFI (the added `signer_handle`).
-- **Feedback (scope S4):** the auto-accepted contact lands in `ContactsView` via `@Query`;
- add a light signal (the needs-unlock banner already counts the pending `AutoAccept`, so it
- shows "1 contact waiting…" until the drain completes, then it appears as a contact).
-
-## 6. Security (4 must-fixes folded)
-
-1. **Consensus-authenticated sender binding (must-fix #1):** verify binds `$ownerId =
- doc.owner_id()`. A malicious holder of a leaked QR key *can* sign a proof naming any
- sender, but cannot broadcast a contactRequest *as* a victim — platform consensus
- requires the doc to be signed by the owner's identity key. The verify gate MUST use the
- document owner id, never a proof-internal/client value.
-2. **No expired-but-valid foot-gun (must-fix #2):** the only acceptance entry checks expiry
- against the same timestamp that keyed verification; no public bare `verify` returns
- `true` for an expired proof.
-3. **Drain verdict mapping (must-fix #3):** invalid/expired/bad-index → permanent-clear;
- signer/network → transient-leave (§4.2). Prevents forever-churn.
-4. **Queue bound + verify-before-fetch (must-fix #4):** cap `AutoAccept` per owner; run the
- local ECDSA verify + expiry before any `Identity::fetch`, so a spam-N-identities attacker
- can't turn the owner's unlock into O(N) network round-trips.
-- **Private key in QR:** intrinsic to DIP-15 (the scanner must sign; the owner can't
- pre-sign without the scanner's id). Scoped (only auto-accept, not payments/identity),
- expiry-bounded (**1h**), blast radius = unwanted contact spam (removable via ignore).
- Acceptable documented trade-off, tightened by the short TTL (no off-switch since
- auto-accept is always-on, so the short TTL is the mitigation).
-- **Replay:** signed message binds `(sender, owner, accountReference)`; the doc's unique
- index is `($ownerId, toUserId, accountReference)` — no cross-sender replay, no on-platform
- dup. Cross-network separated by coin-type in the path.
-
-## 7. Failure modes
-- **Signer absent when a proof arrives:** enqueued (bounded), completes on next drain;
- surfaced by the needs-unlock banner.
-- **Expired / invalid / forged proof:** verify-gate rejects, entry cleared (permanent);
- request remains manually acceptable.
-- **`du` resolves to wrong/missing identity:** scanner send fails loudly; no contact.
-- **Owner has no DPNS name:** `build_auto_accept_qr` errors at QR-create (v1 requires `du`).
-- **Queue flood:** bounded per owner; junk cleared by local verify before any fetch.
-- **Expiry index overflow (> 2^31−1):** rejected at encode (and verify path-derivation errors → permanent-clear).
-
-## 8. Test plan
-- **Rust unit (auto_accept.rs):** key-blob round-trip; URI round-trip; **cross-actor** sign
- (loose key, scanner) → verify-with-pubkey (owner's re-derived pubkey) succeeds; wrong
- sender/owner/accountRef fails; expiry extraction + now ≤/≥ expiry; truncated/oversize/bad
- key-type rejected; structural pre-check.
-- **Rust flow (contact_requests.rs):** parser reads `autoAcceptProof`; ingest-with-proof →
- `AutoAccept` enqueued (and bounded — Nth+1 dropped); drain valid+unexpired → reciprocal
- sent + cleared; expired → cleared, not accepted; invalid → cleared; signerless/transient →
- stays queued; **signed `accountReference` == document's** (scope M3).
-- **FFI:** null/oversize/bad-URI input validation; build-QR → parse round-trip; drain with
- the new signer handle.
-- **Swift build:** `build_ios.sh` green.
-- **On-device (two sims):** A "Show my QR" → B scans → sends; A unlock/drain → contact
- auto-accepts (established, no tap on A); expired-QR path rejected.
-
-## 9. Decisions (resolved 2026-06-24)
-1. **TTL = 1 hour, fixed** (named constant `AUTO_ACCEPT_TTL_SECS = 3600`). DIP-15 is silent
- on the value (only mandates the timestamp *is* the expiry); 1h is the safe default given
- auto-accept is always-on (no off-switch). No picker in v1.
-2. **Auto-accept = always automatic.** No opt-in toggle. Valid + unexpired proofs
- auto-accept in the drain.
-3. **Scope = whole feature in one pass** (Rust + FFI + Swift + on-device), committed in
- logical layers on the branch.
-4. **`du`-only** (no `di=` fallback); require a DPNS name to build a QR.
-
-## 10. Review resolutions (4-lens, 2026-06-24)
-- **DIP-fidelity:** wire-faithful, no byte fixes; pinned BE + SHA256/LE as normative (§3).
-- **Security:** 4 must-fixes folded (§6); private-key-in-QR accepted as DIP-intrinsic,
- mitigated by the 1h TTL.
-- **Feasibility (§4 rewrite):** verify via `provider.receiving_xpub(path).public_key` (no
- `&Wallet`); drain gains the identity signer + FFI `signer_handle`; the sweep parser must
- read `autoAcceptProof`; use `resolve_name`. Queue variant change-list = 9 sites.
-- **Scope:** `du`-only + fixed TTL (cut `di=`/picker); cross-actor + `accountReference`
- ordering tests; banner counts `AutoAccept`; My-QR + Scan are net-new UI; clean stale
- `auto_accept.rs` docstrings.
diff --git a/docs/dashpay/SIGNER_SEED_ELIMINATION_SPEC.md b/docs/dashpay/SIGNER_SEED_ELIMINATION_SPEC.md
deleted file mode 100644
index a0594bd68d6..00000000000
--- a/docs/dashpay/SIGNER_SEED_ELIMINATION_SPEC.md
+++ /dev/null
@@ -1,666 +0,0 @@
-# DashPay Signer-Based Seed Elimination — Spec
-
-Status: DRAFT v3 (revised after a 3-agent deep design review: seedless
-background-sync architecture, signer/host-primitive model, security &
-failure-mode audit)
-Branch: `feat/dashpay-m1-sync-correctness` (PR #3841)
-Cross-repo: required key-wallet method (`extended_public_key`) has LANDED;
-pinned `rust-dashcore` rev already bumped.
-
-## 1. Problem
-
-PR #3639 ("external signable wallets", v3.1-dev) set this codebase's
-posture: a registered/restored wallet holds **no resident seed**
-(`WalletType::ExternalSignable`). Private-key work is done by passing a
-**Keychain-backed `Signer`** per operation; the seed lives only in the iOS
-Keychain.
-
-The DashPay paths added in PR #3841 did not follow that model — they reach
-for the resident seed (`send_payment` passes the `Wallet` to `build_signed`;
-`derive_contact_xpub` calls `wallet.derive_extended_public_key`; contact
-encrypt/decrypt + `accountReference` + contactInfo derive raw secrets off
-the `Wallet`). To make them work, `manager/attach_seed.rs::attach_wallet_seed`
-re-derives a signing `Wallet` from the Keychain seed and grafts it onto the
-loaded wallet via `std::mem::swap`. **That defeats the external-signable
-posture** (the seed becomes resident for the whole session) and is a
-workaround.
-
-The resolved **import-wallet bug** was the same disease for identity keys
-(Swift re-derived identity scalars from the mnemonic during discovery);
-fixed by the carry-scalar change (later reworked into the derive-sign-destroy resolver — see `IDENTITY_KEY_SCALAR_ELIMINATION_SPEC.md`).
-
-### Goal
-
-Every DashPay private-key operation runs through a Keychain-backed host
-primitive; the wallet seed is **never made persistently resident**;
-`attach_wallet_seed` (+ `unlockWalletFromKeychain`'s re-attach + the FFI
-export + the dual-gate/`mem::swap`) is **deleted** — but only after the
-background sync sweep is seedless-safe (§4.9 ordering constraint).
-
-### Honest scope of the security win (corrected after the security audit)
-
-This does **not** make the seed "never resident." Verified facts, to be
-stated plainly so the win is not over-credited:
-
-- **The full BIP-39 64-byte seed + master xprv are reconstructed in one
- contiguous buffer per operation** (`MnemonicResolverCoreSigner::resolve_derived_xprv`
- — `seed: Zeroizing<[u8;64]>`, master xprv on the next lines; sibling
- `sign_with_mnemonic_resolver.rs`). The whole wallet is derivable from that
- buffer for the duration of the op.
-- **The read is unlock-gated, not biometric-gated.** The Keychain mnemonic is
- `kSecAttrAccessibleWhenUnlockedThisDeviceOnly` with **no `LAContext` /
- `SecAccessControl`** on the read path (`WalletStorage.swift`). The
- `.biometryCurrentSet` stash exists but is **unused**. So "user present"
- really means "device unlocked" — any in-process code can drive the resolver
- while unlocked.
-- **The wipe is best-effort.** Byte buffers use `Zeroizing` (volatile + fence,
- runs on unwind). The two `ExtendedPrivKey` scalars use secp256k1's
- `non_secure_erase` (fills `[1u8;32]`, self-disclaimed as non-secure;
- `ExtendedPrivKey` has no `Drop`/`Zeroize` upstream). There is an **error-
- /unwind-path residue gap** (§4.2 hardening).
-
-The real, defensible benefit is a **smaller in-memory time-window** for the
-root secret — per-operation-and-wiped vs `attach_wallet_seed`'s session-long
-resident `Wallet` — plus consistency with the #3639 posture. It is a modest,
-honest improvement, not "the seed is never in RAM." (The dashj /
-dash-shared-core reference clients hold the decrypted seed for the whole
-session — see §8 — so there is no off-the-shelf signer-based DashPay to copy.)
-
-### Non-goals
-
-- Changing `downgrade_to_external_signable` (`wallet_lifecycle.rs:251`) — it
- is what makes the seed absent; it stays.
-- Re-introducing an in-memory-seed wallet (the dashj model).
-- Wiring QR-based auto-accept — tracked in the backlog (dashpay/platform#4020) (helpers KEPT, not
- deleted; §2 note).
-
-### Q2 scope, status & completion criteria (2026-06-23)
-
-**Q2** (the PR reviewer ask) = *remove the assign-seed workaround* = delete
-`attach_wallet_seed` (this §; §4.9). Live status: the backlog issue dashpay/platform#4020 (authoritative tracker).
-
-**Done + verified** (branch `feat/dashpay-m1-sync-correctness`; platform-wallet
-292/292; glue builds host + `aarch64-apple-ios-sim`): §2 inventory sites **#1–#6**
-— the entire seedless contact-request flow (send/accept/drain/always-enqueue
-sweep), the `ContactCryptoProvider` seam + signer host primitives (ECDH,
-accountReference, contactInfo seal/open, wrong-seed check), the deferred-crypto
-queue + SQLite persistence, and C3 (resident ECDH path deleted). Discovery is
-NOT a rewrite target — the **carry-scalar fix is kept** (§1).
-
-**Remaining for Q2 (do in this order). Folds in the 4-lens plan review
-(feasibility / security / scope-dedup / adversarial), which corrected two
-over-optimistic items in an earlier draft of this banner — see the MUST-FIX
-notes.**
-
-1. **#7 contactInfo (the load-bearing item).**
- - Add `contact_info_seal` AND **`contact_info_open`** to `ContactCryptoProvider`
- (+ glue impl over the signer primitives + `SeedCryptoProvider` test impl).
- **MUST-FIX (review):** publish is NOT seal-only — it must DECRYPT existing
- owned docs to decide update-vs-create (the doc↔contact binding lives inside
- `encToUserId` ciphertext; `contact_info.rs:435-447`), so it needs `open`.
- - Refactor the shared helper `fetch_decrypted_contact_infos` (resident-hardcoded
- at `:206`/`:450`, used by BOTH publish and the signerless sweep) into a public
- high-water scan (no keys) + a provider-`open` decrypt step. Thread `crypto`
- into `set_contact_info_with_external_signer` (publish) and the FFI
- (`platform_wallet_set_dashpay_contact_info_with_signer` gains a
- `core_signer_handle` — ABI break, like send/accept).
- - **MUST-FIX (security): root-path provenance.** Build the contactInfo root
- path in **Rust** via `identity_auth_derivation_path_for_type(net, ECDSA,
- identity_index, root_key_id)` (never Swift); pin the parity test against that
- REAL path, not `test_path()` — a wrong root silently writes undecryptable
- contactInfo with no on-chain oracle.
- - **MUST-FIX (security): high-water.** Publish is signer-present, so derive the
- high-water `derivationIndex` from a FRESH full decrypt, or refuse to publish —
- NEVER fall back to `0`/stale (collides the unique `($ownerId, rootIndex,
- derivationIndex)` index / reuses a key).
- - Implement the `ContactInfoDecrypt` **drain op** (currently a no-op stub,
- `contact_requests.rs:1440`): re-fetch owned docs + `contact_info_open` via the
- provider + re-run `validate_contact_request`-equivalent. Re-fetch = **testnet
- validated**. `derive_contact_info_keys` (resident twin) is deleted ONLY after
- both publish AND this sweep/drain path no longer call it.
- - **MUST-FIX (security): confused-deputy.** The drain (and opportunistic drains)
- must re-validate the queue entry's `owner_identity_id` is owned by THIS wallet
- before decrypting/registering.
-
-2. **Discovery/loading — NO library change (corrected; was wrong in the earlier
- draft).** The FFI already routes external-signable wallets through
- `discover_from_master` / `load_identity_by_index_from_master` (via
- `resolve_master_from_resolver`), and the `ResidentWallet` variants are the
- **live path for genuine resident-key wallet TYPES** (`WalletType::Mnemonic`/
- `Seed`, e.g. raw-seed imports with no persisted mnemonic) — NOT dead duplicates.
- So **keep** them; do **NOT** attempt the deep `verified_scalar`-drop rewrite
- (that's the env-blocked architectural change, and unnecessary — carry-scalar is
- kept). The only Q2 work here: (a) confirm every discovery/loading caller passes
- a non-null resolver after the deletion makes external-signable the universal
- posture (the FFI already errors on null — `identity_discovery.rs:194`); (b) the
- test-helper rework in step 3. NOTE: the §2 "exhaustive" table is exhaustive over
- *DashPay-contact* paths; the discovery resident derive `derive_identity_auth_keypair`
- (`identity_handle.rs:191`) is a resident-key-TYPE path that legitimately stays.
-
-3. **Test-helper rework + Swift.** ✓ DONE. The test helpers were made seedless
- earlier (`c42d2e413e`). Swift (`70aaf32f9f`): `unlockWalletFromKeychain(_:)`
- now verifies-binds + drains-on-unlock (no seed graft); send/accept/contactInfo
- thread the resolver `core_signer_handle`. Verified by regenerating the
- xcframework header (`build_ios.sh --target sim`) + an arm64-sim SwiftExampleApp
- build (BUILD SUCCEEDED).
-
-4. **Delete** `attach_wallet_seed` + FFI export + dual-gate/`mem::swap` + the dead
- `dash_sdk_dashpay_*` rs-sdk-ffi surface (4 fns, zero Swift callers — confirmed)
- + the legacy `KeychainSigner.sign(...)->Data?` nil-swallow.
- - **MUST-FIX (security): atomic wrong-seed wiring.** ✓ DONE (Rust, `fe3ab74e19`):
- `attach_wallet_seed` (lib + FFI + dual gate) deleted and `verify_seed_binds`
- (`PlatformWallet` method + `platform_wallet_verify_seed_binds_to_wallet` FFI)
- landed in the same commit — signer-derived BIP44-0 xpub vs the persisted one,
- mismatch → `SeedMismatch`. The comparison lives in `verify_seed_binds`, which
- derives the xpub through the `ContactCryptoProvider::receiving_xpub` seam (a
- generic derive-at-path) — no redundant trait method. (The signer's
- `verify_binds_to_xpub` primitive was NOT used and was deleted post-review as
- dead code; the live path reimplements the equality in `verify_seed_binds`.)
- The dead `dash_sdk_dashpay_*` surface was removed earlier (`db18688545`).
- ✓ Swift wiring DONE (`70aaf32f9f`): the verify FFI is called at unlock and the
- `KeychainSigner.sign(...)->Data?` nil-swallow is deleted (see step 3 Swift).
- - **SHOULD-FIX (security): §4.2 sibling-FFI leak.** ✓ DONE (`feb266fd1b`): the
- `WipingXprv` RAII guard now wraps `master`/`derived` in
- `rs-platform-wallet-ffi/src/sign_with_mnemonic_resolver.rs`, scrubbing on the
- error/unwind paths the Ok-only `non_secure_erase` missed.
-
-**Done when:**
-- ✓ `git grep attach_wallet_seed` empty (outside docs/regenerated headers).
-- ✓ `cargo test -p platform-wallet` green (292) + glue (`platform-wallet-ffi`) green.
-- ✓ `build_ios.sh --target sim` regenerates the header (verify FFI present, attach
- gone, send/accept/contactInfo carry `core_signer_handle`) + SwiftExampleApp
- builds clean (arm64 sim, BUILD SUCCEEDED).
-- **On-device acceptance — VALIDATED on devnet `paloma` (2026-06-23)** via idb UI
- automation across **two simulators** (sim A = funded `Test_devnet`, 9 DASH, 3
- identities; sim B = freshly created `SimB` wallet + new identity "Eve"). Every
- signing op below ran through the Keychain resolver with **no resident seed**:
- - ✓ App builds, installs, launches, runs full (SDK init, network switch, wallet
- list/detail, DashPay sync) on the new seedless binary — no crash.
- - ✓ **Seedless Core payment + IS-lock** — sim A sent 1 DASH to sim B's address;
- tx signed via the resolver, broadcast, **InstantLock validated** (this is the
- real wrong-seed-would-fail path: the seedless wallet signed a real Core tx).
- - ✓ **Seedless identity registration** — sim B: asset lock → InstantSend proof →
- ChainLock proof → Platform registration, all via the resolver ("Identity
- created" `BfWHEg…`).
- - ✓ **Seedless DPNS name registration** (`eveqaseed1ess2026`) + **profile publish**
- ("Eve") — both Platform writes via the resolver.
- - ✓ **Seedless contactInfo publish** — `setDashPayContactInfo`
- (`core_signer_handle`) published an encrypted contactInfo doc, **create**
- (`updated=false`) **and update** (`updated=true`).
- - ✓ **Seedless contact request SEND** (Alice→Eve) — `send_contact_request_with_signer`
- (`core_signer_handle`): registered the DashpayReceivingFunds account.
- - ✓ **Seedless contact request ACCEPT** (Eve→Alice) —
- `accept_contact_request_with_signer` (`core_signer_handle`): reciprocal request
- + registered DashpayReceivingFunds **and** DashpayExternalAccount (the ECDH +
- accountReference path). Contact established cross-device (sim A shows Eve as
- contact #3).
- - ⏳ NOT exercised on-chain (covered elsewhere): (a) **wrong-seed rejected loud**
- via `verify_seed_binds` at unlock — gated behind the `loadFromPersistor`
- restorable path; cleanly forced only by a destructive wipe+reimport. Covered by
- the high-fidelity unit test (real `key_wallet` wallet, same BIP44-0 path,
- accept/reject) — and the live Core/Platform signing above proves the resolver
- derives correct keys. (b) **cross-device contactInfo decrypt** (same owner on a
- 2nd device) — publish validated; decrypt-drain unit-tested.
-
- **Unrelated finding (not a seed defect):** the DashPay contact-profile chunk fetch
- fails with a DAPI error `missing order by for range error: query must have an
- orderBy field for each range element` ("Failed to fetch a contact-profile chunk;
- will retry next sweep"). Contact establishment still succeeds; contact *profiles*
- may not render. Track + fix separately.
-
-**Out of Q2 scope** (tracked in the backlog (dashpay/platform#4020)): §6b queue restore (upstream
-`ClientStartState::wallets` — note the security review's caveat that until restore
-works, a contact discovered-then-app-killed-before-unlock never finishes setup);
-the §4.8 present-but-zero-keys import caveat; QR auto-accept wiring.
-
-## 2. Inventory of seed-dependent paths (revised; exhaustive over **DashPay-contact** paths)
-
-Verified by grepping every `derive_extended_p*_key` / `build_signed(wallet` /
-`.has_seed()` reader, and by tracing reachability from the background sweep.
-NOTE (plan review): this table enumerates the DashPay-contact seed paths (#1–#7).
-The **identity-discovery** resident derive `derive_identity_auth_keypair`
-(`identity_handle.rs:191`) is a separate resident-key-TYPE path — it legitimately
-stays for `WalletType::Mnemonic`/`Seed` wallets and is NOT a Q2 deletion target
-(external-signable wallets already route discovery through the resolver). See the
-Q2 banner step 2.
-
-`bg?` = reachable from the **signerless** recurring sweep
-(`DashPaySyncManager` → `dashpay_sync` → `build_contact_accounts` /
-`sync_contact_infos`). The sweep FFI takes **no signer handle** and runs with
-no user present — so any `bg?`+secret op is a deferral problem (§4.6).
-
-| # | Call site | Capability | bg? | Phase |
-|---|---|---|---|---|
-| 1 | `send_payment` (`payments.rs`, build site) | ECDSA sign at path | no | **1 — DONE** |
-| 2 | `derive_contact_xpub` (`dip14.rs:98`), caller = send-contact-request flow (signer present) | xpub at hardened path | no | **2** (bundled with #4 — same function) |
-| 2b | `register_contact_account` → `wallet.derive_extended_public_key` (`contacts.rs:186`) | xpub at hardened path | **yes** | **2** (steady-state no-op — see note) |
-| 3 | contact-request / profile / contactInfo **doc signing** | `Signer` | — | already done |
-| 4 | contact-request xpub **encrypt** — `derive_encryption_private_key` (`identity_handle.rs:476`) → `EcdhProvider::SdkSide` (`sdk_writer.rs:240`) | ECDH | no | **2** |
-| 5 | contact xpub **decrypt** — `register_external_contact_account` (`contacts.rs:472/510/514`) | ECDH | **yes** | **2 — DEFERRED via queue** |
-| 6 | `accountReference` (`contact_requests.rs:204`; `dip14.rs:262`) | HMAC keyed by the **same** ECDH key | partial | **2** |
-| 7 | contactInfo AES keys — `derive_contact_info_keys` (`contact_info.rs:80`) via sync (read) + publish (write) | raw hardened-child bytes as AES-256 key | **yes (read)** | **2 — DEFERRED via queue** |
-
-Reclassification vs v2 (from the review):
-
-- **Sites 2 and 2b moved Phase 1 → Phase 2.** Both live in functions that
- *also* perform Phase-2 ECDH (#4 in `send_contact_request_with_external_signer`;
- #5 in the `register_external` path that the sweep drives). Converting their
- xpub piecemeal in Phase 1 would double-touch the same functions. Bundle each
- xpub conversion with the ECDH conversion of its host function (one
- wallet-HD closure threading per function). Phase 1 stays exactly the
- already-shipped, green slice (#1 + the `extended_public_key` foundation +
- dead-API deletion).
-- **2b is a steady-state no-op.** `register_contact_account` has an early-exit
- (`contacts.rs:153–174`): once the receiving account exists it returns before
- the derive at `:186`. The account **is persisted** —
- `AccountRegistrationEntry.account_xpub: ExtendedPubKey` (`changeset.rs:967`)
- bincode round-trips (`persistence.rs:2387`/`2869`) and restores via
- `Account::from_xpub` into a watch-only `new_external_signable` wallet
- (`persistence.rs:2874/2889`), with the address pool + `used` flags
- (`AccountAddressPoolEntry`). Gap-limit refill is pure public `ckd_pub`
- (`KeySource::Public`, key-wallet `managed_account_trait.rs:295`). So the
- derive at `:186` fires **only** on a contact's first-ever registration in a
- session where it was never persisted — the deferral edge case (§4.6).
-- **Only #5 (ECDH decrypt) and #7-read (contactInfo decrypt) are genuine
- background blockers.** Everything else the sweep does (request ingest,
- profile sync, address matching, gap-limit refill, payment reconcile) is
- pure public derivation or no derivation at all.
-
-Notes (unchanged from v2):
-- **#6 is not a separate key** — `calculate_account_reference` is HMAC-keyed by
- the same ECDH scalar as #4/#5; the ECDH handling covers it.
-- **auto-accept (`auto_accept.rs:80`) is KEPT** — real but unwired DIP-15
- feature; converts cleanly to a sign path when wired. Tracked in the backlog (dashpay/platform#4020).
-- **Dead read APIs** `contact_xpub` / `contact_payment_addresses` had zero
- callers → **DELETED in Phase 1**.
-
-`attach_wallet_seed` consumers to remove (Phase 2, §4.9): FFI
-`platform_wallet_manager_attach_wallet_seed_from_mnemonic` (`manager.rs:430`);
-Swift `unlockWalletFromKeychain` (`PlatformWalletManager.swift:468`); test
-helpers (`payments.rs`).
-
-## 3. Capabilities the Keychain signer must expose
-
-Two distinct, correctly-separate signer notions exist and stay separate
-across the FFI seam (§4.4):
-
-- **Doc-signer** — `dpp::identity::signer::Signer`
- (state-transition signing). Impl = `VTableSigner`; FFI handle =
- `SignerHandle`/`VTableSigner` (seed never enters Rust). Already wired.
-- **Wallet-HD signer** — `key_wallet::signer::Signer` (ECDSA-at-path,
- `public_key`, `extended_public_key`). Impl = `MnemonicResolverCoreSigner`;
- FFI handle = `MnemonicResolverHandle` (mnemonic transiently enters Rust;
- all crypto runs in FFI-crate Rust and wipes).
-
-Wallet-HD capabilities:
-
-1. **ECDSA sign at path** — EXISTS (`sign_ecdsa`).
-2. **public_key at path** — EXISTS.
-3. **extended_public_key at path** — DONE (key-wallet method + host primitive).
-4. **ECDH** `(path, peer_pubkey) -> shared_secret` — NEW host primitive (§4.5).
-5. **contactInfo seal/open** — NEW host primitive (§4.5).
-
-Capabilities 4–5 are added as **inherent methods on `MnemonicResolverCoreSigner`**
-(in `rs-sdk-ffi`, where it already lives) and consumed by platform-wallet via
-**closures** (the existing `EcdhProvider::ClientSide` seam) — no new trait, no
-new crate (§4.4).
-
-## 4. Design
-
-### Phase 1 — sign + xpub (low-risk, SHIPPED green)
-
-#### 4.1 key-wallet change — LANDED
-
-`key_wallet::signer::Signer::extended_public_key` added as a **provided
-default that errors** (not a breaking required method); `InMemorySigner` test
-impls override it; pinned rev bumped. `TransactionSigner`/`build_signed`
-unchanged.
-
-#### 4.2 host primitive: extended_public_key (FFI + Swift) — DONE + 1 hardening
-
-`MnemonicResolverCoreSigner::extended_public_key` reuses the shared
-`resolve_derived_xprv` helper, computes `ExtendedPubKey::from_priv`, and wipes
-both scalars. Interop-guard test pins signer-xpub == `Wallet::derive_extended_public_key`.
-
-**Phase-1 hardening (from the security audit) — TODO before merge:** wipe the
-`master` scalar on the **error/unwind path** of `resolve_derived_xprv`. Today
-the explicit `non_secure_erase` runs only in the `Ok` arms of `derive_priv`
-and `extended_public_key`; if `master.derive_priv(path)` returns `Err`, or a
-panic unwinds between materialization and the wipe, the `master` scalar leaks
-(no `Drop`/`Zeroize`). Same gap in `sign_with_mnemonic_resolver.rs`. Preferred
-fix: a small RAII wipe-guard around the two `ExtendedPrivKey`s (so all exit
-paths wipe), rather than more hand-placed calls — there will be **five** such
-sites once §4.5 lands.
-
-**Path provenance (security requirement).** Paths are built in Rust
-(`AccountType::…derivation_path()`, `identity_auth_derivation_path_for_type`,
-the `dip14`/`contact_info` path builders) and passed **opaquely** through the
-FFI; Swift never assembles a path.
-
-#### 4.3 call-site conversions (Phase 1) — DONE
-
-- `send_payment` takes `` and signs via
- `build_signed(signer, …)`; FFI `platform_wallet_send_dashpay_payment` takes
- a `MnemonicResolverHandle`; Swift threads the resolver under
- `withExtendedLifetime`.
-- Dead `contact_xpub` / `contact_payment_addresses` deleted.
-- Sites 2 and 2b are **not** converted here — moved to Phase 2 (§2).
-
-### Phase 2 — raw-secret paths + seedless sweep + delete the workaround
-
-#### 4.4 Signer & host-primitive model (no duplicate logic)
-
-**Decision: two FFI handles; raw-secret ops as inherent methods on the
-existing wallet signer, consumed via closures — NO new trait, NO new crate.**
-
-- Keep `VTableSigner` (doc-signer) and `MnemonicResolverHandle` (wallet-HD) as
- **two** FFI handles. Merging them would either regress the doc-signer (seed
- currently never enters Rust) or force DIP-15 crypto into Swift — both
- rejected.
-- `MnemonicResolverCoreSigner` (in `rs-sdk-ffi`) already **is** the wallet-HD
- binding (impls `key_wallet::signer::Signer`) and is constructed only by the
- `rs-platform-wallet-ffi` glue crate. `rs-sdk-ffi` and `platform-wallet` do not
- depend on each other, so a shared `WalletKeyProvider` *trait* would need a new
- crate or a layering inversion (the external `key-wallet` is ruled out — keep
- the cross-repo PR to one method, and ECDH must not become a `Signer` method).
- Avoid all of that: add the raw-secret capabilities as **inherent methods** on
- `MnemonicResolverCoreSigner` (it gains a `platform-encryption` dep — a leaf
- crypto crate, no cycle), and let platform-wallet consume them via **closures**
- — the `EcdhProvider::ClientSide { get_shared_secret }` seam it already uses,
- plus a closure param on the decrypt/drain path. The glue crate wires the
- closures from the signer's methods (it already owns the signer's construction
- + lifetime). *(An earlier draft proposed a `WalletKeyProvider: Signer`
- extension trait; dropped — no shared home given the crate graph, and the
- closure seam already exists.)*
-
-Inherent methods on `MnemonicResolverCoreSigner` (sync — derivation is
-CPU-bound + the resolver call is synchronous; the consuming `ClientSide` closure
-wraps each in a future at the FFI seam):
-
-```text
-ecdh_shared_secret(path, peer_pubkey) -> Zeroizing<[u8;32]> // #4/#5 DONE
-ecdh_shared_secret_and_account_reference(path, peer, compact_xpub, account_index, version)
- -> (Zeroizing<[u8;32]>, u32) // #6
-unmask_account_reference(path, prior_reference, compact_xpub) -> (u32, u32) // #6
-contact_info_seal(root_path, derivation_index, contact_id, plaintext, iv) -> ContactInfoSealed // #7
-contact_info_open(root_path, derivation_index, enc_to_user_id, blob) -> ContactInfoOpened // #7
-```
-
-- Document-ops conversion: send/accept contact-request already thread a
- `doc_signer` for the state transition; for the xpub/ECDH they additionally
- receive the wallet-HD closure(s) the glue crate builds from the signer.
- `send_payment` keeps only its `key_wallet::Signer`. Swift passes the two
- handles it already holds — **no new Swift class, no new Swift crypto**.
-- **Delete the dead `dash_sdk_dashpay_*` ClientSide FFI surface**
- (`rs-sdk-ffi/src/dashpay/contact_request.rs`: the two entry points,
- params/results, `DashSDKEcdhMode`, the four `*_with_{shared_secret,private_key}`
- helpers) and regenerate the cbindgen header. Zero non-Rust callers; it is a
- divergence-prone parallel orchestration of the same `rs-sdk` core, and its
- `SdkSide` raw-scalar ABI contradicts the new posture. The `rs-sdk`
- contact-request core + `EcdhProvider` stay (single source).
-
-**No-duplicate-logic trace:** every host method body is "derive scalar at a
-Rust-built path (the shared `resolve_derived_xprv`, scrubbed by the `WipingXprv`
-guard on every exit path) → call the existing `platform_encryption` / `dip14`
-fn → return the result." Zero new crypto. Single sources stay: DIP-15 ECDH/AES →
-`platform-encryption`; accountReference HMAC + masking → `dip14`; contact-request
-orchestration → `rs-sdk platform::dashpay::contact_request`; contactInfo wire
-codec → `crypto/contact_info.rs` (plaintext-only, runs outside the primitive).
-
-#### 4.5 raw-secret host primitives (option iii) + EcdhProvider collapse
-
-For #4–#7 the host primitive derives the key **and runs the crypto in
-FFI-crate Rust**, returning only the result; the raw scalar never reaches
-`rs-platform-wallet` or Swift.
-
-- **ECDH (#4/#5):** switch `sdk_writer.rs:240` from
- `EcdhProvider::SdkSide { get_private_key }` to
- `ClientSide { get_shared_secret }` backed by a closure calling
- `MnemonicResolverCoreSigner::ecdh_shared_secret` (DONE; parity-pinned).
- For all DashPay paths the model **collapses to `ClientSide` only**; delete
- `derive_encryption_private_key` (`identity_handle.rs:476`) and
- `SendContactRequestParams.ecdh_private_key` — the two places that
- materialize the raw scalar in `rs-platform-wallet`. Shared secret MUST be
- byte-identical to `platform_encryption::derive_shared_key_ecdh`
- (`SHA256((0x02|y_parity)‖x)`); peer pubkey validated on-curve before ECDH.
-- **accountReference (#6):** folded into `ecdh_shared_secret_and_account_reference`
- / `unmask_account_reference` so the encryption scalar is used for both ECDH
- and the `dip14` HMAC in one derivation and never returns raw.
-- **contactInfo (#7):** `contact_info_seal` / `contact_info_open` derive the
- two hardened-child AES keys and run encrypt/decrypt via `platform_encryption`,
- returning only ciphertext/plaintext. The DIP-15 wire codec stays in
- `crypto/contact_info.rs` (no key material).
-
-**Interop parity (required):** host-path accountReference, contactInfo blob,
-and ECDH secret MUST equal the in-process results for the same seed+path —
-each pinned by a test (DIP-15 interop vs dashj/dash-shared-core).
-
-#### 4.6 Seedless background-sync & the deferred-crypto queue (NEW — the core)
-
-The recurring sweep has no signer and no user present (verified:
-`DashPaySyncManager` holds no signer; FFI `platform_wallet_sync_contact_requests`
-/ `…_dashpay_sync_start` / `…_sync_now` take none). Design rule: every sweep
-op is **public-derivable** or **deferred** — never resident-seed.
-
-**Persisted pending-crypto queue.** Add `pending_contact_crypto:
-Vec` to `PlatformWalletChangeSet`, keyed per
-`(owner_identity_id, contact_id)` with an op discriminant:
-
-```
-PendingContactCrypto { owner_identity_id, contact_id,
- op: RegisterReceiving // our friendship xpub (2b, first-time only)
- | RegisterExternal { encrypted_public_key, our_decryption_key_index,
- contact_encryption_key_index } // #5 ECDH decrypt
- | ContactInfoDecrypt, // #7 (idempotent re-fetch+decrypt)
- enqueued_at_ms }
-```
-
-- Stores **only ciphertext + public key indices** (safe to persist). Rides the
- existing `persister.store(...)` changeset pipeline (no side channel);
- restored through `build_wallet_start_state`; a missing/old column restores as
- an empty queue (skip-and-continue convention).
-- **Persisted, not in-memory**, because restore-from-Keychain is exactly when
- it's needed and the app may be killed between background discovery and the
- next foreground unlock.
-
-**Enqueue (background, seedless).** In `build_contact_accounts` and
-`sync_contact_infos`, when key material is unavailable (the `Unavailable`
-classification, §4.7), **enqueue** instead of the current silent skip-and-log
-/ retry-forever / channel-kill. Idempotent per `(owner, contact, op-kind)`.
-
-**Drain (foreground, signer present)** — no new signer plumbing into the
-background manager:
-
-1. **On-unlock (primary):** a **new FFI** `platform_wallet_drain_pending_contact_crypto(wallet, core_signer)`
- wired into the same Swift code path that previously called
- `unlockWalletFromKeychain` — so deleting the re-attach and adding the drain
- are one change. It runs each entry through the §4.5 host primitives, then
- the public registration, and clears the entry.
-2. **Opportunistic:** `send_payment` / `send_contact_request_with_external_signer`
- / `accept_…` / `set_contact_info_with_external_signer` each drain queue
- entries **for the identity they operate on** before their own work — so the
- first user action on a contact resolves its deferred crypto (e.g. tapping
- "Pay" on an inbound-only contact builds its `DashpayExternalAccount`).
-
-Drain is idempotent (each op re-checks its early-exit guard). While queued, the
-sweep still does all PUBLIC work for the contact (ingest, profile, address
-match, reconcile); the contact is visible but "needs unlock to finish setup."
-
-#### 4.7 Error classification: Transient / Unavailable / Permanent (must-fix)
-
-The live bug behind "is deferral safe": `build_contact_accounts` today maps an
-ECDH/derive failure to `Permanent` → `mark_contact_channel_broken`
-(irreversible, `warn!`-only). A **locked Keychain is transient** but would
-permanently disable payments to a contact. Fix before any Phase-2 coding:
-
-- Introduce a **third** arm, `Unavailable` (signer/key material absent), in
- `RegisterExternalError` (and the contactInfo path), **distinct** from
- `Transient` (retry soon) and `Permanent` (malformed data → mark broken).
-- `Unavailable` → **enqueue (§4.6) and pause this contact's build; never kill,
- never churn-retry every 15s.** Only genuinely malformed inputs (bad
- ciphertext, off-curve pubkey) are `Permanent`.
-- **Fix the `is_seedless` gate** (`contact_requests.rs:904–921`): it currently
- keys on `identity_index.is_none()`, which a seedless-but-indexed identity
- passes — falling through to the channel-kill. Gate on "can I derive **right
- now**?" (key material available), not "is this a wallet-owned identity?".
-- Add a **needs-rebuild / needs-unlock marker** surfaced to the UI for
- deferred contacts.
-
-#### 4.8 wrong-seed safety check (replaces the deleted dual gate)
-
-`attach_wallet_seed`'s dual id/xpub gate also verified the Keychain mnemonic
-binds to the loaded wallet. Replace it with a **one-time xpub self-check** at
-signer construction / first use: derive BIP44 account-0 xpub from the resolved
-mnemonic and compare to the wallet's persisted account-0 xpub; mismatch fails
-loud. **Caveat (security audit):** this catches a *wrong* seed but **not** a
-*present-but-zero-keys* import (the open imported-identity bug, §7
-prerequisite).
-
-#### 4.9 delete the workaround — ORDERING CONSTRAINT
-
-Remove `attach_wallet_seed`, the FFI export, `unlockWalletFromKeychain`'s
-re-attach (replaced by the §4.6 drain), the dual gate + `mem::swap`; rework
-the test helpers to inject a test `Signer` / pre-seed the queue. Also delete
-or make-throwing the legacy `KeychainSigner.sign(identityPublicKey:data:)
--> Data?` swallow (returns reasonless `nil` on any failure).
-
-**Do NOT execute this deletion until the sweep is seedless-safe** (§4.6 queue
-+ §4.7 classification landed and tested). Until then the resident seed is what
-keeps the signerless sweep working; removing it first turns every background
-tick on an unbuilt contact into an irreversible channel-kill.
-
-## 5. Failure modes
-
-- **Signer unavailable mid-sweep (ECDH/xpub build):** classified `Unavailable`
- → enqueued + paused, **not** channel-killed, **not** retried every 15s
- (§4.7). Resolves on next drain.
-- **Keychain locked while the tokio sweep ticks:** the loop has no
- scenePhase/BG gating; it MUST hit the `Unavailable`+enqueue path, never the
- kill path.
-- **Pending queue never drains:** mitigated by the on-unlock drain wired to the
- old re-attach trigger + opportunistic drain on any signer-present action +
- a UI "needs unlock" marker. Pinned by an on-device test.
-- **Poison entry (permanently malformed ciphertext):** `Permanent` → mark
- broken + clear entry (no requeue). Transient → keep for next drain.
-- **Partial registration** (persist-ok / in-memory-insert-fail): unchanged
- (store-before-insert; relaunch rebuilds) — but the rebuild must classify a
- locked Keychain as `Unavailable`, not escalate.
-- **Restore-from-Keychain / app-reinstall → zero signing keys:** the
- imported-identity bug — **RESOLVED** by carry-scalar (the materialization path no
- longer re-derives from a not-yet-stored mnemonic). Phase 2 only confirms imported
- wallets reach the resolver for xpub/ECDH (mnemonic stored by `createWallet`).
-- **Wrong/mis-mapped mnemonic:** §4.8 xpub self-check, fails loud.
-- **Read-path:** converted readers propagate signer errors; never return
- empty/zero/stale addresses.
-
-## 6. Test plan
-
-- **key-wallet:** `extended_public_key` on `InMemorySigner` matches
- `derive_extended_public_key` (DONE).
-- **platform-wallet:** test `Signer` replaces `attach_wallet_seed`;
- signer-based tests for `send_payment` (DONE: interop-guard) + contact-request
- create/accept (ECDH), contactInfo round-trip; **interop parity** tests
- (accountReference, contactInfo, ECDH byte-equal to in-process).
-- **Seedless sweep:** watch-only wallet + persisted xpubs → sweep does all
- PUBLIC ops with no signer; `register_contact_account` early-exit no-op once
- persisted.
-- **Deferral queue:** background-discover inbound contact → `Unavailable` →
- enqueue (not kill); drain on unlock → contact payable. A `Permanent` error
- clears the entry AND sets `payment_channel_broken`. A locked-Keychain
- (transient) does **not** mark broken.
-- **Per-primitive no-residue:** each inherent host-primitive method wipes scalars
- on Ok **and error/unwind** paths.
-- **FFI:** input-validation (null/oversize/bad-path) incl. contactInfo depth-6
- paths (hardened 65536/65537).
-- **Acceptance grep:** `git grep attach_wallet_seed` empty; no surviving
- `derive_extended_private_key` / `build_signed(wallet` on DashPay paths.
-- **On-device:** clean wipe → import → discover → send/accept contact request,
- send payment, publish profile + contactInfo, **background-discover an
- inbound contact then unlock → it becomes payable** — all with **no**
- re-attach.
-
-## 7. Rollout order (revised)
-
-1. **Phase 1 (SHIPPED, green):** key-wallet method → host `extended_public_key`
- primitive → `send_payment` conversion → delete dead read APIs. **Remaining
- before merge:** the §4.2 error-path wipe hardening + iOS `build_ios.sh`
- verification.
-2. **Phase 2 prerequisites (design + fix BEFORE feature code):**
- - §4.7 three-state error classification + `is_seedless` gate fix.
- - §4.6 persisted pending-crypto queue + drain FFI.
- - ~~Resolve the imported-identity zero-signing-keys bug~~ **RESOLVED** by the
- carry-scalar change (commit `c567981c46`; on-device 23/23 signable,
- regression tests green). No longer a blocker. Phase 2 need only confirm an
- imported wallet reaches the resolver for xpub/ECDH (its mnemonic is stored in
- the Keychain by `createWallet`, so the resolver-backed signer works for it).
-3. **Phase 2 feature code:** inherent host primitives on
- `MnemonicResolverCoreSigner` (ECDH + accountReference + contactInfo) →
- `EcdhProvider` collapse to `ClientSide` →
- convert #2/#2b/#4–#7 (each xpub bundled with its function's ECDH) → delete
- the dead `dash_sdk_dashpay_*` surface → §4.8 self-check.
-4. **Only then §4.9:** delete `attach_wallet_seed` + re-attach + legacy
- `KeychainSigner.sign(...)->Data?`.
-5. Build + clippy + tests + on-device acceptance after each phase.
-
-Cross-repo: the key-wallet edit (already landed) needed Claude Code run from
-`/Users/ivanshumkov/Projects/dashpay/` (sibling-repo writes). FFI/Swift work
-goes through the **swift-rust-ffi-engineer** agent.
-
-## 8. Alternatives rejected
-
-- **In-memory seed (dashj / dash-shared-core).** Reference clients hold the
- decrypted seed in-session — no signer-based DashPay to copy. Rejected:
- abandons the #3639 posture.
-- **Phase-1-only (xpub/sign):** does not delete `attach_wallet_seed`. Adopted
- *as Phase 1*, not the end state.
-- **Keep the workaround:** rejected by product decision.
-- **Unified single signer handle (doc + wallet + ECDH):** rejected — regresses
- the doc-signer (seed never in Rust today) or pushes DIP-15 crypto into Swift.
- Add the wallet-side raw-secret ops as inherent methods on the existing
- signer, consumed via closures, instead (§4.4).
-- **§4.5 option (i) (return raw scalar):** rejected for option (iii) — exposes
- the ECDH key raw, defeating the hashing; the carry-scalar precedent
- (write-once Rust→Swift) does not sanction the read-many reverse flow.
-- **Skip-and-log deferral (current code):** rejected — silently strands
- contacts and (worse) the `Permanent` path irreversibly kills channels.
- Replaced by the §4.6 persisted queue + §4.7 classification.
-
-## 9. Security must-fixes from the multi-agent review (priority order)
-
-1. **[CRITICAL]** Three-state classification (`Unavailable` ≠ `Permanent`);
- never auto-`mark_contact_channel_broken` on unavailable key material (§4.7).
-2. **[CRITICAL]** Give the sweep a seedless-safe path: enqueue+defer, never
- derive-or-die; do not delete `attach_wallet_seed` until this lands (§4.6,
- §4.9 ordering).
-3. **[CRITICAL]** Fix the `is_seedless` gate predicate — key-availability, not
- `identity_index.is_none()` (§4.7).
-4. **[RESOLVED]** Imported-identity zero-signing-keys bug — fixed by carry-scalar
- (commit `c567981c46`), on-device-verified, regression tests green. No longer a
- Phase-2 blocker; only confirm imported wallets reach the resolver for xpub/ECDH.
- (Note: §4.8's xpub self-check still does not cover a present-but-zero-keys import,
- but the carry-scalar fix means imports now materialize keys, so this is moot.)
-5. **[HIGH]** Tighten §1 honest-scope wording (done) + fix the
- `resolve_derived_xprv` error-/unwind-path scalar leak; prefer one RAII
- wipe-guard over five hand-placed `non_secure_erase` calls (§4.2).
-6. **[MEDIUM]** Delete / make-throwing the legacy `KeychainSigner.sign(...)->Data?`
- nil-swallow (§4.9).
-7. **[MEDIUM]** UI marker for contacts pending an unlock-drain (§4.6/§4.7).
-
-## 10. Resolved review questions
-
-- **key-wallet method:** provided-default-that-errors — §4.1.
-- **raw-secret:** option (iii) via inherent host primitives on the wallet
- signer, consumed by closures — §4.4/§4.5.
-- **signer surface:** two FFI handles; raw-secret ops as inherent methods +
- `EcdhProvider::ClientSide` closures (no new trait/crate) — §4.4.
-- **dead `dash_sdk_dashpay_*` surface:** delete — §4.4.
-- **read-API ripple:** dead → deleted — §2.
-- **dual-gate deletion:** safe for grafting; wrong-seed detection preserved via
- the §4.8 self-check (but not zero-keys — §7).
-- **ECDH placement:** FFI-layer host primitive, not a key-wallet trait method —
- §3/§4.5.
-- **"defer site 2b":** safe **only** with §4.6 queue + §4.7 classification +
- §4.9 ordering. Without them it is silent, irreversible channel corruption.
-- **pre-check:** Swift uses `platform_wallet_send_contact_request_with_signer`;
- the rs-sdk-ffi ClientSide surface is the reference template for the ECDH
- switch and is deleted afterward.
diff --git a/docs/dashpay/SPEC.md b/docs/dashpay/SPEC.md
deleted file mode 100644
index 6306eec5f1f..00000000000
--- a/docs/dashpay/SPEC.md
+++ /dev/null
@@ -1,1362 +0,0 @@
-# DashPay — Implementation Spec & Gap Analysis
-
-> **Purpose.** A single working spec for getting the **full DashPay flow** — sync,
-> create/update profile, send contact request, approve/reject contact requests,
-> send money to a contact — *done and tested* in the **platform wallet**
-> (`rs-platform-wallet` + FFI) and surfaced as a **nice UI** in the
-> **SwiftExampleApp**.
->
-> **Status (2026-06-10).** DashPay is **already ~80% implemented end-to-end.** This
-> document maps the protocol, inventories what exists, isolates the gaps & bugs,
-> and lays out the remaining work + test plan. It is *not* a greenfield design —
-> it is a finish-and-polish plan.
->
-> **Status update (2026-06-18) — the finish-and-polish work is essentially done
-> on `feat/dashpay-m1-sync-correctness` (PR #3841).** Resolution of the Part-0 gap
-> table: **G1, G2, G12, G13, G14, G15** (the P0 sync/wire/key-purpose blockers) and
-> **G3, G6, G7, G8, G9, G10** — all **DONE** (M1–M4). **G5** reworked and shipped as
-> a per-sender, reversible, **local-only Ignore** (Spec 2) across every layer incl.
-> the SQLite persister; cross-device sync deferred to a future encrypted `profile`
-> field (contract track — the `contactInfo` route was rejected for the R1 leak).
-> **G11**: the `network/` layer now has unit coverage; the live cross-client e2e
-> ride PR #3549 and stay blocked on devnet funding. **G4** (watch-only ECDH) is
-> **deferred** with an amended design (needs xpub hooks, not just an ECDH hook).
-> Three follow-on specs were written and **implemented** this pass:
-> **`SYNC_CORRECTNESS_SPEC.md`** (Spec 0 — paginated/high-water sync + contact-profile
-> cache + durable persistence), **`CONTACTINFO_FORMAT_SPEC.md`** (Spec 1 — privateData
-> CBOR→DIP-15 varint), and Spec 2 (Ignore). Also resolved: `accountReference`
-> byte-order (**keep ours** — recipient-ignored one-time pad, no interop break) and
-> the friendship-path `account'` hardcode (fixed upstream in **rust-dashcore#813**,
-> pulled in via the dashcore bump **PR #3936**). Remaining is all blocked on external
-> resources (devnet funding for e2e/UAT; contract governance for cross-device ignore
-> + DoS filter; an upstream rust-dashcore change for multi-account). See
-> the DashPay backlog issue [#4020](https://github.com/dashpay/platform/issues/4020) for the authoritative item-by-item status.
->
-> **How to read.** Part 0 is the TL;DR. Parts 1–2 are reference (protocol +
-> architecture). Part 3 is the current-state inventory. Part 4 is the prioritized
-> gap/bug list. Part 5 is the work plan. Part 6 is the Swift UI design. Part 7 is
-> the test plan. The durable evidence base ships alongside this spec:
-> [`INTEROP_DESK_CHECK.md`](./INTEROP_DESK_CHECK.md) (cross-client wire-format
-> verdicts + testnet census) and [`CONTACTINFO_FORMAT_SPEC.md`](./CONTACTINFO_FORMAT_SPEC.md)
-> (Appendix A). The transient working-research files that once backed the
-> remaining citations were trimmed from the tree; they remain in this branch's
-> git history (`docs/dashpay/research/`, up to the trim commit).
-
----
-
-## Part 0 — Executive summary
-
-### What works today (end-to-end, real broadcast)
-
-- **Profile**: create / update / fetch / sync — `rs-platform-wallet`
- (`network/profile.rs`), FFI, and Swift `DashPayProfileEditorView` are all wired
- and broadcast real document state transitions.
-- **Send contact request**: builds the `contactRequest` doc, ECDH + AES-256-CBC
- encrypts the receiving xpub (via `dash-sdk` → `platform-encryption`), signs,
- broadcasts. Wrapped in Swift (`AddFriendView`). ⚠ **Correction (2026-06-10):
- "works" was overstated — the G2 entropy bug meant every broadcast through this
- path was rejected by consensus until the M1 task-4 fix. The code path existed;
- it did not function. (Exactly what G11's zero-network-tests predicted.)**
-- **Accept contact request**: sends the reciprocal request, decrypts the
- contact's xpub, registers a watch-only sending account. Wired in Swift
- (`ContactRequestRow` Accept).
-- **Send money to a contact**: derives the next contact address, builds + signs +
- broadcasts an L1 tx, records the payment. Wired in Swift
- (`SendDashPayPaymentSheet`).
-- **DIP-14 / DIP-15 derivation**: 256-bit non-hardened child derivation, the
- `m/9'/5'/15'/0'//` friendship path, the two account
- types (`DashpayReceivingFunds`, `DashpayExternalAccount`), gap limit 20 — all
- implemented and test-vector-pinned in `rust-dashcore/key-wallet`.
-- **Persistence**: contacts / profiles / payments round-trip through the
- changeset → SQLite pipeline. SwiftData mirror models exist.
-- **Swift FFI coverage**: all ~14 DashPay/DPNS FFI functions are already wrapped
- on `ManagedPlatformWallet`.
-
-### The gaps that block a *complete, correct* flow (detail in Part 4)
-
-| # | Gap | Severity | Layer |
-|---|-----|----------|-------|
-| G1 | **Sync never builds sending accounts** — a contact who accepts you *while you're offline* has no spendable account after sync; `send_payment` fails until `register_external_contact_account` is manually called | **P0** | `rs-platform-wallet` |
-| G2 | **`send_contact_request` entropy mismatch** — document-ID entropy diverges from the broadcast entropy (`rs-sdk` admits the "simplification"); severity needs verification against `PutDocument` | **P0** | `rs-sdk` |
-| G3 | **`accountReference` hardcoded to 0** — DIP-15 masking unused; the unique index `(ownerId,toUserId,accountReference)` makes key rotation / re-send impossible | **P1** | `rs-platform-wallet` |
-| G4 | **Watch-only wallets can't send/accept** — ECDH is derived from the in-process seed; only `EcdhProvider::SdkSide` is used, the `ClientSide` push-across-FFI path is unbuilt | **P1** | wallet + FFI |
-| G5 | **Reject is local-only** — no tombstone at all (recurring sync would resurrect rejects — stage-1 fix in **M1**) and no `contactInfo` `displayHidden` doc (cross-device — stage 2 in **M3**) | **P1** | wallet + SDK |
-| G6 | **Wrong fallback contract ID** — `rs-sdk` `#[cfg(not(feature="dashpay-contract"))]` path hardcodes the **DPNS** id (dead code in default builds, latent bug) | **P2** | `rs-sdk` |
-| G7 | **Dead code**: `calculate_account_reference`, `validate_contact_request`, auto-accept proof gen/verify — implemented + tested but never called by live paths | **P2** | `rs-platform-wallet` |
-| G8 | **Local sent-request placeholder** — stores `vec![0u8;96]` for `encrypted_public_key` instead of the real ciphertext | **P2** | `rs-platform-wallet` |
-| G9 | **No contract cache** — the bundled system contract is re-loaded on every op | **P2** | `rs-platform-wallet` |
-| G10 | **No `contactInfo` support** — alias/note/hidden private metadata never syncs across devices | **P2** | wallet + SDK |
-| G11 | **Network layer is untested.** Primitives/state/persistence are well covered, but the *whole* `network/` layer (send/sync/accept/pay/profile-broadcast) has **0 tests**; no full send→sync→accept→pay integration test; Swift has **0** DashPay tests | **P0** | both |
-| G12 | **DashPay sync is not in the recurring sync loop.** The background `IdentitySyncManager` syncs **token balances only**; `dashpay_sync()` (contact requests + profiles) runs **only on-demand via FFI** — it must be folded into the recurring loop alongside the other syncs | **P0** | `rs-platform-wallet` |
-| G13 | **Sync never reconciles own sent requests** — after restore-from-seed or on a second device an established contact renders as a mere incoming request; Accept re-broadcasts a duplicate reciprocal and is **rejected forever** by the unique index | **P1** | `rs-platform-wallet` |
-| G14 | **Wrong encrypted-xpub wire format** (desk-check 2026-06-10, `INTEROP_DESK_CHECK.md`): we encrypt the 107-byte DIP-14 `ExtendedPubKey::encode()` instead of DIP-15's **69-byte compact** (`fingerprint‖chaincode‖pubkey`) used by iOS+Android → our send fails its own 96-byte check; our receive can't parse mobile payloads | **P0** | `platform-encryption` + `rs-sdk` + wallet |
-| G15 | **Key-purpose convention mismatch**: mobile clients use key 0 (AUTHENTICATION) for both key indices; our send/validation require ENCRYPTION/DECRYPTION-purpose keys → cross-client requests blocked both directions. Verify against a real testnet mobile contactRequest, then align | **P1** | wallet + `rs-sdk` |
-
-### UI verdict
-
-The Swift DashPay UI exists but is **buried** (Identities → IdentityDetail →
-`Section("DashPay")`) and **utilitarian**. The plan (Part 6) **promotes it to a
-first-class `DashPay` tab**, renders the missing outgoing-requests section, moves
-lists onto reactive `@Query`, and polishes styling (AsyncImage avatars, empty
-states, toasts). No new happy-path FFI is required.
-
-### Recommended sequencing
-
-**Milestone 1 (correctness)**: G11-seam, G12, G1+G13 (+G5 tombstone), G2, interop
-desk-check, G11-Rust. → the offline-accept→pay path works, is integration-tested,
-and the background sync is wired.
-**Milestone 2 (UI)**: first-class DashPay tab + polish + Swift tests (Part 6/7).
-**Milestone 3 (spec-completeness)**: G3, G5, G10 (accountReference + contactInfo
-for rotation/hide/alias sync).
-**Milestone 4 (hardening)**: G4 (watch-only ECDH), G6–G9 cleanup.
-**Milestone 5 (invitations)**: asset-lock voucher + claim + auto-accept wiring
-(new scope 2026-06-10; design pass first).
-
----
-
-## Part 1 — What DashPay is, and the layered stack
-
-**DashPay** (DIP-0015) is a Dash Platform application that creates *bidirectional
-direct settlement payment channels* between two Dash **identities**. User-facing
-model:
-
-- **Username** → a **DPNS** name (DIP-0012) resolving to an identity. DashPay
- itself never stores usernames; it references identities by their 32-byte id.
-- **Identity** (DIP-0011) → the cryptographic actor; holds keys + credit balance;
- signs all state transitions.
-- **Profile** → public presentation (`displayName`, `publicMessage`, avatar).
-- **Contact / friend** → an identity you have exchanged `contactRequest`
- documents with **in both directions**.
-- **Pay a contact** → decrypt the xpub from *their* contactRequest addressed to
- you, derive the next L1 address, and pay it with an ordinary Dash transaction.
- DashPay is the *key-sharing / coordination* layer; value transfer is plain L1.
-
-### The implementation stack (bottom → top)
-
-```
-┌──────────────────────────────────────────────────────────────────────────┐
-│ SwiftExampleApp Views: FriendsView, IdentityDetailView (profile), │
-│ (packages/swift-sdk/ SendDashPayPaymentSheet, AddFriendView [Part 6] │
-│ SwiftExampleApp) State: PlatformWalletManager, AppState, SwiftData │
-├──────────────────────────────────────────────────────────────────────────┤
-│ swift-sdk wrappers ManagedPlatformWallet.{sync,send,accept,reject,pay, │
-│ (Sources/SwiftDashSDK) profile…}, ContactRequest, EstablishedContact, │
-│ DashPayProfile, KeychainSigner (thin, marshal-only)│
-├──────────────────────────────────────────────────────────────────────────┤
-│ rs-platform-wallet-ffi C ABI: platform_wallet_{sync_contact_requests, │
-│ send_contact_request_with_signer, accept…, reject…, │
-│ send_dashpay_payment, *_dashpay_profile_with_signer} │
-├──────────────────────────────────────────────────────────────────────────┤
-│ rs-platform-wallet IdentityWallet (network façade): contact_requests.rs,│
-│ (the brains) contacts.rs, payments.rs, profile.rs, dashpay_sync.rs│
-│ ManagedIdentity state; crypto/{dip14,validation,…} │
-├───────────────────────────────┬──────────────────────────────────────────┤
-│ rs-sdk (dash-sdk) │ platform-encryption │
-│ dashpay/contact_request.rs: │ derive_shared_key_ecdh (libsecp256k1 ECDH),│
-│ create/send_contact_request, │ encrypt/decrypt_extended_public_key │
-│ EcdhProvider, queries │ (AES-256-CBC + PKCS7), account-label crypto │
-├───────────────────────────────┴──────────────────────────────────────────┤
-│ rust-dashcore/key-wallet DIP-9 paths, DIP-14 256-bit CKD, AccountType │
-│ (HD wallet primitives) ::Dashpay{ReceivingFunds,ExternalAccount}, │
-│ managed accounts, gap limit 20, tx checking │
-├──────────────────────────────────────────────────────────────────────────┤
-│ dashpay-contract v1 schema: profile / contactRequest / │
-│ contactInfo; id Bwr4WHCP…NS1C7 │
-└──────────────────────────────────────────────────────────────────────────┘
-```
-
-Key architectural facts:
-- **ECDH + AES live in `platform-encryption`**, *not* in `key-wallet`
- (rust-dashcore has **zero** ECDH code). The wallet calls `dash-sdk`'s
- `send_contact_request`, which calls `platform-encryption` internally; the
- receive/decrypt path calls `platform-encryption` directly.
-- **`key-wallet` only consumes an already-decrypted friend xpub**
- (`wallet_add_dashpay_external_account_with_xpub_bytes`).
-- **The DashPay system contract is bundled** (`load_system_data_contract`), no
- network fetch needed.
-- **Swift never orchestrates** — every multi-step DashPay op is *one*
- `platform-wallet` FFI call (per `swift-sdk/CLAUDE.md`).
-
----
-
-## Part 2 — Protocol reference (authoritative numbers)
-
-Condensed from the DIPs themselves (DIP-9/11/13/14/15,
-cross-checked against the deployed v1 contract). **Where DIP prose and the v1
-schema disagree, the schema wins.**
-
-### 2.1 Friendship lifecycle
-
-- A `contactRequest{ $ownerId: sender, toUserId: recipient }` is **one-directional**.
-- **Established contact (DIP-15: "friendship") = both directions exist** (A→B *and*
- B→A). One request = "pending"; the reciprocal = "accept".
-- **To pay X**, read **X's** request addressed to you (`$ownerId==X`,
- `toUserId==you`), decrypt its `encryptedPublicKey`, derive addresses.
-- Contact requests are **immutable & non-deletable** (`documentsMutable:false`,
- `canBeDeleted:false`). Key rotation = a **new** request with a bumped
- `accountReference` version.
-
-### 2.2 Friendship derivation path (DIP-15 + DIP-14)
-
-```
-m / 9' / 5' / 15' / 0' / / / index
- └─────── hardened ──────┘ └── non-hardened 256-bit (DIP-14) ──┘ └ non-hardened u32
-```
-
-- `9'`=feature purpose, `5'`=Dash (`1'` testnet), `15'`=DashPay, `0'`=account.
-- The two 256-bit levels are the raw 32-byte identity ids (owner first). **Must**
- stay non-hardened (so a watch-only xpub at `…/0'` covers all contacts) and full
- 256-bit (truncating to 31 bits is a DIP-14 security violation).
-- Auto-accept proof keys use a **separate** path `m/9'/5'/16'/'`.
-
-### 2.3 ECDH shared secret
-
-libsecp256k1 ECDH (**not** raw X-coord): `sharedKey = SHA256( ((y[31]&1)|2) || x )`
-of the shared point `d_self · Q_other`. The participating identity keys are
-selected by `senderKeyIndex` / `recipientKeyIndex` (identity public-key `id`s,
-encryption/decryption purpose). Both parties derive the identical 32-byte key →
-the AES-256 key.
-
-### 2.4 Encryption layout
-
-- Plaintext = **compact** xpub `parentFingerprint(4) || chainCode(32) ||
- pubKey(33)` = **69 bytes** (not the 78-byte BIP32 xpub). *(Implementation note —
- corrected by the 2026-06-10 desk-check: our stack actually fed
- `ExtendedPubKey::encode()` in, which for the DashPay path is the **107-byte**
- DIP-14 form → 128-byte ciphertext → our own send path failed. See **G14**;
- reference clients confirm the 69-byte compact form.)*
-- `encryptedPublicKey` = `IV(16) || AES-256-CBC-PKCS7(80)` = **exactly 96 bytes**.
-- `encryptedAccountLabel` = `IV(16) || ciphertext(32–64)` = **48–80 bytes**.
-- `contactInfo.privateData` uses **BIP32-derived** symmetric keys (self-encrypt),
- not ECDH; `encToUserId` uses AES-ECB.
-
-### 2.5 `accountReference`
-
-```
-ASK = HMAC-SHA256(senderSecretKey, extendedPublicKey)
-AccountRef = (Version << 28) | (ASK[28 msb] XOR (Account & 0x0FFFFFFF))
-```
-Top 4 bits = version (rotation signal), low 28 bits = account number masked by a
-PRF of the xpub. Uniqueness not required. The recipient un-masks the account and
-reads the version.
-
-### 2.6 DashPay v1 contract document types
-
-Contract id **`Bwr4WHCPz5rFVAD87RqTs3izo4zpzwsEdKPWUT1NS1C7`**
-(hex `a2a1…71bc`), owner all-zero. Full field/index tables in the deployed
-schema, `packages/dashpay-contract/schema/v1/dashpay.schema.json`. Summary:
-
-- **`profile`**: `avatarUrl`(uri,≤2048), `avatarHash`(32B), `avatarFingerprint`(8B),
- `publicMessage`(1–140), `displayName`(1–25). Avatar trio is `dependentRequired`.
- Unique index `$ownerId`; non-unique `$ownerId+$updatedAt`. Mutable.
-- **`contactRequest`**: `toUserId`(32B id), `encryptedPublicKey`(**exactly 96B**),
- `senderKeyIndex`, `recipientKeyIndex`, `accountReference`, optional
- `encryptedAccountLabel`(48–80B), optional `autoAcceptProof`(38–102B, unencrypted).
- Required system fields incl. `$createdAtCoreBlockHeight`. Unique index
- `$ownerId+toUserId+accountReference`; timelines `toUserId+$createdAt` (received)
- and `$ownerId+$createdAt` (sent). Immutable.
-- **`contactInfo`**: `encToUserId`(32B), `rootEncryptionKeyIndex`,
- `derivationEncryptionKeyIndex`, `privateData`(48–2048B encrypted; **DIP-15
- varint** `version`/`aliasName`/`note`/`displayHidden`/`acceptedAccounts` —
- contract enforces length only, see `CONTACTINFO_FORMAT_SPEC.md`). Unique index
- `$ownerId+root+derivation`. Privacy rule: don't publish until ≥2 established
- contacts.
-
----
-
-## Part 3 — Current implementation state (inventory)
-
-Master status matrix. **Legend:** ✅ implemented · 🟡 partial/caveated · ❌ missing.
-Citations are abbreviated (file:line against this branch).
-
-### 3.1 rust-dashcore `key-wallet` (HD primitives)
-
-| Capability | Status | Evidence |
-|---|---|---|
-| DIP-9 DashPay root `m/9'/5'/15'` (`/1'` testnet) | ✅ | `key-wallet/src/dip9.rs:167-198` |
-| DIP-14 256-bit non-hardened CKD (priv+pub) | ✅ | `bip32.rs:575-598,1533-1589,1817+`; vectors `:2521-2594` |
-| `AccountType::DashpayReceivingFunds` / `DashpayExternalAccount` | ✅ | `account/account_type.rs:76-95,469-514` |
-| Managed accounts, single pool, **gap limit 20** | ✅ | `managed_account_type.rs:97-118,706-749` |
-| Tx checking routes contact funds | ✅ | `transaction_checking/account_checker.rs:501-518` |
-| FFI: add receiving / add external(xpub) / get | ✅ | `key-wallet-ffi/src/wallet.rs:397,451`, `managed_account.rs:436,497` |
-| ECDH / shared secret / xpub encryption | ❌ (by design — lives in `platform-encryption`) | repo-wide grep: none |
-| Auto-create DashPay accounts at wallet init | ❌ (per-contact, after the fact) | `wallet/initialization.rs` |
-| Match result carries identity ids | 🟡 (only `account_index`; reverse-lookup needed) | `account_checker.rs:144-153` |
-
-### 3.2 `platform-encryption` + `rs-sdk` (crypto + send flow)
-
-| Capability | Status | Evidence |
-|---|---|---|
-| `derive_shared_key_ecdh` (libsecp256k1) | ✅ | `rs-platform-encryption/src/lib.rs:24-34` |
-| `encrypt/decrypt_extended_public_key` (AES-256-CBC, IV-prepend, 96B) | ✅ | `lib.rs:97-128` |
-| `encrypt/decrypt_account_label` (48–80B) | ✅ | `lib.rs:139-171` |
-| `Sdk::create_contact_request` / `send_contact_request` | ✅ | `rs-sdk/src/platform/dashpay/contact_request.rs:164,378` |
-| `EcdhProvider::{ClientSide, SdkSide}` | ✅ (both defined; only SdkSide used upstream) | `contact_request.rs:31-54` |
-| Queries: sent / received / all contact requests | ✅ | `contact_request_queries.rs:33,76` |
-| SDK helpers for `profile` / `contactInfo` | ❌ (done via generic `Document`+`PutDocument`) | — |
-| **Bug: send entropy ≠ doc-id entropy** | 🟡 **G2** | `contact_request.rs:431-435` (code comment admits it) |
-| **Bug: fallback contract id = DPNS id** | 🟡 **G6** (dead in default build) | `dashpay/mod.rs:33` |
-
-### 3.3 `rs-platform-wallet` (+ FFI, storage) — the brains
-
-| Flow | Status | Evidence |
-|---|---|---|
-| Identity ↔ wallet (managed identities) | ✅ | `state/managed_identity/mod.rs:37`, `network/identity_handle.rs:256` |
-| Profile fetch / sync | ✅ | `network/profile.rs:64,145` |
-| Profile create / update (external signer) | ✅ | `network/profile.rs:240,395` |
-| `dashpay_sync` aggregator | ✅ | `network/dashpay_sync.rs:16` |
-| Sync received contact requests | 🟡 **G1** (ingest guard drops reciprocals; no xpub-decrypt / no external account built) | `network/contact_requests.rs:322,367-372` |
-| Sync own sent requests (restore/multi-device reconcile) | ❌ **G13** | sync calls `fetch_received_contact_requests` only |
-| Send contact request (seed-in-process) | ✅ / 🟡 **G4** | `network/contact_requests.rs:91` |
-| Accept (reciprocal send + register external account) | ✅ | `network/contact_requests.rs:466` |
-| Reject | 🟡 **G5** (local-only) | `network/contact_requests.rs:678` |
-| Auto-establish on reciprocal match | ✅ | `state/managed_identity/contact_requests.rs` |
-| Register receiving / external account | ✅ (UAT 2026-06-12: now also **persisted** — registrations were in-memory only, so accounts vanished on relaunch and restored friendship UTXOs were dropped `dropped_no_account`) | `network/contacts.rs` |
-| Send money to contact | ✅ | `network/payments.rs:93` |
-| Record incoming payment | ✅ (UAT 2026-06-12: the old `try_record_incoming_payment` had **zero callers** — receiver history was always empty. Replaced by live recording in the wallet-event adapter + an idempotent `reconcile_incoming_payments` step in the recurring sync) | `network/payments.rs`, `changeset/core_bridge.rs` |
-| Crypto: DIP-14 xpub / payment addrs | ✅ | `crypto/dip14.rs` |
-| Crypto: `accountReference` | 🟡 **G3/G7** (correct but unused; send hardcodes 0) | `crypto/dip14.rs:147` |
-| Crypto: auto-accept proof | 🟡 **G7** (dead code, `// TODO` at `auto_accept.rs:39`) | `crypto/auto_accept.rs` |
-| Pre-send validation | 🟡 **G7** (never called) | `crypto/validation.rs:76` |
-| Persistence round-trip | ✅ | `wallet/apply.rs`, storage `schema/{contacts,dashpay}.rs` |
-| Local placeholder `encrypted_public_key` | 🟡 **G8** (`vec![0u8;96]`) | `network/contact_requests.rs:283` |
-| Contract cache | 🟡 **G9** (re-load per call) | `network/profile.rs:83` |
-| FFI surface (sync/send/accept/reject/pay/profile) | ✅ | `ffi/src/{dashpay,dashpay_profile,contact_request,established_contact,contact}.rs` |
-
-> **No `todo!()`/`unimplemented!()`/`unreachable!()` anywhere in DashPay paths** —
-> all gaps are caveats, dead helpers, or local-only fallbacks, not panics.
-
-### 3.4 SwiftExampleApp + swift-sdk
-
-| Capability | Status | Evidence |
-|---|---|---|
-| All ~14 DashPay/DPNS FFI functions wrapped | ✅ | `Sources/SwiftDashSDK/PlatformWallet/ManagedPlatformWallet.swift:1452-1779` |
-| Wrapper objects: `ContactRequest`, `EstablishedContact`, `DashPayProfile` | ✅ | same dir |
-| SwiftData mirrors: `PersistentDashpayProfile`, `PersistentDashpayContactRequest` | ✅ | `Persistence/Models/` |
-| Contacts list + incoming requests + accept/reject | ✅ (utilitarian) | `Views/FriendsView.swift` |
-| Add friend by DPNS name / identity id | ✅ | `FriendsView.swift` (`AddFriendView`) |
-| Send money to contact sheet | ✅ (most polished) | `FriendsView.swift` (`SendDashPayPaymentSheet`) |
-| Profile view / editor (DIP-15 avatar hashing) | ✅ | `Views/IdentityDetailView.swift:332,1169` |
-| First-class DashPay tab | ❌ **(Part 6)** buried under Identities | `ContentView.swift` |
-| Outgoing requests rendered | ❌ (loaded, not shown) | `FriendsView.swift` |
-| Lists driven by reactive `@Query` | ❌ (reads live Rust snapshot) | `FriendsView.swift` |
-| DashPay tests (unit / XCUITest) | ❌ **G11** | `SwiftTests/`, `SwiftExampleAppUITests/` |
-
----
-
-## Part 4 — Gap analysis & bugs (prioritized)
-
-### P0 — blocks a correct, complete flow
-
-**G1 — Sync cannot establish contacts, and never builds sending accounts.**
-Two compounding defects in `sync_contact_requests` (`network/contact_requests.rs:322`):
-(1) **the ingest guard drops reciprocal requests** — any received doc whose sender is
-already in `sent_contact_requests` is skipped (`:367-372`), so in the offline-accept
-scenario the reciprocal request never reaches `add_incoming_contact_request` (the
-only auto-establish trigger) and the contact stays pending-sent forever; (2) even
-for contacts that do establish, sync stores the *encrypted* `encryptedPublicKey` and
-never decrypts it or registers a `DashpayExternalAccount` — only the explicit
-**accept** path does. **Consequence:** "they accepted me → I sync → I pay them"
-fails twice over: the contact never establishes via sync, and `send_payment`
-(`network/payments.rs:135`) has no sending account. **Fix:** (a) relax the ingest
-guard so a received doc whose sender matches a `sent_contact_requests` entry flows
-into `add_incoming_contact_request` (which auto-establishes and collapses the
-pending entries); (b) on every sync pass, for **every established contact missing an
-external account** (not only newly-established ones — this also repairs contacts
-left unpayable by the accept path's best-effort registration), validate the
-request's key indices via `validate_contact_request` (purpose ENCRYPTION/DECRYPTION
-+ ECDSA key type — never ECDH against an unvalidated index; an attacker-crafted
-index pointing at an AUTHENTICATION key silently derives a wrong shared secret and
-poisons the account), then decrypt the xpub and register the account — and likewise
-register a missing **`DashpayReceivingFunds`** account (derivable from the wallet's
-own seed, no decryption needed): it is what makes *incoming* contact payments
-visible to SPV, its only creation point today is the fresh-send path
-(`contact_requests.rs:300`), and after restore-from-seed nothing rebuilds it —
-incoming payments would land on unwatched addresses; (c) **failure
-policy** — distinguish transient failures (network: retry next sweep) from permanent
-ones (decrypt/decode failure: mark the contact "payment channel broken", surface to
-FFI/UI, skip until the request changes — no unbounded retry). Must be seed-aware
-(skip + log for watch-only until G4).
-
-**G2 — `send_contact_request` entropy mismatch.**
-`rs-sdk/.../contact_request.rs:431-435`: `create_contact_request` computes the
-document id from entropy E1, but `send_contact_request` generates *fresh* entropy
-E2 for `put_to_platform_and_wait_for_response`. The code comment admits the
-"simplification". **Action:** verify whether `PutDocument` re-derives the id from
-E2 (in which case the returned `ContactRequestResult.id` is merely *stale*, a
-correctness wart) or whether the broadcast actually fails / duplicates. Thread the
-*same* entropy through both. Pin with a test asserting `result.id == on-platform id`.
-
-**G11 — Test coverage (precise breakdown).**
-What's **well covered** (≈60 unit tests): crypto (`crypto/dip14.rs` ×10,
-`validation.rs` ×8, `auto_accept.rs` ×6), the contact state machine
-(`state/managed_identity/contact_requests.rs` ×12, `mod.rs` ×8), the DashPay types
-(`established_contact`/`profile`/`contact_request`/`payment`), and persistence
-(`wallet/apply.rs` ×26). Plus `tests/contact_workflow_tests.rs` (8 tests) — but
-those are **pure in-memory handshake** tests using `noop_persister()` and **fake
-identities** (`data: vec![1u8;33]`, not real keys), so they exercise the state
-machine, *not* real ECDH/derivation/broadcast.
-
-What's **completely untested**: the entire **`network/` layer** — the actual
-broadcast/sync/pay paths. `grep #[test] src/wallet/identity/network/` → only
-`registration.rs` has one. **Zero** tests for
-`send_contact_request_with_external_signer`, `sync_contact_requests`,
-`sync_profiles`, `accept_contact_request_with_external_signer`,
-`register_external_contact_account`, `send_payment`, `create/update_profile`. And
-**zero** DashPay tests in Swift. This is a quality-P0: the flow cannot be declared
-"done and tested" without (a) network-layer tests via a mock SDK/broadcaster seam,
-and (b) a real devnet/regtest end-to-end test. (Test plan: Part 7.)
-
-**G12 — DashPay sync is not in the recurring sync loop.**
-The background `IdentitySyncManager` (`manager/identity_sync.rs`, owned by
-`PlatformWalletManager` at `manager/mod.rs:54,139`, run as a cancel-token loop with
-a configurable interval and a re-entrancy guard) syncs **token balances only**.
-`dashpay_sync()` (= `sync_contact_requests()` + `sync_profiles()`) is invoked
-**only on demand via FFI** — i.e. the Swift app must poll it. There is **no
-recurring DashPay refresh**. **Fix (the constraints matter more than the
-placement):** `dashpay_sync()` is a method on `IdentityWallet` (needs the
-wallet-manager lock, per-wallet persister, broadcaster), while `IdentitySyncManager`
-is deliberately self-contained (constructed with sdk + persister only; documented as
-not reaching into PlatformWallet/WalletManager) and its registry **skips identities
-with empty token lists** — so the recurring DashPay pass must **NOT** be driven "per
-registered identity" off the token registry (a DashPay-only identity with no watched
-tokens would never sync). Instead, inject the wallets map (the same
-`Arc>>>` that
-`PlatformAddressSyncManager` already receives at `manager/mod.rs:135-138`; snapshot
-the wallet `Arc`s under a read guard per sweep) and iterate wallets calling
-`wallet.identity().dashpay_sync()` per pass, reusing the existing
-cadence/cancel/quiesce/re-entrancy machinery. Whether this lives inside
-`IdentitySyncManager` or as a sibling `DashPaySyncManager` is an implementation
-detail; coupling DashPay sync to the token registry is the failure mode to avoid.
-**Error semantics:** log-and-continue per wallet/identity (matching the existing
-loop's contract) — never fail-fast across identities. Note: wrapping `dashpay_sync`
-alone only delivers per-*wallet* continue — `sync_contact_requests` currently
-`?`-aborts its multi-identity loop on the first fetch error and `dashpay_sync`
-propagates immediately, so the per-identity policy must be implemented *inside*
-those loops. This recurring pass is the
-natural home for the **G1** establish/decrypt/register sweep, the **G13** sent-side
-reconcile, and the **G5** tombstone check. Keep the on-demand FFI entry points for
-pull-to-refresh.
-
-### P1 — spec-correctness / production-readiness
-
-**G3 — `accountReference` hardcoded to 0.** The send path uses
-`account_reference = 0` instead of `calculate_account_reference(...)`
-(`crypto/dip14.rs:147`). Because the unique index is
-`(ownerId, toUserId, accountReference)`, a second request to the same recipient
-(key rotation, multi-account) **collides** and is rejected by Platform. Today this
-"works" only because the full account xpub is shared directly (recipient decrypts
-it and doesn't need to un-mask the account). **Fix:** wire
-`calculate_account_reference` into the send path; add the version-bump path for
-rotation; have the receive path tolerate / surface non-zero versions (DIP-15 §7.3
-"sender rotated their addresses" notification). **Receive-side scope note (budget
-into M3):** the in-memory maps, changeset keys, and SQLite contacts schema are
-keyed by counterparty id alone, and the sync ingest guard skips requests from
-already-established contacts — surfacing rotation requires re-keying
-contact-request state + persistence by `(counterparty, accountReference)` and
-letting rotation requests from established contacts through the guard. Without
-this, `dp_005`'s "receive path surfaces the rotation" assertion is unimplementable.
-
-**G4 — Watch-only wallets can't send/accept.** ECDH is derived from the
-in-process seed (`identity_handle.rs:424`); only `EcdhProvider::SdkSide` is used.
-For hardware/watch-only wallets, the `ClientSide` path (host supplies the shared
-secret) must be pushed across the FFI. **Fix:** add an FFI/signer hook that returns
-the ECDH shared secret for `(senderKeyIndex, recipientPubKey)` from the secure
-element, and route `send_contact_request` / `register_external_contact_account`
-through `EcdhProvider::ClientSide`. *(The example app holds the seed, so this is
-not a demo blocker — but it is the right architecture.)* **FFI-hook design lands in
-M3** (design-only task) so the wallet API doesn't churn in M4: the hook accepts
-only the 32-byte ECDH **shared secret** from the host — never the sender's identity
-private key across the ABI (the existing `rs-sdk-ffi`
-`DashSDKContactRequestParams.sender_private_key` field is the antipattern to avoid,
-and worth auditing in its own right).
-
-**G5 — Reject is local-only, and the recurring loop will undo it.**
-`reject_contact_request` (`network/contact_requests.rs:678`, `// TODO` at `:703`)
-drops the local entry but writes no tombstone of any kind — and the still-on-platform
-immutable document is re-ingested as a fresh incoming request on the next sync.
-Today this is masked because sync is on-demand only; **the moment G12 lands, every
-background sweep resurrects every rejected request on the same device.** **Fix (two
-stages):** **M1 (with G12):** a locally-persisted rejected-request tombstone
-consulted by the sync ingest path — keyed by **document id (or
-`(sender, accountReference)`)**, NOT bare sender id: requests are immutable, so the
-only legitimate way a once-rejected sender can ever re-request is a new doc with a
-bumped `accountReference` (the rotation mechanism), and a sender-keyed tombstone
-would silently block that forever with no un-reject affordance. Pinning test:
-"rejected request does not reappear after a recurring re-sync — and a
-bumped-`accountReference` request from the same sender *does*". **M3:** the
-on-platform `contactInfo` `displayHidden` write (see G10) for cross-device sync.
-
-**G13 — Sync never reconciles your own sent requests.** Sync only calls
-`fetch_received_contact_requests`; the identity's own sent requests are never
-fetched into state (the `fetch_sent_contact_requests` query exists but is
-read-only). After restore-from-seed or on a second device, a mutually-established
-contact renders as a mere incoming request; tapping Accept re-broadcasts a duplicate
-reciprocal with the same `(ownerId, toUserId, accountReference)` triple, which
-Platform rejects on the unique index — **Accept fails forever with no recovery
-path**. **Fix (M1):** sync also fetches the identity's own sent contactRequest
-documents and ingests them via `add_sent_contact_request` (auto-establish fires when
-both sides are present) — **with a sent-side ingest guard symmetric to the received
-side**: skip docs whose recipient is already in `sent_contact_requests` or
-`established_contacts` (`add_sent_contact_request` has no such guard today, so an
-unguarded recurring re-ingest creates phantom pending-sent rows + a changeset write
-per contact per sweep). Any (re-)establish path must **merge into an existing
-`EstablishedContact`** — `EstablishedContact::new` resets alias/note/is_hidden/
-accepted_accounts, so naive re-establish wipes user metadata every sweep. Accept
-detects an existing on-platform reciprocal and adopts it instead of re-broadcasting;
-"adopt" includes the local registrations a fresh send performs (receiving-account
-registration, G1(b)) and runs the same `validate_contact_request` gate before any
-`register_external_contact_account` call — the gate applies to **all three paths**
-(sync sweep, normal Accept, Accept-adopt). *Pin:* "established contact is stable
-and metadata-preserving across two recurring sweeps".
-
-**G14 — Wrong encrypted-xpub wire format (P0; found by the M1 desk-check,
-`INTEROP_DESK_CHECK.md`).** DIP-15 and BOTH reference clients
-(iOS dash-shared-core `ecdsa_key.rs:333-341`, Android dashj
-`serializeContactPub()` with a hard `len == 69` receive check) use the compact
-**69-byte** plaintext `parentFingerprint(4) ‖ chainCode(32) ‖ pubKey(33)`. Our
-stack fed `ExtendedPubKey::encode()` into `encrypt_extended_public_key` — and the
-DashPay account xpub ends in a `Normal256` child, so that's the **107-byte
-DIP-14** serialization → 128-byte ciphertext → fails our own `== 96` assertion
-and the contract's `maxItems: 96`. **Consequence:** our send path errored before
-broadcast (nothing nonconforming reached chain — blast radius ≈ zero), and our
-receive (`ExtendedPubKey::decode`, 78/107 only) rejects every mobile payload and
-would mark the channel permanently broken. **Fix (M1 task 7):** compact 69-byte
-assembly on send + compact parser on receive (path context reconstructs
-depth/child-number); byte-exact vectors from the reference clients.
-
-**G15 — Key-purpose convention mismatch (P1; same desk-check).** Mobile clients
-populate `senderKeyIndex`/`recipientKeyIndex` with key id 0 — an
-**AUTHENTICATION**-purpose ECDSA key — while we *send* selecting
-ENCRYPTION/DECRYPTION-purpose keys and *validate* (G1(b)) requiring those
-purposes. Cross-client requests would be blocked in both directions. **Action
-(M1 task 8):** verify empirically against a real testnet mobile contactRequest,
-then align — liberal-on-receive (accept the purposes mobile actually uses; keep
-the ECDSA key-*type* gate), compatible-on-send (fall back to the mobile
-convention when the recipient lacks a DECRYPTION-purpose key).
-
-### P2 — cleanup / completeness
-
-- **G6 — Wrong fallback contract id** (relabeled P2 — dead code in default builds):
- `rs-sdk/.../dashpay/mod.rs:33` hardcodes the **DPNS** id under
- `#[cfg(not(feature="dashpay-contract"))]` — a latent foot-gun. **Fix:** correct
- the constant to the DashPay id `Bwr4WHCP…NS1C7` or delete the fallback.
-
-- **G7 — Dead code:** wire `validate_contact_request` into the send path (replace
- the ad-hoc `find(...)`) — note the *receive/sync* side of this validator is pulled
- forward into **M1** by G1(b); decide whether to ship auto-accept (then call the
- proof gen/verify) or delete it and its FFI param. **Acceptance criterion if
- shipped:** any handler acting on `autoAcceptProof` MUST call
- `verify_auto_accept_proof` before triggering automatic acceptance — sync stores
- the blob unverified today, so wiring auto-accept without the gate lets forged
- 38–102-byte blobs auto-establish attacker contacts.
-- **G8 — Local placeholder:** store the real 96-byte ciphertext on the local sent
- `ContactRequest` (`contact_requests.rs:283`) so the persisted/SwiftData row
- matches Platform.
-- **G9 — Contract cache:** hold one `Arc` on the wallet instead of
- re-loading the bundled contract per op.
-- **G10 — `contactInfo` support:** add SDK + wallet + FFI for the `contactInfo`
- document (self-encrypted alias/note/displayHidden/acceptedAccounts) so contact
- metadata and hides sync across a user's devices. Respect the "≥2 contacts before
- publishing" privacy rule.
-- **Compact-xpub note:** DIP-15 specifies a 69-byte compact xpub plaintext;
- `platform-encryption` currently encrypts a 78-byte serialization (still 96 bytes
- out). Both sides of *this* implementation agree, so it interoperates with itself;
- verify against the reference DashPay clients (iOS/Android) before declaring
- cross-client compatibility. **Sequencing:** the desk-check (compare reference
- client code or captured vectors) is cheap and lands in **M1** (task 5) — a wrong
- wire format must be caught before three milestones of tests harden it; the live
- cross-client e2e stays in M4.
-
----
-
-## Part 5 — Work plan (what to build, per milestone)
-
-Each task notes the layer and the **test** that proves it (TDD: write the failing
-test first — see Part 7 and the repo's TDD discipline).
-
-### Milestone 1 — Correctness (the flow actually completes)
-
-Ordered so the test seam exists before the TDD-gated tasks that need it.
-
-1. **G11-seam: make the network layer testable.** The **fetch half needs no new
- seam** — use the SDK's built-in mock (`SdkBuilder::new_mock` +
- `expect_fetch`/`expect_fetch_many`, already used by `identity_sync.rs` tests)
- for the sync/establish tests below. The **put/broadcast half** cannot hide
- behind a dyn trait (`Sdk::send_contact_request` is generic over 7 type params):
- define ONE object-safe trait exposing only the concrete operations
- `IdentityWallet` performs (send contact request with SdkSide ECDH, put profile
- document), held as a new `IdentityWallet` field defaulting to an
- `Arc`-backed impl so public construction and FFI are untouched.
- (`send_payment` already takes an injected `broadcaster: B`.)
- **DONE (2026-06-10; revised 2026-07-04):** originally shipped as a
- Send-boxed `#[async_trait]` `DashPaySdkWriter` trait; the trait was later
- removed as an unused seam (no test ever injected it — the sync/establish
- tests mock the fetch half via `SdkBuilder::new_mock` instead).
- `network/sdk_writer.rs` now ships a concrete `SdkWriter` held as an
- `Arc`-backed `IdentityWallet` field: it still erases the
- 7-type-param `send_contact_request` / `PutDocument` generics behind two
- concrete methods, it just isn't swappable.
-2. **G12: fold DashPay sync into the recurring loop.** Per G12: inject the wallets
- map and iterate wallets calling `dashpay_sync()` — do **not** drive off the
- token registry; log-and-continue error semantics; keep the on-demand FFI entry
- points for pull-to-refresh.
- - *Test:* a recurring pass drives DashPay sync for every wallet **including
- identities with zero watched tokens**; re-entrancy + quiesce still hold.
- **DONE (2026-06-10):** sibling **`DashPaySyncManager`** (`manager/dashpay_sync.rs`,
- modeled on `PlatformAddressSyncManager`) — the in-struct option was rejected
- because `IdentitySyncManager` is registry-driven. Red→green pinned by
- `recurring_pass_syncs_every_wallet_including_zero_token_identities`; per-identity
- continue pushed into `sync_contact_requests`.
-3. **G1 + G13 + G5-tombstone: sync establishes, reconciles, and builds accounts.**
- Relax the ingest guard (G1a); ingest own sent requests (G13); consult the
- persisted rejected-senders tombstone (G5 stage 1); then, for every established
- contact missing an external account: validate key indices → decrypt → register
- (G1b), with the transient/permanent failure policy (G1c). **Lock ordering:**
- collect candidates while the wallet-manager write guard is held, drop the
- guard, then call `register_external_contact_account` — it re-acquires read
- locks on the same tokio `RwLock`, which is **non-reentrant**; calling it inline
- under the write guard deadlocks on first execution (mirror the accept path's
- guard-drop ordering — `network/contact_requests.rs:466` drops the write guard
- before calling `register_external_contact_account`).
- - *Tests:* offline-accept→pay (Part 7, `dp_004`); rejected request does NOT
- reappear after a recurring re-sync; restore-from-seed then Accept does not
- double-broadcast (G13); permanent decrypt failure marks the contact unpayable
- and stops retrying.
- **DONE (2026-06-10):** tombstone keyed `(owner, sender, accountReference)`
- (new `rejected_contact_requests` SQLite table + `ContactChangeSet.rejected`);
- broken channel = `EstablishedContact.payment_channel_broken` (new `contacts`
- column + FFI accessor `established_contact_is_payment_channel_broken`);
- metadata-preserving `reestablish_preserving_metadata()`; sweep candidates
- collected under the write guard, registered after guard drop. 192 tests green.
- *Deviations:* (1) tests pin the decision logic + state machine; the full
- mock-SDK offline-accept→pay and accept-adopt flows live in `dp_004`/`dp_005`
- on #3549 (per Part 7.4) — too heavy to stub as unit tests; (2) the Swift
- persister bridge does NOT yet project `rejected` / `payment_channel_broken` —
- added to M2 plumbing (task 8).
-4. **G2: entropy threading.** Fix `rs-sdk` `send_contact_request` to reuse the
- creation entropy; assert returned id == on-platform id.
- - *Test:* `rs-sdk` unit/integration pinning id equality.
- **DONE (2026-06-10) — severity verdict: REAL BROADCAST BUG.** `put_document`
- uses the document as-is (E1-derived id) with the supplied fresh entropy E2;
- drive-abci consensus recomputes `generate_document_id_v0(…, E2)`, compares to
- `base.id`, and rejects with `InvalidDocumentTransitionIdError` — **every
- `send_contact_request` through this path failed at consensus**. Fix: additive
- `entropy: Bytes32` on `ContactRequestResult`, reused by send; pinned by
- `contact_request_result_entropy_derives_returned_id` (red = inexpressible
- pre-fix; green post-fix). 136 lib + all test targets green; FFI ABI unchanged.
-5. **Interop desk-check (verify-only).** Compare the compact-xpub plaintext
- (69B DIP-15 vs 78B current), ECDH derivation, and accountReference masking
- against reference DashPay iOS/Android client code or captured vectors; record
- the result. A mismatch found here re-scopes M1 before tests harden the wrong
- format; the live cross-client e2e stays in M4.
- **DONE (2026-06-10)** — `INTEROP_DESK_CHECK.md`. Verdicts:
- xpub plaintext **FAIL** (→ new **G14**, task 7 below); ECDH **PASS**;
- accountReference **PASS-for-now** (mobile ignores it on receive; our masking
- helper has two latent bugs for M3 — 107-byte HMAC input + ASK28 byte order,
- where iOS and Android also disagree with *each other*). Bonus hazard → **G15**
- (key-purpose convention).
-7. **G14: compact-xpub wire format (re-scoped into M1 by task 5).** Send: assemble
- the 69-byte compact plaintext (`parentFingerprint(4) ‖ chainCode(32) ‖
- compressedPubKey(33)`) from the already-derived contact xpub instead of
- `ExtendedPubKey::encode()`; receive: parse the 69-byte compact (both sides
- already know the derivation path, so depth/child-number are reconstructable).
- Pin with byte-exact vectors mirroring the reference clients (iOS
- `ecdsa_key.rs:333-341`, dashj `serializeContactPub()` — quoted in
- `INTEROP_DESK_CHECK.md`). 69 → PKCS7 → 80 ‖ IV 16 = exactly 96 bytes.
- **DONE (2026-06-10):** codec in `platform-encryption`
- (`compact_xpub_bytes`/`parse_compact_xpub`, `COMPACT_XPUB_LEN=69`);
- `ContactXpubData::compact_xpub()` + `reconstruct_contact_xpub` in
- `crypto/dip14.rs`; rs-sdk callback contract = "69-byte compact", validated
- pre-encryption; receive reconstructs from `chain_code`+`pubkey` (metadata
- depth/child synthesized — non-hardened CKD unaffected, pinned by
- `reconstructed_xpub_derives_identical_addresses`); legacy 78/107 fallback
- branch kept. 194 platform-wallet tests green; FFI ABI unchanged (caller doc
- contract tightened).
-8. **G15: key-purpose verification (decision gate, cheap).** Fetch a real
- mobile-created `contactRequest` + its sender identity from testnet; inspect
- `senderKeyIndex`/`recipientKeyIndex` purposes. Then align: likely
- liberal-on-receive (accept ECDSA keys of the purposes mobile actually uses)
- + compatible-on-send (fall back to the mobile convention when the recipient
- has no DECRYPTION-purpose key). Implementation in M1 if the verification
- confirms the mismatch; the validation gate from G1(b) stays for key *type*.
- **VERIFIED (2026-06-10, all 368 testnet contactRequests —
- `INTEROP_DESK_CHECK.md` §G15):** the "key 0 AUTHENTICATION" desk-check reading was
- stale. Dominant mobile cohort (223 docs): **unbound ENCRYPTION/MEDIUM key
- (id 2) for BOTH indices** (recipientKeyIndex → ENCRYPTION — mobile identities
- carry no DECRYPTION key); 2026 cohort (68 docs): contract-bound ENC(4)/DEC(5)
- — our convention. Consensus enforces neither purpose nor boundedness on these
- fields. **Alignment (task 9):** send — prefer recipient DECRYPTION, fall back
- to recipient ENCRYPTION; receive — accept ENCRYPTION for sender,
- ENC-or-DEC for recipient; keep the ECDSA type gate; purpose mismatch alone
- never marks a channel permanently broken. No AUTHENTICATION fallback.
-9. **G15 alignment implementation** (per the verified verdict above): relax the
- sender/recipient purpose assertions in `rs-sdk` `create_contact_request`
- (`:200-239`) and `rs-platform-wallet` key selection + `validate_contact_request`
- wiring; tests for the mobile-cohort shape (ENC/ENC, unbound) and our own
- (ENC/DEC, bound).
- **DONE (2026-06-10):** recipient selection prefers DECRYPTION, falls back to
- ENCRYPTION (ECDSA gate kept, no AUTH fallback); validation gained a recipient
- purpose gate (AUTH was silently accepted before!) + `purpose_mismatch` flag;
- purpose mismatches log-and-skip, never `payment_channel_broken`; ECDH decrypt
- path confirmed index-generic (pinned). 204 platform-wallet + 139 dash-sdk
- tests green. **M1 complete** (task 6 e2e rides #3549, non-gating).
-6. **G11-Rust: full-cycle e2e confirmation** (`dp_003`): `profile → send → sync →
- accept → established → pay`, on live testnet via the #3549 bank harness.
- **Not M1-exit-gating** (Part 7.4): M1 exits on tasks 1–5; this task is tracked
- on #3549 and lands when the framework does.
-
-**coreHeight backfill rescan (DIP-15 §8.7/§12.6) — DONE (2026-06-24).** Surfaced by
-the re-audit after M1's original plan: an incoming payment to a contact's receival
-address that landed *before* the address was watched (restore-from-seed, second
-device, or the offline-accept→pay window) was silently missed. Fixed by (a) re-import
-scanning from birth-height `Some(0)` (`cba515aaf1`) so on-chain history isn't skipped,
-and (b) `reconcile_dashpay_rescan` (`18483e4232`, a local-only step of `dashpay_sync`
-in `manager/dashpay_sync.rs`) that lowers the wallet's SPV `synced_height` toward
-`min($coreHeightCreatedAt)` over established receival contacts so dash-spv's filter
-manager re-matches the now-watched addresses against blocks it already scanned. It uses
-the inner unconditional height setter (no upstream change) with a per-contact
-`dashpay_rescan_triggered` one-shot guard so the recurring sweep doesn't re-lower the
-height every pass. The regression is safe (`synced_height` is the filter-scan
-checkpoint, decoupled from the monotonic `last_processed_height`); the floor is clamped
-to the engine's header/birth floor, and the one SPV caveat is that a rewind into a
-never-stored filter range is retried silently every 30s rather than erroring.
-
-### Milestone 2 — Swift UI (first-class, polished) + Swift tests
-
-See Part 6 for the screen design. Tasks:
-
-> **STATUS (2026-06-10): tasks 7–10 DONE** (Phases A–D on
-> `feat/dashpay-m1-sync-correctness`). Delivered: FFI sync-control surface
-> (`platform_wallet_manager_dashpay_sync_{start,stop,is_running,is_syncing,
-> last_sync_unix_seconds,set_interval,sync_now}`), persister payload extended
-> (callback arity 8→10: `payment_channel_broken` on `ContactRequestFFI`,
-> `ContactRequestRejectionFFI` tombstones), payment-history getter
-> (`managed_identity_get_dashpay_payments`); Swift SDK wrappers +
-> `PersistentDashpayPayment` + `@Published dashPaySyncIsSyncing`; the full
-> DashPay tab (`Views/DashPay/` — 7 files) with all §6.4 states and
-> `dashpay.*` accessibility ids; simulator BUILD SUCCEEDED.
-> **Spec deltas accepted:** (1) alias/note/hide are a UserDefaults-backed
-> device-local store until M3's `contactInfo` (no SwiftData model added);
-> (2) contact DPNS labels captured as an add-time hint (not persisted
-> elsewhere); (3) AddContact ID-mode preview is cache-only (no
-> fetch-profile-by-id FFI). Task 11 (tests) = Phase D.
-> **Task 11 DONE (2026-06-10):** 15 SDK unit tests (persister bridge: broken-flag
-> on both rows, tombstone scoped to `(owner,sender,accountReference)` w/ rotation
-> survival, 10-arg C-callback round-trip, payment upserts, FFI marshalling) + 2
-> UI smoke tests (§6.4 picker states; passed on-simulator). Phase D also found &
-> fixed a changeset-atomicity defect (`persistDashpayPayments` missing the
-> `!inChangeset` guard — red→green). Totals: swift test 29/29; app tests 237
-> passed / 18 pre-existing network-gated skips; UI smoke green; BUILD SUCCEEDED.
-> Full add→approve→pay XCUITest = documented TODO gated on funded testnet
-> identities (tracks `dp_003`).
-
-7. Add `RootTab.dashpay` + `DashPayTabView` with an active-identity picker
- (`ContentView.swift`, `SwiftExampleAppApp.swift`) — picker states per §6.4.
-8. Extract/rebuild `ContactsView`, `ContactRequestsView` (incoming **+ outgoing**),
- `AddContactView`, `ContactDetailView`, `ProfileView`/editor, reusing the
- already-wrapped `ManagedPlatformWallet` methods; implement the §6.4 interaction
- states (DPNS resolution states, send-collision flow — **AddContactView only**,
- not the payment sheet — in-flight rows). Payment
- history requires the persister mapping + `PersistentDashpayPayment` model (§6
- intro). Additional persister-bridge plumbing from M1: project the
- `rejected` tombstones and `payment_channel_broken` flag into SwiftData (the
- Rust SQLite pipeline already persists them; the Swift `on_persist_contacts_fn`
- bridge does not yet).
-9. Move lists onto `@Query [PersistentDashpayContactRequest]` /
- `[PersistentDashpayProfile]` with the §6.4 optimistic-overlay policy; refresh
- via `syncContactRequests()` + `syncDashPayProfiles()` in `.task` /
- pull-to-refresh, coordinated through the §6.4 single sync-in-progress signal
- (requires M1 task 2 — the three-caller invariant can't be exercised until the
- G12 background loop exists). **Realtime cadence (per §6.4):** the tab drives the
- background loop's interval to 4s on foreground / 15s on background via
- `setDashPaySyncInterval` (NavigationStack `onAppear`/`onDisappear`), and every
- local mutation (send/accept/QR-send/pay) fires a non-blocking `kickDashPaySync`
- so the counterparty side converges without waiting for the next tick.
-10. Polish: AsyncImage avatars w/ initial-circle fallback, empty states, loading &
- error states, inline success feedback (§6.4), accessibility identifiers on
- every interactive control (for XCUITest).
-11. **G11-Swift:** unit tests (wrapper round-trips) + XCUITest (add→approve→pay).
- (Part 7.)
-
-### Milestone 3 — Spec completeness (rotation, hide, alias sync)
-
-12. **G3:** wire `calculate_account_reference` + version bump into send;
- receive-path version handling + "addresses rotated" surfacing — **includes the
- receive-side re-keying** of contact-request state/persistence by
- `(counterparty, accountReference)` (see G3 scope note).
-13. **G10 + G5 stage 2:** `contactInfo` document support (SDK + wallet + FFI) →
- cross-device reject/hide + alias/note sync.
-
- **DONE (2026-06-12), 4 commits:** crypto core (DIP-15 derivation
- `root/65536'+65537'/idx'`, AES-256-ECB encToUserId, IV‖CBC privateData;
- the `privateData` plaintext was initially a CBOR array but is being
- migrated to the **DIP-15 varint** format — the contract enforces length
- only, so it's a free convention; see `CONTACTINFO_FORMAT_SPEC.md` /
- Spec 1), stateless doc↔contact resolution (decrypt every owned doc's
- encToUserId), sync step 3 of the recurring pass, publish with the
- DIP-15 ≥2-contacts privacy gate (deferred publishes update local state
- only), FFI `platform_wallet_set_dashpay_contact_info_with_signer`,
- persister round-trip (alias/note/hidden on the established rows, both
- directions), and **contact restore at load** (new contacts array on
- `IdentityRestoreEntryFFI`) — without which the re-establish sweep wiped
- metadata during the deferred-publish window and contacts were invisible
- on offline launches. Verified on-sim: alias save → relaunch → survives
- and renders.
-14. Swift UI for alias/note edit (reuse `EditAliasView`) now backed by
- `contactInfo` — remove the M2 "This device only" labels.
-
- **DONE (2026-06-12):** ContactDetailView reads alias/note/hidden off
- the `@Query` contact rows and writes through
- `ManagedPlatformWallet.setDashPayContactInfo`; ContactsView hidden
- filter + alias display moved off the UserDefaults meta store (which
- now only keeps the add-time DPNS hint); labels updated.
-
-**Receive-side `encryptedAccountLabel` surfacing (DIP-15 §8.5) — DONE (2026-06-24).**
-The send side already length-normalizes the label; the receive side now decrypts and
-shows it. Decrypted in Rust at the two signer-bearing register sites (the drain
-`RegisterExternal` Ok-branch + `accept_register_external_validated`, where the ECDH
-`shared` key lives) by `store_contact_account_label` (`network/contact_requests.rs`),
-stored on the derived `EstablishedContact.contact_account_label` field (reset in `new`,
-`reestablish_preserving_metadata`, and `apply_rotated_incoming_request` so it never
-goes stale). It is surfaced **incoming-only** — it is the contact's label for *their*
-account, so it is derived strictly from the incoming request and projected onto the
-incoming FFI row only (the outgoing row's label is one *we* sent and is never shown),
-via `contact_persistence.rs`. Decrypt failures / garbage / control-chars sanitize to
-`None` (cosmetic — never breaks the channel), and it resets on rotation. Renders as a
-read-only "Their account" row through Swift `contactAccountLabel` → `ContactDetailView`.
-
-15. **G4 design-only:** specify the FFI ECDH hook (shared-secret-only across the
- ABI — never a raw private key; see G4) so M4's implementation doesn't churn
- the wallet API.
-
- **DONE (2026-06-12) — design:**
- - **ABI surface (one new callback on the existing host-signer table** —
- the same registration path external-signable wallets already use for
- transaction signing**):**
- ```c
- int32_t (*ecdh_shared_secret_fn)(
- void *context,
- const uint8_t (*wallet_id)[32],
- const uint8_t (*identity_id)[32],
- uint32_t key_id, // sender's encryption key id
- const uint8_t (*counterparty_pubkey)[33],
- uint8_t (*out_shared_secret)[32]); // SHA256((y&1|2)||x) — finished secret
- ```
- The host derives the identity encryption private key for
- `(identity_id, key_id)` from its keychain/secure element and computes
- the **finished DIP-15 shared secret host-side**. The private key never
- crosses the ABI (the `rs-sdk-ffi` `DashSDKContactRequestParams.
- sender_private_key` field is the antipattern this replaces; flagged for
- its own audit). Non-zero return = "host cannot produce the secret"
- (locked keychain, missing key): the operation fails with a typed
- `EcdhUnavailable` error and is NOT treated as a broken payment channel
- (our side failed, not the contact's request).
- - **Rust routing:** `send_contact_request` /
- `register_external_contact_account` branch on wallet key-residency:
- seed-resident wallets keep today's in-process derivation
- (`EcdhProvider::SdkSide`); external-signable wallets route
- `EcdhProvider::ClientSide { get_shared_secret }` where the closure
- calls the FFI hook. No public wallet-API signature changes — the
- provider choice is internal, which is what de-risks M4.
- - **Zeroization:** Rust wipes `out_shared_secret` (`Zeroizing`) after
- deriving the AES key; hosts are instructed to do the same with their
- intermediate private key (Swift: `withUnsafeTemporaryAllocation` +
- explicit reset, mirroring the signer callback's key handling).
- - **Same hook serves decrypt-side** (`register_external_contact_account`
- needs ECDH with the *contact's* pubkey at OUR key id) — the
- `counterparty_pubkey` parameter covers both directions; no second
- callback needed.
-
-### Milestone 4 — Hardening / cleanup
-
-16. **G4:** watch-only ECDH via `EcdhProvider::ClientSide` pushed across FFI
- (implements the M3 design).
-
- **DEFERRED with design amendment (2026-06-13):** implementation scoping
- found the M3 hook (ECDH shared secret only) is **insufficient** for true
- watch-only DashPay: the friendship-xpub derivations are hardened and
- seed-bound on BOTH flows — send derives the sender↔recipient receiving
- xpub (`m/9'/coin'/15'/account'/`, recipient-dependent so not
- pre-derivable), and accept derives our receiving xpub for the new
- account. A watch-only host therefore needs **three** hooks: ECDH shared
- secret (designed in M3), friendship-xpub derivation, and
- receiving-account-xpub derivation — or one combined
- "derive-DashPay-context" hook returning `(compact_xpub, shared_secret)`.
- The contactInfo self-encryption keys (M3 task 13) are seed-bound the
- same way and need a fourth surface (or ride the combined hook).
- Since the example app attaches the seed at launch (this gap is
- explicitly not a demo blocker), shipping the ECDH-only ABI change would
- add churn without enabling any watch-only flow. Revisit as its own
- design+implementation slice when a hardware/watch-only host exists.
-17. **G6:** fix/delete fallback contract id.
-
- **DONE (2026-06-13):** fallback corrected from the DPNS id to the
- deployed DashPay id.
-18. **G7:** wire send-path validation; ship-or-delete auto-accept (verify-gate
- acceptance criterion applies if shipped — see G7).
-
- **DONE (send half, 2026-06-13):** the selected key pair gates through
- `validate_contact_request` before any ECDH/broadcast. Auto-accept:
- decision = **keep dormant** — it activates with M5 invitations behind
- the `verify_auto_accept_proof` hard gate (per Part 8.5), not deleted.
-19. **G8/G9:** real local ciphertext; contract cache.
-
- **DONE (2026-06-13):** sent rows store the real 96-byte ciphertext off
- the broadcast document; the bundled DashPay contract is cached
- process-wide (OnceLock) replacing five per-call re-parses.
-20. Live cross-client interop e2e (compact xpub, ECDH, accountReference) vs
- reference DashPay clients (the M1 desk-check verified the formats on paper).
-
- **BLOCKED-EXTERNAL (2026-06-13):** requires driving real DashWallet
- iOS/Android builds against a shared network — not runnable in this
- environment. The M1 desk-check (`INTEROP_DESK_CHECK.md`) + on-chain census remain
- the interop evidence; the contactInfo research (`CONTACTINFO_FORMAT_SPEC.md` Appendix A) found no
- reference client implements contactInfo at all, shrinking the live-e2e
- surface to contactRequest + payment addresses. Run manually when a
- mobile test build is available.
-
-### Milestone 5 — Invitations (new scope, 2026-06-10; needs its own design pass)
-
-Onboard users who don't have Dash yet: inviter creates an asset-lock-funded
-credit voucher + link (DIP-13 invitation subfeature, `m/9'/5'/5'/3'`); invitee
-claims it → identity created from the voucher → invitee's contact request to the
-inviter carries an `autoAcceptProof` (path `m/9'/5'/16'/timestamp'`, helpers
-already implemented in `crypto/auto_accept.rs`) → auto-established contact after
-`verify_auto_accept_proof` (hard gate, see G7/Part 8.5). Scope before
-implementation: a research+design slice (invitation create/claim wallet flows,
-deep-link format, expiry/revocation, UI) — the platform wallet has the asset-lock
-and identity-registration machinery to build on but no invitation flows today.
-
----
-
-## Part 6 — Swift UI design (the "nice UI")
-
-**Decision: promote DashPay to a first-class tab (Option B).** Lower-risk Option A
-(polish in place under Identities) is the fallback if tab real estate is contested,
-but a "nice DashPay UI" wants its own home. All screens reuse already-wrapped
-`ManagedPlatformWallet` FFI — **no new network FFI**; the one new plumbing item is
-**payment history** (map the Rust `dashpay_payments` changeset overlay in the Swift
-persister into a new `PersistentDashpayPayment` SwiftData model + `@Query` — today
-no FFI exposes `PaymentEntry` and no SwiftData model exists for it).
-
-### 6.1 Navigation
-
-Add `case dashpay` to `RootTab` (`ContentView.swift`), between `identities` and
-`contracts`. Tab icon `person.2.fill`, title "DashPay".
-
-```
-DashPayTabView (NavigationStack)
-├─ Active-identity picker (top) — most DashPay UIs assume one active identity;
-│ menu of the wallet's managed identities (DPNS name → truncated id).
-├─ Profile header card → tap → ProfileView / ProfileEditorView
-│ (empty state → "Set up your DashPay profile" CTA → ProfileEditorView)
-├─ Username prompt card → "Register a username" → RegisterNameView
-│ (shown only when an on-chain DPNS check confirms the active identity
-│ has no name; explains that without a username people can't find you
-│ by name, and that the profile display name is cosmetic, not searchable)
-├─ Segmented control: [ Contacts | Requests ]
-│ ├─ Contacts → ContactsView (@Query established)
-│ │ row tap → ContactDetailView → "Send Dash" / alias / note / hide
-│ └─ Requests → ContactRequestsView
-│ ├─ Incoming (Accept / Reject)
-│ └─ Outgoing (pending — NEW, currently unrendered)
-└─ Toolbar: + (AddContactView) · refresh (sync)
-```
-
-Wire DashPay sync into the `.task` of `DashPayTabView` and/or the existing global
-`GlobalSyncIndicator`: run `syncContactRequests()` then `syncDashPayProfiles()`.
-
-### 6.2 Screens
-
-**ProfileView / ProfileEditorView** (promote from `IdentityDetailView`)
-- View: large avatar (`AsyncImage` w/ initial-circle fallback), `displayName`,
- DPNS handle, `publicMessage`. "Edit" button. Empty state → "Set up your DashPay
- profile" CTA (opens `ProfileEditorView` as a sheet — same target as "Edit").
-- Editor: `Form` with `displayName` (≤25), `publicMessage` (≤140), `avatarUrl`;
- live char counters; on save fetch avatar bytes for DIP-15 hash/fingerprint; call
- `createDashPayProfile` / `updateDashPayProfile(…signer:)`.
-
-**Username prompt** (`usernamePromptCard`, below the profile header)
-- A second setup CTA, independent of the profile one: shown when the active
- identity has **no DPNS username** (confirmed by an on-chain `dpnsGetUsername`
- check in `.task` + on app-foreground, so an identity that already has one —
- just not yet cached, or registered meanwhile on another device — is never
- nagged; mirrors `IdentitiesView`'s lazy fetch. A found name is persisted and
- saved; a definitive empty result shows the card; a thrown error retries.
- Residual: a name registered on another device *while the user sits on this
- tab* clears on the next tab switch / app-foreground). Tap → `RegisterNameView`.
- Copy makes the username-vs-profile distinction explicit: a **username** is the
- searchable handle people type to add you (contact search); the profile
- **display name** is cosmetic and not searchable. On registration the Rust path
- persists `dpnsName`, so the prompt hides reactively via `@Query`.
-
-**ContactsView**
-- `@Query` established contacts (joined to `PersistentDashpayProfile` for
- display). Row = avatar + (alias → displayName → DPNS → truncated id) + last
- payment hint. Search bar. Pull-to-refresh = sync. Empty state → "Add your first
- contact".
-
-**ContactRequestsView** (the **new outgoing section** is the headline UI gap)
-- **Incoming**: row + Accept (`borderedProminent`) / Reject (`.tint(.red)`),
- relative timestamp, sender profile. On accept → success toast + move to Contacts.
-- **Outgoing**: pending sent requests (`fetchSentContactRequests` /
- `getSentContactRequestIds`), "Pending" badge, sent timestamp. Currently loaded
- but never shown — render it.
-
-**AddContactView** (restyle `AddFriendView`)
-- Segmented: **Username (DPNS)** | **Identity ID**. DPNS mode: live prefix search
- (`searchDpnsNames`) with result rows (avatar + name); ID mode: paste + validate
- base58. Resolve → preview the target profile → "Send request" → `sendContactRequest`.
-
-**ContactDetailView**
-- Profile header; **Send Dash** (presents the polished `SendDashPayPaymentSheet`);
- payment history (from `PaymentEntry` via the `PersistentDashpayPayment` mapping —
- see §6 intro); editable **alias** / **note** and **Hide** toggle, each labeled
- **"This device only"** in M2 (until M3's `contactInfo` backing replaces the
- label) so users don't assume sync semantics that don't exist yet.
-
-**SendDashPayPaymentSheet** (already polished — restyle only)
-- Amount in DASH→duffs, spendable balance, over-spend block, recipient
- profile/avatar/DPNS, result txid. (Memo is local-only; keep the field hidden or
- label it "private note" until on-chain memo exists.)
-- Zero-balance state: when spendable balance is 0 (after the async load), disable
- the amount field + Send and show "Your balance is 0 DASH — top up your wallet
- before sending." instead of an always-disabled interactive form.
-
-### 6.3 Conventions (must match house style)
-
-From `SwiftExampleApp/CLAUDE.md`:
-- `@EnvironmentObject var walletManager: PlatformWalletManager`,
- `var appState: AppState`; `@Environment(\.modelContext)`.
-- Lists via `@Query` on `Persistent*` (move off the live-snapshot read).
-- Async FFI: `Task { @MainActor in … }`, `defer { isLoading = false }`, resolve
- wallet via `walletManager.wallet(for:)`, fresh `KeychainSigner` per submit,
- `errorMessage` red caption on catch.
-- `Form`/`Section` for editors, `List`/`Section` with count headers for lists.
-- SF Symbols (`person.2`, `person.badge.plus`, `paperplane`, `pencil`,
- `person.crop.circle`), blue accent, red destructive, green success,
- `.borderedProminent` primaries.
-- Amounts entered in DASH, converted to duffs (`× 100_000_000`).
-- Display precedence: alias → DashPay `displayName` → DPNS → truncated hex.
-- **`.accessibilityIdentifier(...)` on every interactive control** (needed for
- XCUITest) — e.g. `dashpay.tab`, `dashpay.addContact`, `dashpay.request.accept`,
- `dashpay.send.amount`, `dashpay.send.confirm`.
-- **Never orchestrate in Swift** — one FFI call per DashPay op.
-
-### 6.4 Interaction states & edge cases (normative for M2)
-
-- **Identity picker** (tab root) — three states: (1) no wallet loaded → disabled
- "No wallet loaded" label + link to the Wallets tab; (2) wallet but zero
- identities → "No identities yet" + CTA to the Identities tab; (3) ≥1 identity →
- menu. Exactly one identity → auto-select and hide the picker. Selection persists
- across launches via `@AppStorage`.
-- **AddContactView (DPNS mode)** — four states: typing → searching (inline
- `ProgressView`) → not-found (inline message + clear-and-retry affordance, never a
- dead end) → found (profile-preview card; "Send request" enabled only from this
- state). ID mode: inline base58 validation gates the send button. (The current
- `AddFriendView` dead-ends on "DPNS name not found".)
-- **Send-collision flow** — if the target already has an incoming request to us,
- alert "This person already sent you a request — Accept it instead?" with
- Accept / Continue anyway. (Sending anyway is protocol-valid; it just establishes
- the contact.)
-- **Request rows in flight** — on Accept/Reject tap, replace both buttons with a
- `ProgressView` for that row (prevents double-tap → duplicate accepts); on
- success remove the row optimistically; on failure restore the buttons + inline
- error on the row.
-- **Optimistic overlay over `@Query`** — accept/reject/send mutate Rust state and
- the persister callback lands later; bridge the latency window with a local
- `@State` overlay set of affected ids filtering the `@Query` results, cleared
- when the query reflects the change. (The old `loadFriends()` re-read pattern is
- incompatible with pure `@Query` reactivity.)
-- **Single sync-in-progress signal** — one `@Published` flag on
- `PlatformWalletManager` observed by all three sync callers (`.task`,
- pull-to-refresh, the G12 background loop); a pull-to-refresh during an in-flight
- sync attaches to it instead of double-firing.
-- **Realtime cadence (foreground-fast / background-slow)** — the G12 background
- loop runs on a tunable interval (`setDashPaySyncInterval`, clamped ≥ 1s Rust-side;
- default = `backgroundSyncSeconds` = 15s). The DashPay tab drops it to
- `foregroundSyncSeconds` = 4s at *effective foreground* — the tab is on screen
- **and** the app is active — and restores 15s otherwise. "On screen" is driven
- from the tab's **NavigationStack** `onAppear`/`onDisappear` (so drilling into a
- contact detail or presenting a sheet, neither of which fires the stack's
- `onDisappear`, keeps the fast cadence; only a *tab switch* relaxes it); "app
- active" is driven from `scenePhase`, so backgrounding the app while on the tab
- also relaxes to 15s. The cadence acts only on transitions. This keeps neither an
- inactive tab nor a backgrounded app sweeping every few seconds, while incoming
- requests / acceptances / payments surface in near real time when the user is
- actually looking. **Entry kick:** `setDashPaySyncInterval` only takes effect on
- the loop's *next* sleep (it stores an atomic, no wakeup — `dashpay_sync.rs:157`),
- so entering the foreground also fires one `kickDashPaySync` — otherwise a tab
- re-entry could wait out a leftover up-to-15s sleep before the first fast tick.
- (A Rust-side `Notify` on `set_interval` would shorten the in-flight sleep
- directly; deferred as an internal refinement — the entry kick achieves the same
- user-visible result app-side.) Best-effort: a not-yet-configured manager keeps
- its current interval.
-- **Post-mutation sync kick** — after a local mutation (send request, accept,
- send-via-QR, pay) the handler fires a non-blocking `dashPaySyncNow()`
- (`kickDashPaySync`) so the counterparty's state and the established pair converge
- promptly instead of waiting a full poll tick. Non-blocking: the sheet dismisses
- right away and the Rust manager folds an in-flight pass into a no-op (the single
- sync-in-progress signal above). *Bounded, not instant:* if a pass was already
- running when the mutation landed, the kick no-ops and convergence waits for the
- next tick (≤ the foreground 4s) rather than enqueuing a coalesced re-run.
- Complements, doesn't replace, the optimistic `@Query` overlay — the overlay
- covers the sender's own row, the kick pulls the other side.
-- **Success feedback** — reuse the existing inline success pattern
- (`SendDashPayPaymentSheet`'s green inline text); no new toast component in M2
- (the app has no shared toast — only a clipboard `CopiedToast`).
-- **Broken payment channel** (surfaces G1(c)) — ContactsView row shows a warning
- badge; ContactDetailView disables Send Dash with "Payment channel broken — ask
- the contact to send a new request" (re-enables when a new request arrives).
-- **Needs-unlock / verify-failed banner** (seedless wallets) — **DONE (2026-06-23;
- `9963923e05` Rust+FFI, `841802c587` Swift).** A signerless sweep enqueues
- contact-crypto ops it can't finish while the Keychain is locked;
- `pending_contact_crypto_count` (`network/contact_requests.rs`) → FFI
- `platform_wallet_pending_contact_crypto_count` (`dashpay.rs`) feeds the ~1 Hz Swift
- poller into a per-wallet `DashPayUnlockStatus` / `@Published dashPayUnlockStatus`,
- rendered as a banner in `DashPayTabView.swift` (orange "N contact(s) waiting to
- finish setup" + Unlock, red on seed-mismatch). The count **excludes**
- `ContactInfoDecrypt` ops — those re-enqueue every sweep, so counting them would
- falsely re-trip the banner ~15s after every unlock; only the account-build ops
- (`RegisterReceiving`/`RegisterExternal`) converge to 0 once the payment account is
- built.
-- **Profile save flow** — on save: disable Save + inline `ProgressView`; success →
- dismiss the editor sheet; failure → re-enable Save + red caption below the form.
-- **Payment history list** — empty state "No payments yet"; loading = single
- inline `ProgressView`; error = keep last-known list + inline caption.
-
----
-
-### Multi-reviewer code review (2026-06-14) — 8 findings fixed
-
-Five specialized reviewers (crypto-security, FFI-memory, sync-correctness,
-Swift/iOS, silent-failures) audited the M1–M4 diff. Crypto + FFI-memory
-boundaries came back clean. Correctness/silent-failure reviewers found bugs
-the live UAT had missed (UAT only hit the pending-sent rotation path, which
-the reject-tombstone masked). All fixed with red→green regression tests:
-
-- **P0** rotation re-send to an ESTABLISHED contact reset the version to 0
- → unique-index rejection → contact unrotatable. Lookup now consults
- `established_contacts.outgoing_request` (`prior_sent_account_reference`).
-- **P0** multi-doc sweep thrash: immutable docs from a rotated sender both
- returned every sweep, flipping state + rebuilding the external account
- forever. `newest_received_per_sender` collapses to newest-per-sender
- before ingest; `apply_rotated` is idempotent.
-- **Critical** swallowed persist errors → memory/disk divergence (reject
- resurrection). New `PlatformWalletError::Persistence`; reject + send_payment
- propagate, self-healing sweep writes log.
-- **H1** Sent payments lost at relaunch (map restored empty) → new
- `PaymentRestoreEntryFFI` + `restore_dashpay_payments` fold + Swift builder.
-- **H2** deferred-publish lied as "synced" → 3-state `ContactInfoPublishOutcome`
- through the FFI; ContactDetailView shows the real state.
-- Med: zero-ciphertext fallback → hard Err; contactInfo derivation-index
- high-water mark; Swift silent contact-drop now logged; crypto
- account_index/accountReference invariant documented.
-
-230/230 Rust lib + FFI tests green, clippy clean, full iOS build green.
-NOTE: on-device re-verify of H1/H2 pending — the sim SwiftData store was
-reset environmentally (identities gone), so it needs the devnet identity
-setup rebuilt first.
-
-### Devnet UAT round 2 (2026-06-13) — rotation / reject / DPNS verified live
-
-On paloma with three identities: **reject + tombstone** (rejected request
-suppressed across forced re-sync), **G3 rotation end-to-end** (re-send from
-the rejected sender broadcast with a bumped accountReference — accepted by
-the unique index — and reappeared through the tombstone on the recipient:
-the dp_005 scenario, live), **DPNS register → live search → found preview →
-send** and the **not-found state** (inline + retry, no dead end), **accept
-of the rotation request** (re-established, accounts rebuilt). Findings
-fixed: optimistic pending-sent overlay leaked across identity switches
-(now reset on picker change). Open UX item: "SPV client is not running"
-dead-ends both Send Dash and identity creation — needs auto-start or a
-"Start & retry" affordance (product decision pending).
-
-## Part 7 — Test plan
-
-Follow the repo TDD discipline (failing test first; red→green in the commit
-message). DashPay's correctness-critical pieces are the crypto and the
-state-machine handshake — those get the deepest coverage.
-
-### 7.1 Rust — `rs-platform-wallet` / `rs-sdk` / `platform-encryption`
-
-Already covered (keep, don't duplicate): crypto round-trips, the contact
-state-machine handshake (`tests/contact_workflow_tests.rs` + inline), and
-persistence (`wallet/apply.rs`). See G11 for the inventory.
-
-Unit — **the missing tier is the `network/` layer** (currently 0 tests). Add behind
-a mock SDK/broadcaster seam:
-- **Recurring sync (G12):** a recurring pass drives `dashpay_sync` for each wallet
- — including identities with zero watched tokens (see G12: do not couple to the
- token registry); re-entrancy guard + `quiesce` shutdown still hold; interval
- changes are picked up.
-- **Sync builds external accounts (G1):** given an established contact with an
- encrypted xpub, the sync pass decrypts it and registers a `DashpayExternalAccount`
- (and skips gracefully for watch-only).
-- **Crypto/derivation wiring:** `calculate_account_reference` is actually used by
- the send path (G3) and round-trips (un-mask recovers account + version).
-- **State machine** (extend existing): idempotent re-sync; accept when both present;
- reject removes incoming.
-
-Offline crypto/encode tier (rs-sdk, no network) — follow the existing
-`packages/rs-sdk/tests/fetch/` harness with `--features mocks,offline-testing`
-(`Config` from `tests/.env`, `mock::Mockable` + recorded vectors):
-- **G2 (entropy):** after `create_contact_request`, the returned id matches the id
- derived from the *broadcast* entropy. Pin id equality.
-- contact-request wire-shape: `encryptedPublicKey == 96B`, properties map matches
- the v1 schema, `accountReference` round-trips.
-
-**E2E tier — build on the existing framework (PR #3549,
-`packages/rs-platform-wallet/tests/e2e/`).** This is the canonical "how we do e2e"
-for this crate (see [Part 7.4](#74-alignment-with-the-existing-e2e-framework)):
-gated behind the **`e2e` cargo feature**, funded by the testnet **`bank` wallet**
-harness (`framework/bank.rs` — `BankWallet::load`, `fund_address`,
-`cross_check_balance`), config via `tests/.env`
-(`PLATFORM_WALLET_E2E_BANK_MNEMONIC`), one file per case under
-`tests/e2e/cases/_NNN_*.rs` registered in `cases/mod.rs`, run with
-`cargo test -p platform-wallet --test e2e --features e2e -- --nocapture`. Add a
-**DashPay case family** (proposed prefix `dp_*`), modeled on the shielded `sh_*`
-suite (PR #3727) which stacks the same way:
-
-- **dp_001 (profile):** `create_profile` → fetch from Platform → fields match;
- `update_profile` bumps revision.
-- **dp_002 (send request):** fund 2 bank-derived identities; A
- `send_contact_request(B)`; assert the on-platform `contactRequest`
- (`encryptedPublicKey==96B`, key indices, accountReference) + id equality (G2).
-- **dp_003 (full cycle — the "done" gate):** A `send_contact_request(B)` → B
- recurring-sync sees incoming → B `accept` → **both** established → A
- `send_payment(B)` confirms on L1 → B records incoming.
-- **dp_004 (offline accept → pay, pins G1+G12):** A sends → B accepts → A offline →
- A's **recurring sync** runs → A `send_payment(B)` **succeeds** (external account
- built during the sweep). *Must fail before the G1/G12 fix, pass after.*
-- **dp_005 (rotation, pins G3):** second `send_contact_request` to the same
- recipient with a bumped version is accepted (distinct `accountReference`); the
- receive path surfaces the rotation.
-- **dp_006 (recurring cadence):** the background recurring sync refreshes
- contacts/profiles without an explicit FFI call — including for an identity with
- zero watched tokens (assert via the bank harness over a couple of sweeps).
-- **dp_006b (foreground-fast cadence + post-mutation kick, §6.4):** entering the
- DashPay tab lowers the sweep interval to 4s **and fires one immediate sweep**
- (because `set_interval` only applies on the loop's next sleep); leaving the tab
- *or backgrounding the app while on it* restores 15s (`setDashPaySyncInterval`
- round-trips through the FFI). A send/accept fires an extra `dashPaySyncNow()`
- that no-ops when a pass is already in flight. UI-level cadence + the
- scenePhase/tab-visibility state machine + the entry kick are covered by a manual
- two-sim e2e — these are SwiftUI-lifecycle/wall-clock timing properties the
- simulator harness can't assert deterministically; the FFI set-interval/sync-now
- round-trip is unit-tested Rust-side.
-
-### 7.2 Swift — `SwiftTests` + `SwiftExampleAppUITests`
-
-Unit (`SwiftTests/SwiftDashSDKTests/`):
-- `ContactRequest(ffi:)`, `EstablishedContact`, `DashPayProfile(ffi:)` /
- `DashPayProfileUpdate` round-trips and marshalling (32-byte id in/out, optional
- C-strings, `_free` correctness, no leaks).
-- `PersistentDashpay*` SwiftData upsert from the persister callback.
-
-Flow (mirror the existing `PlatformWalletIntegrationTests.swift` harness, testnet):
-- send → sync → accept → established → pay, asserting SwiftData rows + balances.
-
-XCUITest (`SwiftExampleAppUITests/`, keyed on accessibility ids):
-- Open DashPay tab → AddContact by DPNS → request appears in Outgoing → (peer
- accepts) → appears in Contacts → open contact → Send Dash → confirm txid.
-- Use the `simulator-control` skill for SwiftData inspection + screenshots in UAT.
-
-### 7.3 "Definition of done" per flow
-
-| Flow | Done when |
-|---|---|
-| Create/update profile | `dp_001` + Swift editor XCUITest green; profile visible to peer |
-| Send contact request | `dp_002` + G2 (entropy) offline test + AddContact XCUITest green |
-| Approve request | `dp_003` accept step → both established; Accept XCUITest green |
-| Reject request | local reject unit test green; (M3) `contactInfo` hide syncs across devices |
-| Send money to contact | `dp_003` pay step + **`dp_004` (offline accept→pay)** + Send XCUITest green |
-| Sync (recurring) | `dp_004`/`dp_006` build external account on the recurring sweep; idempotency unit test green |
-
-### 7.4 Alignment with the existing e2e framework
-
-The platform-wallet e2e framework **already exists but is unmerged** — PR
-**#3549** (`feat/rs-platform-wallet-e2e`, draft). DashPay e2e cases must be authored
-**on that branch** (or rebased onto it after it merges); they are not standalone.
-Conventions to follow exactly (from `tests/e2e/README.md`):
-- Modeled on `dash-evo-tool/tests/backend-e2e/`; runs against **live Dash testnet**
- (v3.0) via DAPI, gated behind the `e2e` cargo feature.
-- Funding via the **platform-address `bank` wallet** (seed in
- `PLATFORM_WALLET_E2E_BANK_MNEMONIC` / `tests/.env`); most DashPay cases never
- touch L1 except the `send_payment` step (which spends Core funds → needs the
- bank's Core balance, like CR-003/AL-001).
-- Test attribute `#[tokio_shared_rt::test(shared, flavor = "multi_thread",
- worker_threads = 12)]`; context provider `TrustedHttpContextProvider`.
-- New cases: add `tests/e2e/cases/dp_NNN_*.rs`, register in `cases/mod.rs`, document
- in `tests/e2e/TEST_SPEC.md` (pin accounting). The shielded suite (PR #3727,
- `sh_*`) is the worked example of stacking a feature-area suite on this framework.
-
-**Sequencing implication:** the DashPay e2e suite rides #3549 — but **M1's exit
-criterion is the mock-seam unit/integration tier** (no #3549 dependency), so M1 is
-never blocked on the draft PR. `dp_003`/`dp_004` are the e2e *confirmation* of the
-same behaviors, tracked on #3549 (authored stacked on it, or added right after it
-merges). The offline crypto/encode tier likewise lands immediately.
-
----
-
-## Part 8 — Risks, decisions, open questions
-
-1. **UI shape — first-class tab vs polish-in-place.** Recommended: first-class
- `DashPay` tab (Part 6). *Decision owner: product.* Fallback documented.
-2. **Cross-client interop. RESOLVED (2026-06-10, desk-check
- `INTEROP_DESK_CHECK.md`):** xpub plaintext FAIL → G14 fix in M1 task 7; ECDH PASS;
- accountReference PASS-for-now (+2 latent masking bugs noted for M3); new G15
- key-purpose hazard → verification gate in M1 task 8. Live cross-client e2e
- stays M4. ⚠ A side-finding: our stack was **not** self-consistent either —
- the 107-byte plaintext broke our own send path (see G14).
-3. **Watch-only / hardware wallets (G4).** Out of scope for the demo app (it holds
- the seed) but required for production. **FFI-hook design lands in M3 (task 15)**
- — shared secret only across the ABI, never a raw private key (see G4);
- implementation in M4.
-4. **`accountReference` semantics (G3).** Decide whether to keep "share full
- account xpub, ignore masking" (simpler, but breaks rotation via the unique
- index) or implement the DIP-15 masking + version flow. Recommended: implement it
- (M3) — rotation is a real user need and the unique-index collision is a latent
- bug.
-5. **Auto-accept (G7). DECIDED (2026-06-10): keep.** Invitations are now in scope
- (Milestone 5) and are built on `autoAcceptProof`, so the helpers + FFI param
- stay (dormant until M5 wires them). **Hard requirement when wired:** the
- `verify_auto_accept_proof` gate before any automatic acceptance (see G7).
-6. **`send_contact_request` entropy (G2). RESOLVED (2026-06-10):** real broadcast
- bug — consensus rejected every send (`InvalidDocumentTransitionIdError`).
- Fixed in M1 task 4; see the DONE note there.
-7. **E2E framework dependency.** The DashPay e2e suite rides PR **#3549** (draft,
- unmerged). **M1's exit criterion is the mock-seam tier** (Part 7.4), so M1 never
- blocks on it; the `dp_*` cases are authored stacked on #3549 or right after it
- merges. *Open: name the owner who decides stack-vs-wait before M1 starts.*
-
----
-
-## Part 9 — Related in-flight work (open PRs)
-
-Surfaced from the live PR list — these intersect this plan and should be tracked /
-coordinated rather than duplicated:
-
-| PR | Branch | Relevance |
-|----|--------|-----------|
-| **#3549** (draft) | `feat/rs-platform-wallet-e2e` | **The e2e framework** the DashPay suite must build on (Part 7.4). |
-| **#3727** (draft) | `test/rs-platform-wallet-shielded-e2e` | Shielded `sh_*` e2e suite — the **worked template** for a feature-area suite on #3549. |
-| **#3787** | `codex/dashpay-dip15-contact-request-docs` | "DashPay contact request encryption guide" — cross-check against Part 2; avoid doc drift. |
-| **#3639** | `feat/platform-wallet-external-signable-wallets` | External/signable wallets — the substrate for **G4** (watch-only ECDH via `ClientSide`). Coordinate before building G4. |
-| **#3692** | `feat/platform-wallet-rehydration` | Watch-only rehydration from persistor — touches the same watch-only path as G4. |
-| **#3817** | `feature/coinjoin-sweep-and-recovery` | DashSync→SDK migration context (the broader effort DashPay sits inside). |
-| **#3750** (NO MERGE) | `feat/platform-wallet-consumer-hardening` | FFI/consumer hardening — may move FFI signatures the Swift layer depends on. |
-
----
-
-### Appendix — evidence sources
-
-- [`INTEROP_DESK_CHECK.md`](./INTEROP_DESK_CHECK.md) —
- cross-client (iOS DashSync / Android dashj) interop evidence + testnet census.
-- [`CONTACTINFO_FORMAT_SPEC.md` Appendix A](./CONTACTINFO_FORMAT_SPEC.md) —
- contactInfo wire conventions (this repo sets the de-facto convention).
-
-The transient working-research files (DIP paraphrase, SDK/contract survey with
-worktree-relative file:line citations) were trimmed from the tree; find them in
-this branch's git history under `docs/dashpay/research/`.
diff --git a/docs/dashpay/SYNC_CORRECTNESS_SPEC.md b/docs/dashpay/SYNC_CORRECTNESS_SPEC.md
deleted file mode 100644
index 4b8ab6b4a87..00000000000
--- a/docs/dashpay/SYNC_CORRECTNESS_SPEC.md
+++ /dev/null
@@ -1,441 +0,0 @@
-# DashPay sync correctness — contact requests **and** profiles (mirror Android `PlatformSyncService`)
-
-Status: **IMPLEMENTED (2026-06-18)** — both stages shipped on
-`feat/dashpay-m1-sync-correctness` (PR #3841), 5-lens review (§9) folded in first.
-Stage 1 = paginated retrieve-all + per-identity high-water cursor + 10-min overlap
-at a 15s cadence (`network/contact_requests.rs`); stage 2 = id-keyed
-`contact_profiles` cache for established + pending senders (`network/contact_info.rs`,
-`accessors.rs`). Both are surfaced in the UI and **durably persisted** through the
-changeset pipeline to *both* backends (SQLite persister + SwiftData); the high-water
-cursor stays in-memory by design (a cold restore does one safe full re-fetch).
-Owner: rs-sdk / platform-wallet
-Priority: **FIRST** of the DashPay correctness track (ahead of the contactInfo
-format migration and the ignore feature).
-
-This spec covers **two consecutive stages of the same Android sync loop**:
-
-| Stage | Android (`PlatformSyncService`) | Us before | Delivered |
-|-------|--------------------------------|-----------|-----------|
-| 1. Contact-request fetch | `updateContactRequests()` — incremental, paginated, high-water | present but **broken** (truncated at 100, no high-water) | **fixed** — retrieve-all + high-water cursor |
-| 2. Contact-profile fetch | `updateContactProfiles(userIds)` — batch `whereIn $ownerId` | **absent** (synced only our *own* profile) | **added** — id-keyed cache, established + pending senders |
-
-Neither is an optimization: stage 1 is a **correctness bug** (real requests are
-permanently buried) and stage 2 is a **missing feature** (contacts have no name
-or avatar in the UI). The Android wallet (`dash-wallet`, on `kotlin-platform`)
-already does both; this spec mirrors that proven design. Delivered as **two
-commits** (stage 1, then stage 2) on one branch.
-
----
-
-## 1. Problem
-
-### 1.1 Stage 1 — our contact-request fetch is wrong, not just slow
-
-`packages/rs-sdk/src/platform/dashpay/contact_request_queries.rs`:
-
-```rust
-where toUserId == me, order_by $createdAt, limit: 100, start: None
-```
-
-`start: None` + a fixed `limit: 100`, re-run every sweep:
-
-- **Re-fetches the first page from the beginning every sweep** — pays the full
- fetch + GroveDB proof-verify each time for data we already have.
-- **Truncates at 100 and never paginates** — with ≥100 requests, newer (or, by
- `$createdAt asc`, older) legitimate requests are **never fetched**. A spammer
- (or a popular identity) **buries real requests permanently**.
-- **No durable high-water / cursor** — no notion of "what's new since last sweep".
-
-### 1.2 Stage 2 — contact-profile sync is entirely absent
-
-`packages/rs-platform-wallet/.../network/profile.rs::sync_profiles` runs over
-`identity_manager.all_identities()` — only **managed** identities (our own),
-never contacts (`manager/accessors.rs:54`). So we publish and refresh **our own**
-profile but **never fetch a contact's** displayName / avatar / publicMessage. The
-UI shows only a raw identity id (or a local alias). Neither `EstablishedContact`
-nor any incoming-request sender has a cached profile anywhere.
-
-## 2. The reference — Android `PlatformSyncService`
-
-Verified 2026-06 against `github.com/dashpay/dash-wallet` +
-`github.com/dashpay/kotlin-platform` (the current JVM platform lib,
-`org.dashj.platform:dash-sdk-*`; **not** the stale `android-dashpay`). One
-re-entrancy-guarded ticker (`TickerFlow(15.seconds)`) runs, in order:
-
-```
-updateContactRequests() // stage 1: incremental, paginated, high-water
- → discovers userIds (contacts + pending senders)
-updateContactProfiles(userIds) // stage 2: batch whereIn $ownerId, cache by userId
-checkDatabaseIntegrity()/FixMissingProfiles() // self-heal missing profiles
-```
-
-- Stage 1 high-water: `SELECT MAX(timestamp)` per direction; **10-min overlap
- rewind**; incremental `$createdAt > afterTime` + `startAfter` cursor +
- `limit(-1)` = retrieve-all.
-- Stage 2 fetches profiles for the userIds drawn from contact-request rows
- (**including pending incoming senders** — that's how the request UI shows a
- requester's name/avatar), keyed in a `dashpay_profile` table by `userId`,
- independent of relationship state.
-
-## 3. Goal
-
-1. Make our **contact-request** sync incremental, fully-paginated,
- high-water-tracked, and skew-safe, for **both** directions — no truncation,
- each request fetched ~once, a flood can't bury anything.
-2. Add **contact-profile** sync: fetch established contacts' **and pending
- incoming senders'** profiles in batches, cache them (id-keyed) so the UI shows
- name + avatar on both the contacts and the requests screens, refresh, and
- self-heal any missing profile without unbounded re-querying.
-
-## 4. Design
-
-### 4.1 High-water cursor (stage 1)
-
-**Storage (resolves Q-a).** Android keeps every contact-request row and derives
-`MAX(timestamp)`. Our model collapses requests, so we can't `MAX()` a raw table.
-We persist **two scalar fields on `ManagedIdentity`** — `high_water_received_ms:
-Option` and `high_water_sent_ms: Option` — riding the existing
-`IdentityEntry` snapshot (changeset → both persisters → FFI restore), **not** a
-separate table. Two integers per identity need no relational shape.
-
-**The advance invariant (the heart of stage-1 correctness).** Get this wrong and
-we reintroduce the burying bug. The cursor:
-
-1. **Advances only on a fully-exhausted, error-free paginate** of that direction.
- "Exhausted" = a page returned `< limit` docs (possibly empty); a final page of
- exactly `limit` requires one more fetch to confirm. **Any** fetch/proof error
- mid-loop ⇒ **do not advance that direction's cursor this sweep** (leave it at
- the prior value; the overlap re-fetches next sweep).
-2. **Advances to `max($createdAt)` over every doc *fetched* this sweep** —
- *including* docs that ingest then parse-skips, collapses
- (`newest_received_per_sender`), or suppresses (ignore/tombstone). The cursor
- records **fetch-completeness, not ingest-success**. Ignore `unwrap_or(0)`
- sentinels: advance to the max of *present* (`Some`) timestamps only, and never
- below the current value.
-3. **Never stamps to wall-clock `now`.** On a zero-doc fetch the cursor is left
- unchanged. States: `Absent` ⇒ query `$createdAt > 0` (full); `Present(t)` ⇒
- query `$createdAt > (t − OVERLAP_MS)`.
-
-**Why cursor-loss is safe (the written contract):** every collapsed / suppressed
-doc is, by construction, deterministically reproducible from a full re-fetch of
-the immutable on-chain set. So **under-shoot is free** (a lost/low cursor just
-triggers one full re-fetch; ingest is a fixpoint) and **over-shoot buries**.
-Therefore **restore tolerates only under-shoot**: on any restore-consistency
-doubt, clamp the cursor to `min(persisted, max($createdAt) over restored contact
-rows)`, or reset to `0`. A restored-too-high cursor is a correctness bug.
-
-**`OVERLAP_MS` is correctness-load-bearing, not cosmetic.** The lower bound is
-exclusive (`>`) and the `userIdCreatedAt` index is non-unique on `$createdAt`, so
-multiple requests can share a `$createdAt` at a page boundary. The overlap is
-what re-includes them; **`OVERLAP_MS = 0` is an invalid configuration**, not a
-tuning knob. Default `10 * 60_000` (copy Android).
-
-### 4.2 The request query (rs-sdk, stage 1)
-
-`fetch_received_contact_requests` / `fetch_sent_contact_requests` gain
-`after_created_at: Option` + cursor pagination:
-
-```rust
-where: [ toUserId == me, $createdAt > (high_water − OVERLAP_MS) ]
-order_by: $createdAt asc // REQUIRED — binds the userIdCreatedAt index and
- // avoids the "verified-absent" proof trap
-start: StartAfter(last_doc_id) // ephemeral, per-loop pagination cursor
-```
-
-Two distinct cursors, do not conflate: within-sweep pagination uses
-`Start::StartAfter(last_document_id)` (a 32-byte doc id, per-loop); the **durable
-high-water** persists `max($createdAt)` (cross-sweep, §4.1). Loop pages until
-exhausted (§4.1 rule 1). **Precondition (Q-c, stage 1):** before replacing the
-working `limit:100` query, verify on testnet that the paginated `$createdAt > t`
-+ `StartAfter` form returns a known existing doc (not a verified-absent empty
-proof) — the current query's `order_by` comment documents this exact trap.
-
-### 4.3 Request sweep flow (platform-wallet `sync_contact_requests`)
-
-1. Read `high_water_received` / `high_water_sent` (Absent ⇒ full).
-2. Fetch received `> (hw_received − OVERLAP)`, paginated; fetch sent likewise.
-3. Ingest via the existing path — `newest_received_per_sender` collapse, ignore
- suppression, auto-establish. **Idempotency is load-bearing**: the overlap
- re-delivers seen docs every sweep, so ingest MUST be a fixpoint (it is).
-4. **Per direction, iff its paginate exhausted without error** (§4.1 rule 1):
- advance the cursor to the max `$createdAt` *fetched* this sweep (§4.1 rule 2).
- On any error, skip the advance for that direction.
-
-### 4.4 Contact-profile fetch (rs-sdk + platform-wallet, stage 2)
-
-**Query (resolves Q-c stage 2 + Q-cap).** New `fetch_profiles_for(owner_ids)`:
-
-```rust
-where: [ $ownerId In [id0, id1, …] ] // ≤ IN_CAP ids per query
-order_by: [] // EMPTY — unique ownerId index, no trap
-start: None // each owner yields ≤1 profile; no pagination
-```
-
-The `profile` doctype has a **unique single-property `ownerId` index**, so an
-`In $ownerId` set lookup proves presence/absence cleanly with **empty
-`order_by`** and **no pagination** (mirrors the working `profile.rs` point query,
-Equal→In). `IN_CAP = 100` is a **hard cap** enforced at query-build
-(`rs-drive/src/query/conditions.rs:361`); the `In` array **rejects duplicates**
-(`:368`) and **rejects empty** (`:355`). So the caller **dedups** the id set and
-**skips** the query entirely when a chunk (or the whole target set) is empty.
-
-**Target set (resolves the §4.3-vs-§4.4 contradiction): iterate the FULL set
-every sweep, not "touched ids".** Each sweep, collect:
-`{ established_contacts[].contact_identity_id } ∪ { incoming_contact_requests[].sender }`
-across managed identities, **dedup**, and **skip ids that are themselves managed
-identities on this wallet** (their profile is their own `dashpay_profile`, which
-is authoritative — see §4.7). The stage-1 "touched this sweep" set is at most a
-*fetch-these-first hint*, never the iteration set — the existing aggregator
-discards `sync_contact_requests`'s return value anyway, and a "touched-only" set
-would break both self-heal and first-run backfill (every pre-existing contact is
-uncached but untouched).
-
-**Filter, then chunk, then fetch:**
-
-1. Drop ids that are **cached and fresh**, and ids that are **confirmed-absent
- and checked recently** (negative cache, see below). What remains is the fetch
- set — on first run after upgrade this is *every* contact (the dominant,
- expected first-sweep cost; bounded by contact count).
-2. Chunk the remaining ids into groups of `IN_CAP`, run one `In` query per chunk.
-3. **Per-chunk log-and-continue isolation:** a chunk's fetch/proof failure logs
- and continues to the next chunk; the freshness/checked markers advance **only
- for ids in successfully-fetched chunks**, never sweep-wide on partial failure.
- A persistently-failing chunk must not starve the others.
-
-**Self-heal & the no-profile negative cache.** A contact may have **no `profile`
-document on-platform** (profiles are optional). The `In` query simply omits them.
-Without a guard, "cached? false" stays true forever and they're re-queried every
-sweep — the unbounded-retry pathology `payment_channel_broken` (G1c) exists to
-avoid. So record a **confirmed-absent marker with a checked-at timestamp**; the
-fetch set targets "no cached profile **and** not checked within the backoff
-window". Self-heal then *is* the normal path (an uncached/expired contact re-enters
-the fetch set) — no separate `FixMissingProfiles` loop.
-
-### 4.5 Profile storage — **Option B** (id-keyed cache)
-
-A new map on `ManagedIdentity`:
-
-```rust
-pub contact_profiles: BTreeMap,
-// ContactProfileEntry = { profile: Option, checked_at_ms: u64 }
-// profile: Some(..) = fetched & present; None = confirmed-absent (negative cache)
-// checked_at_ms: last fetch attempt, drives the self-heal backoff
-```
-
-Chosen over a field on `EstablishedContact` because the cache must serve **every
-relationship state** — established contacts, **pending incoming-request senders**
-(requests screen), and **ignored senders** (future Ignored list) — none of which
-share one struct. This is the product decision (§4.6) and matches Android's
-relationship-independent `dashpay_profile` table. Plumbing (the `dashpay_payments`
-5-site pattern; **the two most-forgotten are the merge rule and the store-side
-apply** — miss either and contacts silently vanish on relaunch):
-
-1. field on `ManagedIdentity`;
-2. `IdentityEntry` field + `from_managed` (`changeset.rs`);
-3. **merge rule** in `IdentityChangeSet::merge` — per-key last-write-wins (the
- `dashpay_payments` merge at `changeset.rs:489-495` is the template);
-4. FFI: a **contact-keyed** accessor (distinct from the existing identity-keyed
- own-profile one), e.g. `platform_wallet_get_contact_profile(wallet,
- owner_identity_id, contact_identity_id) -> profile?`, + an
- `IdentityRestoreEntryFFI` field + `restore_contact_profiles` fn
- (mirror `restore_dashpay_payments`, `persistence.rs`);
-5. SwiftData `PersistentDashpayProfile` keyed by `(ownerId, contactId)` (mirror
- `PersistentDashpayPayment`) + the **store-side write/apply**.
-
-**Boundary invariant:** `contact_profiles` holds **only the five public profile
-fields** parsed from the on-chain `profile` document. It must never receive any
-field derived from the encrypted `contactInfo.privateData` path (which carries
-private relationship state — alias/note/hidden/ignore). Keep these two stores
-distinct so the contactInfo migration (Spec 1) can't accidentally cross them.
-
-### 4.6 Scope & privacy
-
-**Scope (product decision): established contacts + pending incoming-request
-senders now; ignored senders ride the same cache when the Ignored list lands.**
-This matches Android's observable behavior (requester names in the request UI).
-
-**Privacy posture (resolves Q-scope).** Fetching a *pending* sender's profile is a
-public read, but issuing `whereIn $ownerId [sender_ids]` right after their
-requests land is a query-pattern an observer could correlate with your inbound
-set. We **accept** this because the marginal leak is small: the contact-request
-documents are *already public* (indexed by `[toUserId, $createdAt]`), and the
-DAPI node serving our `toUserId == me` request query — which we must run —
-**already learns the entire inbound set**. Fetching those public profiles adds
-little. This is materially weaker than the R1 leak (which *creates a new on-chain
-document* about a non-contact). Documented and accepted; the R1 track may later
-minimize query-pattern metadata if desired.
-
-### 4.7 Cache write semantics
-
-- **Full-REPLACE, not merge.** A fetched profile document is the authoritative
- *complete* state for that owner; storing it **overwrites** the cached entry via
- `profile_from_properties` (full parse). This is the **opposite** of the
- own-profile *update* path (`merge_profile_properties`, read-modify-write) — do
- **not** reuse that helper here, or a contact who *removes* `avatarUrl` would
- keep showing a stale avatar forever.
-- **All-empty parse ⇒ confirmed-absent, not cached-present.** A doc that parses to
- an all-`None` profile is treated as a negative-cache hit (§4.4), not a fresh
- empty profile, so self-heal keeps it honest.
-- **Persist only on change.** Compare the fetched profile to the cached one before
- writing; emit no changeset when unchanged. This keeps the deferred-Q-inc
- "refetch-all each sweep" first cut a **persistence fixpoint** — no write
- amplification, the same discipline stage 1 enforces.
-- **`avatarUrl` validation at insert.** Validate before caching: **`https://`
- scheme only**, length-capped (state the contract's max). Treat the cached url as
- **untrusted** input downstream — it is attacker-controlled and the UI will load
- it (an unsanitized `http:`/`file:`/`javascript:` url is an SSRF / tracking-pixel
- vector; a tracking url tied to your IP confirms "you have this contact").
-- **own-vs-contact authority.** If a target id is itself a managed identity on this
- wallet, skip the contact fetch; that identity's own `dashpay_profile` wins.
-
-### 4.8 Driver wiring (`dashpay_sync.rs`)
-
-Add `sync_contact_profiles()` as a **distinct** step **between** the existing
-`sync_profiles()` (own identities) and `sync_contact_infos()`. It is
-**log-and-continue, not error-returning** (matches `sync_contact_infos` /
-`reconcile_incoming_payments`): a contact-profile fetch failure degrades *display*
-only and must never change the sweep's pass/fail outcome. **Do not** fold it into
-`sync_profiles` — that function is scoped to `all_identities()` (own) and writes a
-different store. Ordering: it must run **after** `sync_contact_requests` so a
-contact established this sweep is fetched the same tick.
-
-### 4.9 Interactions (specify, don't discover)
-
-- **Un-ignore resync (deferred to the ignore refactor, but constrained here):**
- un-ignore must re-fetch the un-ignored sender's requests. The
- ignore/reject tombstone is keyed by `(sender, accountReference)` and **does not
- store `$createdAt`**, so a *precise* "rewind the cursor past their `$createdAt`"
- is **not implementable from the tombstone alone**. Therefore: **un-ignore ⇒
- clear (reset to Absent) the received cursor** → one full re-fetch (cheap, safe
- per §4.1). If a targeted rewind is ever wanted, add `$createdAt` to the tombstone
- first. The ignore work owns the call site; this is the mechanism constraint.
-- **contactInfo-before-contactRequests ordering:** DIP-15 says fetch contactInfo
- first (so contacts don't flicker on `displayHidden`). Out of scope here; noted.
-- **Cursor as at-rest metadata:** the high-water timestamps are derived from public
- on-chain `$createdAt`, but they are a session-activity residue at rest — exclude
- the cursor (and the whole DashPay store) from iCloud backup, since a device-local
- ignore/blocklist and activity residue should not sync to backup.
-
-## 5. Non-goals
-
-- Changing the sweep cadence.
-- The **account** half of `checkDatabaseIntegrity` (we already rebuild contact
- accounts) — only the **profile** half is in scope (§4.4 self-heal).
-- **Avatar image bytes / rendering** — we cache the fields (`avatarUrl` +
- hashes); downloading/showing the image is app-layer (but the url is validated
- at cache insert, §4.7).
-- **Per-profile `$updatedAt`-incremental refetch (Q-inc).** The composite
- `$ownerId In […] AND $updatedAt > marker` is **not provable in one query** (an
- `In` on the first index field plus a range on the second isn't a contiguous
- index range). The first cut refetches all contact profiles each sweep (bounded
- by contact count, and a persistence fixpoint per §4.7); a real incremental would
- be per-owner equality (loses the batch) or client-side staleness — a follow-up.
-
-## 6. Implementation surface
-
-**Stage 1 (commit 1):**
-- `rs-sdk/.../dashpay/contact_request_queries.rs` — `after_created_at` + the
- `StartAfter(doc_id)` pagination loop; drop `limit:100, start:None`.
-- `platform-wallet/.../network/contact_requests.rs::sync_contact_requests` —
- read/advance cursors per §4.1/§4.3 (advance gated on exhaustion + no error).
-- `ManagedIdentity` gains `high_water_received_ms` / `high_water_sent_ms`
- (`Option`) + `IdentityEntry` + merge + both persisters + FFI restore.
-
-**Stage 2 (commit 2):**
-- `rs-sdk/.../dashpay/` — `fetch_profiles_for(owner_ids)` (empty `order_by`, `In`,
- dedup, chunk at `IN_CAP=100`, skip-empty).
-- `platform-wallet/.../network/profile.rs` — `sync_contact_profiles` (full-set
- target, negative cache, per-chunk isolation, full-replace, persist-on-change,
- `avatarUrl` validation); reuse `profile_from_properties`.
-- `ManagedIdentity.contact_profiles` + the 5 plumbing sites (§4.5), incl. the
- contact-keyed FFI accessor + `PersistentDashpayProfile` SwiftData model.
-- `dashpay_sync.rs` — wire `sync_contact_profiles` per §4.8.
-- UI bind in the **real** consumers: `Views/DashPay/ContactsView.swift` (list
- row name/avatar), `ContactDetailView.swift` (header), `ContactRequestsView.swift`
- (requester name/avatar), via the existing `DashPayContactMeta` / `DashPayProfileView`.
- (There is **no** `FriendsView`.)
-
-## 7. Test plan
-
-**Stage 1:**
-- **Incremental:** two sweeps; second issues `$createdAt > hw` and ingests only the
- delta (no re-fetch beyond the overlap).
-- **No-bury:** 150 requests → all eventually fetched via pagination.
-- **Equal-timestamp page boundary:** N>limit requests sharing one `$createdAt`
- straddling a page cut → all eventually ingested (pins the overlap as
- correctness, not just skew).
-- **Partial-page failure:** inject a page-2 error → the cursor does **not** advance
- and the next sweep re-fetches from the old high-water.
-- **Collapsed-doc reachability:** after a cursor wipe, an older-ref doc that was
- collapsed away reappears (proves cursor-loss safety / under-shoot).
-- **Restore over-shoot guard:** a restored cursor higher than the restored contact
- rows still re-fetches the missing contacts (over-shoot clamped to under-shoot).
-- **Idempotency:** overlap re-delivery creates no phantom rows / duplicate writes.
-
-**Stage 2:**
-- **Batch/chunk + dedup:** N>IN_CAP contacts (with a duplicate id) → ⌈N/IN_CAP⌉
- chunked queries, deduped, all cached.
-- **First-run backfill:** a wallet restored with M established contacts and zero
- cached profiles fetches all M on the first sweep even though stage 1 ingests no
- new request.
-- **Pending-sender profile:** a pending incoming-request sender's profile is
- fetched and reachable via the contact-keyed FFI accessor.
-- **No-profile negative cache:** a contact with no on-platform profile is fetched
- at most once per backoff window, not every sweep.
-- **Chunk isolation:** chunk 2 of 3 fails → chunks 1 & 3 cache, chunk 2's contacts
- retried next sweep (not marked done).
-- **Shrinking profile (full-replace):** cache a full profile, then ingest a doc
- missing `avatarUrl` → cached `avatar_url` becomes `None`.
-- **Persist-on-change fixpoint:** a steady-state sweep with unchanged profiles
- writes zero changesets.
-- **avatarUrl validation:** a profile with a non-`https` url is rejected/sanitized
- at cache insert.
-- **own-vs-contact:** a contact that is also a managed identity resolves to the
- own `dashpay_profile`, not a duplicate contact fetch.
-- **Round-trip:** a contact profile survives relaunch (changeset → persister →
- restore), like `dashpay_payments`.
-
-## 8. Open questions (most resolved by the review)
-
-- **Resolved — Q-a** (cursor storage): two scalar `Option` fields on
- `ManagedIdentity` (not a table).
-- **Resolved — Q-store:** Option B (id-keyed `contact_profiles`), per the
- product decision (§4.5/§4.6).
-- **Resolved — Q-scope:** established + pending senders; privacy accepted (§4.6).
-- **Resolved — Q-c:** stage-1 keeps `order_by $createdAt`; stage-2 uses empty
- `order_by` on the unique `ownerId` index, no pagination. (Stage-1 paginated
- form still needs the one-time testnet proof check, §4.2.)
-- **Resolved — Q-cap:** `IN_CAP = 100`, dedup, skip-empty.
-- **Resolved — Q-inc:** not provable as a single batch query; deferred (§5).
-- **Open — Q-b:** `OVERLAP_MS = 10 min` (copy Android) — keep, but confirm it
- comfortably exceeds observed platform time-skew; **must stay > 0** (§4.1).
-- **Open — Q-backoff:** the no-profile negative-cache recheck interval (§4.4) —
- propose "once per N sweeps" or a wall-clock window; pick during impl.
-- **Open — Q-checked-clock:** the `checked_at_ms` backoff may use wall-clock
- (acceptable — it gates re-query cost, not cursor correctness) vs a sweep
- counter; decide during impl.
-
-## 9. Review resolutions (traceability)
-
-Folded in from the 5-lens review (feasibility / scope / adversarial / security /
-flow). The load-bearing changes vs the first draft:
-
-- **Cursor advance invariant rewritten** (§4.1) — advance only on error-free
- *exhausted* pagination, over docs *fetched* (not *applied*), never wall-clock,
- under-shoot-only on restore, overlap mandatory. Closes the two CRITICAL burying
- holes (advance-past-failed-page, advance-past-collapsed-doc).
-- **Cursor storage simplified** to two scalar fields, not a table (Q-a).
-- **Stage-2 query shape resolved from the contract indices** (§4.4) — unique
- `ownerId` index ⇒ empty `order_by`, no pagination, `IN_CAP=100`, dedup,
- skip-empty (Q-c, Q-cap). Q-inc shown unprovable as a batch.
-- **Stage-2 negative cache + per-chunk isolation + full-replace +
- persist-on-change** added (§4.4/§4.7) — closes infinite-refetch, partial-failure
- starvation, stale-field, and write-amplification holes.
-- **Target set = full set (established + pending), every sweep** (§4.4) — closes
- the §4.3-vs-§4.4 contradiction and the first-run-backfill gap.
-- **Storage = Option B** with the full 5-site plumbing called out (merge rule +
- store-apply emphasized), public-data boundary (§4.5).
-- **avatarUrl validation** + **privacy posture for pending-sender fetch** (§4.6/4.7).
-- **Driver hook pinned** as a distinct log-and-continue step (§4.8); **UI surface
- corrected** to the real views (no `FriendsView`).
-- **Un-ignore = clear-cursor** because the tombstone lacks `$createdAt` (§4.9).
diff --git a/docs/sdk/CODE_REVIEW_NOTES.md b/docs/sdk/CODE_REVIEW_NOTES.md
deleted file mode 100644
index af6f1b20b0f..00000000000
--- a/docs/sdk/CODE_REVIEW_NOTES.md
+++ /dev/null
@@ -1,214 +0,0 @@
-# Final multi-agent code review: Kotlin DashPay registration keys
-
-**Review date:** 2026-07-21
-**Branch:** `feat/kotlin-sdk-dashpay-registration-keys`
-**Implementation base:** `6efa83bb53`
-**Reviewed implementation commits:** `70b0852edb`, `3a64940af8`,
-`c31ca592d4`, `4a25717471`, plus the concurrently-added persistence refactor
-`5147d5baa9`
-
-## Review method
-
-Three independent agents reviewed the actual diff and call graph under separate
-lenses, followed by a primary-agent source audit and full build/test run:
-
-1. Rust wire format, registration invariants, FFI callers, and consensus-facing
- key policy.
-2. Kotlin derivation/persistence/zeroization lifecycle, funding policy, resume
- exclusion, and the helper-reuse decision.
-3. Cross-language fixture and edge-case test adequacy, including the invitation
- caller and environment-bound on-chain assertion.
-
-The agents were instructed to review only and made no edits or commits. While
-the review was running, `5147d5baa9` was committed by an external concurrent
-session; it was reviewed as part of the resulting branch head and was not
-rewritten.
-
-## Verified correct in the implementation
-
-- `parse_pubkey_rows` is a pure `&[u8]` parser. JNI conversion is confined to
- thin update/registration adapters.
-- Empty-list, duplicate-key-ID, and key-ID-0 = MASTER + AUTHENTICATION checks
- are registration-only. The shared update parser still accepts ordinary
- add-key lists without key ID 0.
-- All four Android registration JNI exports use the rich registration decoder:
- resume, Core-funded, Platform-address-funded, and shielded-from-pool.
-- The fifth `decode_identity_pubkeys` caller, invitation claim, still uses the
- shared FFI decoder and therefore receives duplicate-ID rejection without
- receiving the JNI-only key-ID-0 policy.
-- `role_for_registration_key_id` and `decode_pubkeys_blob` are deleted, not
- merely superseded.
-- Kotlin explicitly rebuilds the base roles as MASTER/AUTHENTICATION,
- CRITICAL/AUTHENTICATION, HIGH/AUTHENTICATION, and CRITICAL/TRANSFER.
-- Keys 4/5 are ECDSA secp256k1, ENCRYPTION/DECRYPTION, MEDIUM, writable, and
- bounded to the canonical DashPay contract's `contactRequest` document type.
-- Fresh registration performs one six-key derivation pass. It does not derive
- an overlapping base-four set first.
-- Resume derives and submits only the base four keys. The DashPay pair is not
- added to an already-funded asset-lock resume.
-- The checked-in golden binary is genuinely shared: Kotlin compares its encoder
- output to that exact resource, while Rust includes and decodes the same file
- and pins its contract ID to `dashpay_contract::ID_BYTES`.
-- Strict codec coverage includes truncation, legacy layout, trailing bytes,
- invalid bounds/boolean bytes, negative IDs, interior NUL, duplicate IDs, and
- update-without-key-0 behavior. FFI tests cover invalid DPP role bytes.
-
-## Findings fixed during this review
-
-### 1. Private material survived two failure windows
-
-`IdentityKeyPreview.decodeAll` wiped the JNI blob only after a complete parse.
-A malformed/truncated blob could leave both the source blob and already-copied
-private scalars unsanitized. Separately, a blocking JNI preview could finish
-after its coroutine was cancelled; `withContext` would then discard the
-secret-bearing result before the caller reached provisioning cleanup.
-
-Fix:
-
-- Decode under `try/catch/finally`, wiping the source blob on every exit and
- wiping all partially decoded private arrays on failure.
-- Add `opWithCleanupOnCancellation`, which scrubs a completed result if prompt
- cancellation discards it during dispatcher handoff.
-- Route both registration preview APIs through that cleanup-aware gate.
-- Add malformed-blob and blocking-JNI cancellation regressions. Both tests were
- observed failing before the production fix and passing afterward.
-
-### 2. Six-key Core funding minimum was not enforced
-
-The app accepted any numeric Core amount even though six creation keys raise
-the protocol floor from 228,000 to 241,000 duffs. A below-floor asset lock can
-be broadcast before Platform rejects the identity create.
-
-Fix:
-
-- Mirror `IdentityCreateTransition::calculate_min_required_fee_v1` for the
- current key count.
-- Use one amount-policy function for both submit enablement and the click-time
- preflight.
-- Add boundary tests for the four-key/six-key floors and the resume no-new-funds
- case. The new test was observed failing before the implementation and passing
- afterward.
-
-### 3. Invitation claim lacked its required caller-level regression
-
-The existing duplicate-ID unit test called `decode_identity_pubkeys` directly
-and only stated in a comment that invitation claim used it. It did not exercise
-the fifth caller itself.
-
-Fix:
-
-- Add a valid-invitation/duplicate-row test through
- `platform_wallet_claim_invitation`.
-- Assert duplicate rejection happens before wallet lookup, signer use, or
- network work, and that output sentinels remain zeroed.
-
-### 4. Coverage and documentation were weaker than the implementation claims
-
-Fix:
-
-- Correct the Rust module documentation: the pure structural parser does not
- validate DPP role discriminants; the downstream FFI conversion does.
-- Add direct invalid key-type/purpose/security-level FFI coverage and a
- MASTER-with-wrong-purpose registration invariant test.
-- Strengthen Kotlin tests to prove the exact public-key storage key, non-zero
- scalar at persistence time, wallet ownership, post-persist scrubbing,
- DashPay key type/purpose/security/read-only flags, and exact bounds.
-- Correct the helper rationale: the add-key flow persists before broadcast;
- the real incompatibilities are its existing-key/max+1 update policy and
- per-slot derivation, versus registration's fixed 0..N batch policy.
-
-## PR #4173 automated-review follow-up
-
-The first automated PR review found three additional blocking regressions. All
-three were independently traced to the current source and fixed:
-
-1. **Mutable controls could redirect an in-flight registration.** The click
- handler captured the key-count choice before suspension but later reread
- `fundingSource` and `selectedRecoveryLock`. It now resolves source, amount,
- identity index, and a defensive copy of the selected lock into one immutable
- submission snapshot before the coroutine starts. Preparation, dispatch,
- coordinator tracking, and navigation use only that snapshot. A regression
- mutates the backing form values and lock txid after capture and verifies the
- submission is unchanged.
-2. **Platform-address packing did not reserve the six-key creation fee.** The
- default native strategy deducts the fee from the post-spend remainder of
- BTreeMap input 0. The packer now mirrors the consensus formula
- `2,000,000 + 6,500,000 * keyCount + 500,000 * inputCount`, models Rust's
- `(address type, unsigned hash)` ordering, and selects the smallest input set
- that contributes the requested identity balance while leaving the full fee
- on input 0. This preflight runs before key derivation/persistence. The
- reviewer's 70M-balance/30M-spend case was observed failing before the fix;
- exact one-input and two-input boundaries are also covered.
-3. **Resume lost its HD-slot provenance check.** Rich consensus rows do not
- carry a derivation index, so replacing `IdentityKeyPreview` had deleted the
- old per-key guard. Provisioning now returns a `RegistrationKeySet` that keeps
- the common preview identity index beside the rich rows, rejects mixed-slot
- previews before persistence, and resume verifies the set index matches the
- tracked lock before JNI. The wrong-slot regression was observed calling JNI
- before the fix and passing afterward.
-
-The full Kotlin SDK/app build, unit-test suites, and instrumented-test-source
-compilation passed after these fixes. The PR's initial emulator check hit the
-workflow's documented Android Keystore unlock-state race; its failed-job rerun
-passed the complete native build and API-35 instrumented suite without a code or
-workflow change.
-
-## Left for human judgment or environment-bound verification
-
-1. **On-chain bounds assertion is still manual.** No unit test can prove what a
- testnet node persisted. Register a fresh identity, fetch it, and assert keys
- 4/5 contain the exact DashPay contract ID plus `contactRequest` bounds. Then
- run Add Contact as a separate smoke test; Add Contact success alone is not
- proof of the bounds.
-2. **Public Kotlin/logical wire compatibility.** Public fresh-registration
- methods now consume rich `IdentityPubkey` lists, resume consumes a
- provenance-carrying `RegistrationKeySet`, and the opaque `byte[]` layout
- changed without a version/magic prefix. The parser fails closed on
- legacy/malformed shapes, but an old Kotlin artifact must not be paired with
- the new native library. Release coordination or an explicit compatibility
- layer remains a product/API decision.
-3. **No full Compose four-way dispatch seam test.** Static tracing confirms all
- three fresh paths pass the same six-row list and resume passes the four-row
- list; policy tests cover the funding-source gate and immutable submission
- snapshot. Extracting a larger pure screen-dispatch seam solely for
- orchestration testing is a maintainability choice, not a discovered
- production defect.
-4. **Kotlin still trusts Rust's preview row keypair correspondence.** Swift
- independently revalidates private/public correspondence. Kotlin documents
- why it does not duplicate private-key math; whether to add an equivalent
- native validation surface remains a parity-hardening decision.
-5. **No existing-identity backfill.** Already-created Android identities still
- need the Add Identity Key flow. This remains the reviewed non-goal.
-6. **Dedicated provisioning helper.** Keeping `DashpayKeyProvisioning` is
- justified by the fixed registration IDs and one batch derivation. Commit
- `5147d5baa9` shares the dangerous persist-and-scrub primitive with
- `IdentityKeyAdditionFlow`, avoiding lifecycle duplication while preserving
- the distinct policies.
-
-## Verification results
-
-All required local verification passed:
-
-```text
-cargo test -p rs-unified-sdk-jni --lib
- 29 passed
-
-cargo test -p platform-wallet-ffi --lib
- 201 passed
-
-env RUSTC_WRAPPER= cargo clippy --workspace --all-features
- passed (the configured sccache was not permitted in the sandbox)
-
-cargo fmt --check --all
- passed
-
-JAVA_HOME=/opt/homebrew/opt/openjdk@17 \
-ANDROID_HOME=/opt/homebrew/share/android-commandlinetools \
-./gradlew :sdk:assembleDebug :sdk:testDebugUnitTest \
- :app:assembleDebug :app:testDebugUnitTest \
- :sdk:compileDebugAndroidTestKotlin
- passed
-```
-
-Testnet/device registration, the on-chain bounds assertion, and Add Contact
-were intentionally not attempted because they are environment-bound.
diff --git a/docs/sdk/KOTLIN_SWIFT_SHARED_PARITY_SPEC.md b/docs/sdk/KOTLIN_SWIFT_SHARED_PARITY_SPEC.md
deleted file mode 100644
index 25aed864dd6..00000000000
--- a/docs/sdk/KOTLIN_SWIFT_SHARED_PARITY_SPEC.md
+++ /dev/null
@@ -1,705 +0,0 @@
-# Kotlin/Swift SDK parity and shared-logic consolidation
-
-**Status:** REVIEWED v2 — approved after Swift/Rust FFI, Android/JNI/Room, and
-adversarial architecture reviews
-**Baseline:** PR #3999, `feat/kotlin-sdk-and-example-app` at
-`6dbc72a54df72d26eb9c4a014b425d2b95134e4e`
-**Scope:** `rs-platform-wallet`, `rs-platform-wallet-ffi`, `rs-unified-sdk-jni`,
-`swift-sdk`, `kotlin-sdk`, SwiftExampleApp, and KotlinExampleApp
-**Related specifications:** `docs/dashpay/DIP15_INVITATIONS_SPEC.md`,
-`docs/dashpay/KOTLIN_MIGRATION_SPEC.md`,
-`docs/dashpay/KOTLIN_MIGRATION_FOLLOWUPS_SPEC.md`, and
-`docs/dashpay/PENDING_CONTACT_CRYPTO_RELOCATION_SPEC.md`
-
-**Implementation state on this branch:** partial, not release-complete. The
-release-blocker slices are implemented at source/test level, but the executable
-manifest remains authoritative for unclosed device, process-restart, and legacy
-store gates. Android invitations and the S1–S4 shared-policy moves are explicitly
-permitted follow-ups; this commit must not be presented as completing those later
-slices.
-
----
-
-## 1. Problem
-
-PR #3999 establishes an Android/Kotlin SDK and ports SwiftExampleApp. The port is
-large enough that file- or screen-count parity is no longer a reliable correctness
-measure. Review found three kinds of drift:
-
-1. Host persistence callbacks do not implement the same Rust persistence contract.
- Some Android omissions disable shared features; one Android callback writes
- incorrect authoritative data.
-2. Protocol or wallet policy is duplicated in Kotlin and Swift even when Rust
- already owns, or should own, the rule. The copies have started to diverge.
-3. The parity document records source-file presence rather than executable feature
- capability, restart behavior, and protocol-domain coverage.
-
-The goal is not literal host-code equality. Kotlin/Swift UI, lifecycle, secure
-storage, and database adapters remain platform-native. The goal is one shared
-implementation of protocol/wallet decisions and equivalent host persistence,
-recovery, and example-app capability.
-
-### 1.1 Baseline and already-integrated work
-
-Implementation starts from exact PR head `6dbc72a54d` or a descendant. That head
-already contains these squash integrations:
-
-- `516b265bf5` / #4106 — shared wallet deduplication and dead FFI/JNI removal;
-- `f74465227f` / #4041 — shared and iOS DIP-13 invitation implementation;
-- `97477d1c88` / #4093 — generalized existing-asset-lock top-up and iOS resume;
-- `aba6af2420` / #4126 — shared and iOS Orchard viewing-key persistence.
-
-The `refactor/platform-wallet-dedup`, `feat/dip15-dashpay-invitations`, and
-`feat/kotlin-sdk-dashpay-migration` worktrees are historical design/review
-references. They are behind the baseline and **must not be merged or cherry-picked**.
-This effort adds the missing Android adapters and new shared APIs on the exact
-baseline while preserving all later seed-binding, provider, and rollback fixes.
-
-## 2. Non-negotiable invariants
-
-1. **Rust owns protocol and wallet policy.** Coin selection, reservation,
- authorization, pricing, network endpoint interpretation, protocol constants,
- proposal-rule selection, and recovery state machines must not be independently
- reimplemented by Kotlin and Swift.
-2. **Host bridges adapt; they do not orchestrate.** JNI and Swift wrappers may map
- types, dispatch queues, callbacks, cancellation, and errors. A host wrapper must
- not compose multiple fallible Rust calls into a new wallet transaction state
- machine.
-3. **Persistence contracts are capability-checked.** Backends attest individual
- capabilities such as atomic changesets, invitations, Orchard FVKs, provider
- restore, and deferred contact crypto. A feature requiring durability must fail
- closed before broadcast when any required capability is absent. A present no-op
- callback violates the contract.
-4. **Persisted identity is immutable.** Balance/update callbacks may not rewrite an
- address's derivation identity unless the callback explicitly represents a
- derivation-map mutation.
-5. **Protocol integer domains survive every ABI.** Every protocol field whose valid
- consensus domain spans full `u64` must round-trip across Kotlin and Swift. A
- narrower carrier is permitted only where Rust enforces a narrower consensus
- bound.
-6. **Recovery is part of feature parity.** A flow is not “ported” until it can resume
- after process death at every point where durable state or on-chain value exists.
-7. **Parity is executable.** Every capability row names an automated test or an
- explicit device/testnet gate. Counts derived from source files are informational
- only.
-
-## 3. Ownership boundary
-
-| Concern | Shared Rust | Host SDK | Example app |
-| --- | --- | --- | --- |
-| Transaction input selection and reservation | Own | Invoke one composite API | Collect user intent |
-| Masternode endpoint discovery | Own | Map endpoint result | Display diagnostics only |
-| Token authorization, group rules, price quote | Own | Map typed decision/quote | Render decision and collect inputs |
-| Protocol constants and amount validation | Own | Preserve exact domain | Format values |
-| Recovery state machine | Own | Persist/restore required records and invoke resume | Present resumable operations |
-| Persistence schema and callbacks | Define callback semantics/capabilities and ABI version | Implement in Room/SwiftData/Keychain/Keystore | Never bypass SDK persistence |
-| UI/navigation/lifecycle | No | Expose async/cancellation-safe API | Own |
-
-The following duplication is intentional: Room versus SwiftData entities and DAOs,
-Android Keystore versus Apple Keychain, Compose versus SwiftUI, and ABI type mapping.
-Those implementations must nevertheless satisfy the same Rust callback semantics.
-
-## 4. Correctness blockers
-
-**Current status (manifest reconciliation):** C1-C3 are the release-blocker
-slices described in the implementation-state note above as implemented at
-source/test level. For Kotlin, the manifest marks all three SDK capabilities
-supported: C1 (`persistence.platform_address_identity`) has a restart-covering
-Room regression; C2 (`core.atomic_send`) has shared reservation/failure unit
-coverage plus the host `CORE-05` manual case, with restart correctly marked
-`not_applicable`; and C3 (`tokens.full_u64_domain`) has Kotlin/JNI/Room/app
-coverage plus a restart-covering device migration test. Swift still has open
-restart gates for C1 and C3, so the manifest remains authoritative. The
-“Required change” text below is retained as historical design rationale, not as
-a claim that these Kotlin changes remain unimplemented.
-
-### C1 — Android platform-address balance persistence corrupts derivation indices
-
-The Android balance callback currently copies callback `accountIndex` and
-`addressIndex` into the durable address row. Conflict-removal events may carry the
-index of a competing address while intentionally leaving the authoritative
-address/index bijection unchanged. Swift already ignores those two callback fields.
-
-**Required change**
-
-- Update only balance, nonce, used state, and height in the Android balance path.
-- Preserve the stored account/address indices for an existing address.
-- Add a regression test that seeds address A at index A, delivers a zero-balance
- callback for A carrying index B, restarts/restores, and asserts A still maps to A.
-- Document the callback semantics beside the JNI callback declaration.
-- C1 is prevention-only on the unreleased PR #3999 baseline and requires no Room
- migration. If a distributed beta database must later be repaired, recovery must
- come from a Rust-authoritative address-pool re-emit, never SQL or host parsing of
- derivation paths.
-
-**Acceptance:** the pre-existing row's account/address tuple and derivation path are
-unchanged; Rust's restored bijection maps the canonical address at index A; and a
-later valid credit to A is accepted. Database-level uniqueness or cleanup of benign
-zero-balance conflict remnants is not required.
-
-### C2 — Core transaction construction is not atomic
-
-The shared builder separates funding from signing. Two concurrent calls can select
-the same UTXO. Android serializes its wrapper, but Swift exposes the split public API
-and other consumers can bypass the Android mutex.
-
-**Required change**
-
-- Add the composite to `rs-platform-wallet` first and expose it through thin C/JNI
- adapters. Selection and recording in the account `ReservationSet` must be one
- indivisible operation: no competing selection may observe the chosen inputs as
- available.
-- The design must not depend on holding the wallet-manager lock across a host
- mnemonic-resolver callback. If key-wallet cannot reserve before signing, add the
- required reservation primitive there or use an explicit per-wallet atomic gate in
- platform-wallet.
-- Add a finalize API that consumes an unfunded/configured builder, atomically funds
- and reserves it, then signs. Validation/signing failure and explicit abandon
- release the reservation; definitive broadcast rejection releases it; ambiguous
- `MaybeSent` retains it under the existing TTL/reconciliation policy.
-- Return a new opaque/V2 signed-transaction handle containing fee, funding account,
- and reservation metadata. Do not extend `FFICoreTransaction` in place or alter an
- existing C layout. Add explicit broadcast and abandon functions.
-- Route Kotlin and Swift convenience sends through the composite. Deprecate the old
- split builder symbols and public Swift sequence without removing their ABI in this
- release.
-
-**Acceptance:** a barrier-forced concurrent same-UTXO test produces disjoint
-reservations or one typed reserved/insufficient-funds failure. Tests also cover
-validation/sign failure, explicit abandon, definitive rejection, ambiguous send,
-builder consumption, and double-free safety.
-
-### C3 — Host SDKs preserve protocol `u64` values end to end
-
-Kotlin previously narrowed token amounts and direct-purchase costs to signed
-`Long`. SwiftData uses its original signed `Int64` balance column as a
-schema-neutral raw-bit carrier and exposes unsigned values at the SDK/UI boundary;
-no alternate iOS balance column or migration-only code path is introduced.
-
-**Required change**
-
-- Public Kotlin token APIs use `ULong` (or one `TokenAmount` value type backed by
- `ULong`). Internal native declarations remain `Long`/`jlong` raw-bit carriers so
- existing JNI names and descriptors remain stable. Kotlin passes `value.toLong()`;
- Rust reinterprets the bits with `as u64` and does not reject the sign bit.
-- `ULong` and `Long` erase to the same JVM carrier. Any deprecated checked-`Long`
- compatibility adapter must therefore have a distinct Kotlin name or explicit
- `@JvmName`, reject negative values before the raw-bit native call, and be listed in
- a per-method compatibility table.
-- Java callers use one documented `BigInteger` or eight-byte adapter layered on the
- same native path. Do not create a parallel native implementation.
-- Audit mint, burn, transfer, set-price, purchase amount/cost, max supply, distribution
- values, results, persistence, comparisons, and UI formatting. Preserve the existing
- unsigned-decimal max-supply JSON behavior.
-- In Room v5, replace signed SQL semantics such as `TokenBalanceEntity.balance` plus
- `WHERE balance > 0` with an order-preserving unsigned representation. Use a fixed
- eight-byte big-endian BLOB (lexicographically unsigned-order-preserving) and
- explicit zero comparison, or document an equally lossless/order-preserving schema;
- values at/above `2^63` must not disappear from DAO results.
-- Separate codec-boundary tests (`0`, `Long.MAX_VALUE`, `2^63`, `u64::MAX`) from
- operation semantics where zero may still be invalid.
-
-**Acceptance:** every full-domain token `u64` round-trips through JNI, Room, DAO
-queries, and UI formatting without loss or signed-order errors.
-
-Direct-purchase quotes additionally follow Drive's operation semantics rather
-than applying a generic full-`u64` arithmetic rule: amount is limited to DPP's
-`2^48 - 1` distribution maximum; single-price multiplication saturates at
-`u64::MAX`; set-price multiplication rejects overflow; and a configured zero
-price remains valid.
-
-**Kotlin source-compatibility accounting:** PR #3999 introduces an unreleased
-Kotlin SDK surface, so the signed declarations below have no published consumer
-contract to deprecate. They are intentionally corrected in place; adding signed
-overloads would preserve an invalid negative-value domain and, because `Long` and
-`ULong` erase to the same JVM carrier, would complicate the public ABI. Java uses
-`JavaTokenActions`/`BigInteger`; JNI descriptors remain unchanged.
-
-| Kotlin operation | Corrected parameter(s) | Compatibility disposition |
-| --- | --- | --- |
-| `mint`, `burn`, `transfer` | token amount `Long` → `ULong` | unreleased source break; same raw `jlong` JNI ABI |
-| `setPrice` | price `Long` → `ULong` | unreleased source break; same raw `jlong` JNI ABI |
-| `purchase` | amount and expected cost `Long` → `ULong` | unreleased source break; same raw `jlong` JNI ABI |
-| `updateConfig` | optional max supply `Long?` → `ULong?` | unreleased source break; JSON/native encoding remains unsigned |
-| distribution/claim numeric values | protocol `u64` values → `ULong` | unreleased source break; Java checked adapter where exposed |
-
-## 5. Android persistence and restart parity
-
-### P1 — Orchard full-viewing-key persistence
-
-Implement Room storage and JNI callbacks for persist/load/free of shielded viewing
-keys, matching the existing Swift persistence contract.
-
-The exact shared preflight is `atomic_changesets + shielded_viewing_keys`. The
-`shielded_viewing_keys` bit is backend-attested and then intersected with the
-complete persist/load/free callback triplet; generic wallet-list `wallet_restore`
-is not part of this contract because seedless rebind loads FVK rows directly.
-
-- Key rows uniquely by `(walletId, accountIndex)`; wallet IDs are already
- network-specific, so do not add a redundant network key.
-- Store exactly 96 FVK bytes. A present malformed row fails closed instead of
- silently falling back to the mnemonic.
-- Implement callback allocation/free pairing, duplicate-account upsert, wallet purge,
- and wrong-wallet/network isolation.
-- Reserve Room schema v6 for this entity and export its schema JSON.
-
-**Acceptance:** create/bind shielded state, terminate, make the mnemonic unavailable,
-restart, and successfully rebind/sync from the stored viewing key. Unit/instrumented
-tests cover corrupt length, multiple wallets/accounts, callback allocation/free, and
-wallet deletion.
-
-### P2 — Invitation persistence and Android invitation UX
-
-The shared/iOS invitation protocol, durability ordering, and reclaim semantics are
-defined by `DIP15_INVITATIONS_SPEC.md`; this specification does not fork them.
-
-Implement Android:
-
-- Push-only invitation persistence callbacks and Room UI schema, including failure
- propagation. Rust does not rehydrate the invitation list; tracked asset-lock
- restore preserves reclaimability.
-- Create, parse/claim, sent-invitations, and reclaim SDK wrappers.
-- Deep-link/QR handling with the canonical legacy-compatible envelope.
-- Create, Claim, Sent Invitations, and Reclaim screens.
-- Test-plan cases `DP-12...DP-19`, including interrupted create/reclaim and
- already-consumed ambiguity.
-
-Creation must remain gated by Rust's exact invitation capability set:
-`atomic_changesets + asset_lock_funding_indices + invitations + wallet_restore`.
-The narrower `persists_durably()` compatibility wrapper is not sufficient for new
-feature code. A no-op callback is not an acceptable compatibility mode. A callback
-failure returns nonzero inside the changeset begin/end round and rolls it back.
-Persist `reclaimInFlight` transactionally before consume; write `Reclaimed`
-only after observed success and use the canonical conservative `Claimed` ambiguity
-classification. Room is v8 in the serialized migration chain. Purge by wallet and
-never log URI/WIF secrets.
-
-### P3 — Provider-special-transaction restoration
-
-Android already stores raw transaction bytes, but payload-only provider transactions
-create no TXOs. A TXO join therefore cannot reconstruct wallet/account ownership.
-
-**Required change**
-
-- Preserve existing C layouts: `TransactionRecordFFI` already carries
- `block_position`/`has_block_position`, and `AccountChangeSetFFI` already surrounds
- its transactions with the full typed account identity. Change only the JNI/Kotlin
- transaction callback parameters so each call explicitly forwards the enclosing
- account fields and the transaction's existing position fields. Do not add C POD
- fields or rely on mutable “current account” callback state.
-- Add nullable/default block-position columns and a transaction↔account involvement
- cross-reference. Reserve Room v7 for these additions and exported schema.
-- On cold load, select provider transaction kinds 2...5 through wallet/account
- involvement, marshal raw consensus bytes plus context/block metadata into the
- existing `ProviderSpecialTxRestoreEntryFFI`, and keep every backing byte buffer
- alive until the restore release callback.
-- Rust re-decodes the payload and rebuilds ownership/masternode grouping; decoded
- host columns are not restore authority.
-
-**Acceptance:** a payload-only provider transaction belonging to wallet A restores
-only to A, survives without a TXO, preserves same-block ordering and optional block
-position, and malformed raw data is diagnosed/skipped without crashing.
-
-### P4 — Deferred contact-crypto queue
-
-Do not add write-only host persistence. Implement restore in the shared wallet
-start-state path first, then add the callback contract and both Room and SwiftData
-stores in the same slice. After the relocation described by
-`PENDING_CONTACT_CRYPTO_RELOCATION_SPEC.md`, restore hydrates identities first and
-then fans rows into the owning `ManagedIdentity` across both identity buckets.
-Unknown-owner rows are retained/quarantined or explicitly diagnosed, never silently
-dropped. Define POD ownership/free semantics, idempotent operation identity, clear
-tombstones, wallet scoping, and corrupt-row behavior. Until then, document that the
-recurring sweep re-enqueues work and parity is delayed rather than durable.
-
-`PersistenceCallbacks` currently has no struct-size negotiation. New callback slots
-must use a versioned `PersistenceCallbacksV2`/constructor unless the release explicitly
-declares framework and wrapper lockstep source ABI. In either case add cbindgen header,
-C layout, and Swift `MemoryLayout` pins; never silently insert fields into the current
-layout.
-
-## 6. Recovery parity
-
-### R1 — Existing asset-lock resume on Android
-
-Generic Platform-address (`ADDR-03`) and shielded resume are already implemented and
-remain unchanged. Bridge only the missing identity operations:
-
-- `platform_wallet_resume_identity_with_existing_asset_lock_signer` for registration;
-- `platform_wallet_topup_identity_with_existing_asset_lock_signer` for top-up.
-
-Also bridge tracked-lock enumeration/status (with paired array free) so UI rows are
-not reconstructed from private Room assumptions. Eligible generic rows have funding
-type registration (`0`) or top-up (`1`/`2`) and a resumable status from Built through
-ChainLocked (`0...3`). Funding type invitation (`3`) is never offered by generic
-recovery. The shared wallet retains a consumed lock as a terminal tombstone so an
-exact-outpoint retry remains a typed already-consumed error in the same process and
-after restoration; actionable recovery lists exclude status `4`.
-
-The registration result's managed-identity handle must be adopted immediately and
-freed on every post-call failure. Both operations borrow the mnemonic resolver under
-the manager teardown gate. The Android UI uses the same restored outpoint and never
-creates a second funding transaction. Generic paths always pass
-`consumeInvitationVoucher=false`; only P2 reclaim may pass `true`.
-
-**Acceptance:** separate registration-resume and top-up-resume coverage (`ID-16`
-covers top-up), interruption immediately after Core broadcast and before Platform
-submission, and typed handling of untracked, foreign, and already-consumed locks. A
-`Built` row re-broadcasts the same transaction and never creates a second distinct
-funding transaction. A type-3 row is rejected by generic resume.
-
-### R2 — Compact-filter rescan on Android
-
-Expose the shared SPV rescan operation through JNI/Kotlin and add the height picker
-and rescan state to the Android sync screen. The call rewinds an in-memory compact
-filter checkpoint; it does not itself scan. A running SPV manager acts on the next
-tick, a stopped manager acts on next start, equal/forward requests are harmless
-no-ops, and unknown wallets return typed errors. Per-wallet failures are collected.
-The rewind is not durable: process death before the filter loop consumes and persists
-progress loses the rescan request, so the host/user must reissue it. Correct the
-misleading shared Rust documentation when R2 is implemented. Do not promise
-cancellability or durable rewind without adding a durable rescan-intent contract.
-
-### R3 — Contested usernames by identity
-
-Bridge both existing shared operations:
-
-- `platform_wallet_sync_contested_dpns_names`, which performs one network fetch and
- full-snapshot persistence so resolved contests disappear;
-- `managed_identity_get_contested_dpns_names`, which returns the cached array, with
- its paired free function.
-
-Alternatively add one Rust composite returning an owned `DpnsNameArray`. Replace
-Android's bounded local-label probing with this path.
-
-**Acceptance:** an identity with more than eight locally unknown contested names is
-shown completely with one logical query.
-
-## 7. Shared-policy consolidation
-
-### S1 — Masternode discovery
-
-Discovery currently bootstraps DAPI before an SDK handle exists. Add a standalone
-shared Rust entry point taking `(network, quorum_base)` and returning owned typed
-records containing both Core peer address and DAPI URL, with an explicit free
-function; alternatively move discovery into Rust SDK construction and expose its
-cached result. Extract a pure endpoint parser in
-`rs-sdk-trusted-context-provider` so the provider and bridge share one parser.
-
-Define explicit-configuration precedence, HTTP status/timeout/failure fallback,
-version/status filtering, string ownership, testnet's missing-port default (`1443`),
-bracketed IPv6, and malformed/missing port behavior. Delete Kotlin and Swift host
-fetch/parsing only after both consume the shared result; hosts must not fetch twice.
-Replace the Kotlin test that currently pins the incorrect `443` default.
-
-### S2 — Funding selection and account scope
-
-This concerns DIP-17 Platform credit addresses and nonces, not Core UTXOs from C2.
-Add separate account-scoped Rust composites for identity registration/top-up from
-Platform addresses. Inputs are `(wallet_id, PlatformPayment account_index, target
-credits)`. Rust enumerates hydrated and derived candidates, fetches authoritative
-balances/nonces, applies deterministic selection and fee constraints, signs/submits,
-and returns the result.
-
-If multiple Platform Payment accounts exist, the account index is mandatory. Tests
-cover only-the-chosen-account, fresh-restart hydrated candidates, exact target,
-insufficient funds, and concurrent balance/nonce revalidation. Delete every
-Kotlin/Swift pre-enumeration and greedy packing loop in the adopting slice.
-
-### S3a — Token authorization and proposal evaluation
-
-After C3, expose a versioned Rust decision result for action + token configuration +
-actor context containing allowed/denied, a stable reason discriminant, and zero or
-more authorization alternatives/group rules. Include every group-capable action,
-especially `maxSupplyChangeRules`; do not collapse alternatives into one “required
-key.” Define stable `repr` discriminants or versioned JSON plus typed host decoding
-and owned-array/free rules.
-
-### S3b — Direct-purchase quote
-
-Expose a separate Rust quote result for schedule + amount containing selected
-threshold, unit price, and full-domain `u64` total. Both apps render this result and
-delete their local tier/price arithmetic. Rust remains authoritative at broadcast.
-
-### S4 — Protocol constants and codecs
-
-Expose versioned consensus identity-funding denomination sets separately from purely
-presentational UI presets. Address validation returns typed family, network, payload,
-and failure reason rather than `Bool`. Invitation validation already exists in
-`platform_wallet_parse_invitation`; Android bridges that API rather than creating a
-second codec. Remove hard-coded protocol tables and app-local Base58/Bech32 validators
-only when the shared replacements are adopted.
-
-## 8. Immediate example-app parity fixes
-
-These are independent, low-risk fixes and do not wait for the larger shared APIs:
-
-1. Add `DedicatedTransition.CREATE_DOCUMENT`, route Android `documentCreate` to the
- existing `CreateDocument` screen using selected contract/type, and remove the
- unused `documentFields` catalog input unless it is passed as a real prefill.
-2. Include `maxSupplyChangeRules` in Kotlin and Swift pending-proposal discovery
- as an explicitly temporary compatibility fix; delete it when S3a replaces host
- discovery entirely.
-3. Display immature Core balance in Swift WalletDetail and do not label a wallet
- containing only immature funds “Empty Wallet.”
-4. Correct parity rows for document transitions, invitations, rescan, recovery, and
- contested-name discovery.
-
-## 9. Executable parity manifest
-
-Replace manually maintained totals with a checked-in manifest. Each capability has:
-
-```yaml
-id: invitations.reclaim
-shared_apis:
- - platform_wallet_topup_identity_with_existing_asset_lock_signer
- - platform_wallet_resume_identity_with_existing_asset_lock_signer
-required_persistence_capabilities:
- - atomic_changesets
- - asset_lock_funding_indices
- - invitations
- - wallet_restore
-hosts:
- swift:
- sdk: supported
- example_app: supported
- restart: tested
- reason: null
- kotlin:
- sdk: unsupported
- example_app: unsupported
- restart: required
- reason: "P2 not implemented at the PR #3999 baseline"
-verification:
- - host: swift
- kind: manual
- file: packages/swift-sdk/SwiftExampleApp/TEST_PLAN.md
- id: DP-19
-```
-
-Allowed host states are `supported`, `partial`, `unsupported`, and
-`not-applicable`. A capability can be `supported` only when:
-
-- all listed shared APIs are reachable, when the capability needs shared APIs;
-- required persistence capabilities are registered;
-- restart behavior is tested when value or durable state can exist;
-- its automated verification exists and passes, or the manifest records a release
- manual/device gate.
-
-`shared_apis` is optional because some capabilities are host-only. Restart state is
-`required`, `tested`, or `not_applicable`, not a boolean. Verification kinds are
-`unit`, `integration`, `device`, or `manual` and include a file, stable ID/test name,
-and command where automated. CI validates schema, symbols/files/test IDs, reason
-requirements, and generated counts; it does not pretend to prove runtime reachability
-by static assertion. `PARITY.md` becomes generated prose or a thin index.
-
-## 10. Implementation sequence
-
-### Slice 0 — Spec and regression harness
-
-- Land this reviewed spec.
-- Add the parity-manifest schema/checker and initial manifest representing reality.
-- Correct stale documentation without claiming missing features are complete,
- including obsolete Swift `core_wallet_send_to_addresses` test-plan references and
- shipped Swift invitation rows still marked as bridge-only.
-
-### Slice 1a — Android address-index safety
-
-- C1 only. No schema migration.
-
-### Slice 1b — Atomic shared Core send
-
-- C2 Rust/key-wallet reservation primitive, new opaque FFI result, both host
- adoptions, and old-ABI deprecation.
-
-### Slice 1c — Lossless token domains
-
-- C3 compatibility table, JNI raw-bit boundary, v5 unsigned token storage, and both
- host/domain tests. C3 gates S3a/S3b.
-
-These are separate PRs/rollback units.
-
-### Slice 2a — Serialized Android schema foundation
-
-- Land/export v5 for C3 unsigned token storage.
-- Reserve and land Room v6 for P1 FVK storage.
-- Reserve and land Room v7 for P3 block position and transaction-account
- involvement.
-- Reserve Room v8 for P2 invitation UI persistence.
-- Export each schema; test each adjacent migration and v4→latest. Do not develop
- parallel conflicting migrations from the same schema version.
-- Define a versioned, explicit backend-attested `PersistenceCapability` bitset and
- intersect/validate it against structurally required callback groups. Callback
- presence alone cannot attest semantic sub-capabilities: for example,
- `provider_transactions` shares the broad wallet-list restore callback but is valid
- only when the backend actually populates and frees its provider restore payload.
- Canonical v1 capabilities are `atomic_changesets`,
- `asset_lock_funding_indices`, `invitations`, `shielded_viewing_keys`,
- `provider_transactions`, `unsigned_token_storage`, `pending_contact_crypto`, and
- `wallet_restore`. `asset_lock_funding_indices` covers account registration and
- address-pool watermark persistence; `pending_contact_crypto` covers both durable
- queue additions and removals. These names are the public manifest/diagnostic
- namespace; source-level compatibility aliases do not create additional bits.
- Expose initialization diagnostics and add missing-capability preflight tests per
- feature, including a wallet-list callback present while provider capability
- remains absent.
-- Retain `persists_durably()` only as a compatibility wrapper derived from
- `atomic_changesets + asset_lock_funding_indices + invitations`; new feature code
- checks its exact required capability set. Invitation creation additionally
- requires `wallet_restore`, because a committed voucher is not restart-safe unless
- its originating wallet and funding-index state can both be reconstructed.
-
-### Slice 2b — Android viewing-key callbacks
-
-- P1 behavior atop v6, including native instrumentation round trip.
-
-### Slice 2c — Android provider restoration
-
-- P3 behavior atop v7. This may remain `unsupported` in the manifest until an
- Android masternode consumer exists, but must not be claimed as parity.
-
-### Slice 2d — Android identity recovery
-
-- R1 tracked-lock listing plus registration/top-up resume. No DB migration.
-
-### Slice 2e — Android SPV and DPNS queries
-
-- R2 and R3 as independent commits/PRs. No DB migration.
-
-Each vertical slice includes its JNI descriptor/symbol smoke and native Android
-library build; JVM tests alone do not validate native binding.
-
-### Slice 3 — Invitations
-
-- P2 as one vertical feature using v8 and the shipped shared/iOS invitation
- semantics. R1 lands before invitation reclaim.
-- Do not combine it with generic persistence cleanup; invitation broadcast ordering
- and durability gates must remain independently reviewable.
-
-### Slice 4 — Shared-policy consolidation
-
-- S1 endpoint discovery.
-- S2 account-scoped funding selection.
-- S3a proposal/authorization decisions.
-- S3b purchase quote.
-- S4 constants/codecs.
-
-Each host implementation is deleted in the same slice that exposes and adopts its
-shared replacement; do not leave two live paths.
-
-### Slice 5 — Deferred durability
-
-- P4 only after the per-identity queue relocation and shared cold-load restore exist.
-
-### Release gates
-
-PR #3999 release blockers are Slice 0, C1, C2, C3, P1, P3 if provider parity is
-advertised, R1, and truthful manifest/docs. Invitation Android parity, R2/R3, and
-S1-S4 may ship as explicitly `unsupported`/`partial` follow-ups unless product scope
-requires them for the same release. Definition of done for the overall program does
-not force every consolidation item into one unbounded merge gate.
-
-## 11. Test matrix
-
-| Risk | Shared Rust | JNI/Kotlin | Swift | Device/testnet |
-| --- | --- | --- | --- | --- |
-| Address index conflict | provider event fixture | handler restart/restore | existing semantic pin | Android restore smoke |
-| Concurrent Core sends | barrier-forced same-UTXO race and reservation lifecycle | composite wrapper + JNI ownership | composite wrapper + C ownership | two live sends |
-| `u64` boundary | ABI encode/decode | raw-bit JNI + Room/DAO/UI unsigned ordering | `UInt64` parity | Android JNI symbol smoke |
-| Viewing-key restart | bind without seed | v5→v6 + callback/free round trip | existing callback test | Android seedless restart |
-| Provider restore | payload decode/ownership | v6→v7, multiwallet membership, malformed bytes | existing restore | Android callback round trip |
-| Asset-lock resume | tracked-lock state machine | process-death recovery | existing resume tests | funded testnet |
-| Invitations | protocol/persistence ordering | v7→v8 + DP-12...19 | retain DP-12...19 | cross-platform claim |
-| Discovery | port/IPv6 fixtures | no host parser remains | no host parser remains | testnet discovery |
-| Token proposal rules | all action-rule variants | render typed result | render typed result | group co-sign |
-| Deferred crypto | identity-first restore/fan-out | vtable + Room crash restart | vtable + SwiftData crash restart | locked-seed restart |
-
-Required validation per slice:
-
-- `cargo fmt --all -- --check` and targeted Rust tests;
-- `cargo clippy` for changed Rust crates and all targets;
-- `cargo test -p rs-unified-sdk-jni --lib`, Android native library build, Kotlin JVM
- tests under JDK 17, Room `MigrationTestHelper` adjacent and v4→latest tests, and
- instrumented callback round trips for P1/P3;
-- Swift package tests and iOS framework build for Swift/FFI slices;
-- cbindgen header regeneration/diff, C struct size/layout pins, Swift `MemoryLayout`
- pins for direct structs, and null/zero-count/double-free buffer tests;
-- on-device external-function smokes for R1/R2/R3/C3 and resolver-handle teardown;
-- `git diff --check`.
-
-## 12. Compatibility and rollout
-
-- Database migrations are additive and serialized as v5 unsigned token storage, v6
- FVK, v7 provider membership/position, and v8 identity-key derivation
- breadcrumbs (pending-repair durability, dashpay/platform#4060); the
- invitations migration shifts to v9. Do not destructively rewrite address
- identity columns.
-- New Rust FFI functions are additive and new result PODs are opaque/versioned.
- Existing released split-builder entry points are deprecated before removal;
- existing struct layouts and JNI descriptors do not change silently. The
- unreleased Kotlin token source surface is corrected from signed `Long` to
- `ULong` in place as accounted in C3, while retaining identical raw `jlong`
- descriptors and a checked Java `BigInteger` adapter.
-- Persistence exposes feature-specific capabilities. Registration reports missing
- callback sets at initialization where possible, and required feature APIs fail
- closed before broadcast.
-- The parity manifest initially records known gaps. CI prevents regression but does
- not require all gaps to close in the first slice.
-
-### Keystore rework divergences and convergences (dashpay/platform#4060)
-
-Recorded in `sdk-parity-manifest.json`; rationale here:
-
-- **`KeySecurityPolicy` + Keystore alias split (Kotlin-only, deliberate).**
- Android Keystore fixes authentication parameters at key generation, so the
- AUTH_GATED/DEVICE_BOUND policies require distinct aliases; iOS Keychain has
- no per-alias auth-parameter analog (item access control covers the same
- ground), so the manifest marks Swift `not-applicable` — no Swift port is
- planned. The lockless-device degradation (AUTH_GATED writes redirect to the
- DEVICE_BOUND alias, surfaced via `effectiveKeySecurityPolicy`) is likewise
- Android-specific: KeyMint rejects gated key generation without a secure
- lock screen.
-- **Platform-wallet code 98 is now CONVERGENT.** Kotlin previously collapsed
- the blanket Option-miss code into the top-level `DashSdkError.NotFound`
- while Swift kept it in the wallet family; Kotlin now maps 98 to
- `DashSdkError.PlatformWallet.NotFound` (BREAKING for hosts that caught the
- top-level type from platform-wallet operations).
-- **Durable pending-repair surface (Kotlin-only, port candidate).**
- `pendingIdentityKeys` + forced/verified `repairIdentityKey` + the Room v8
- derivation breadcrumbs have no Swift counterpart; the manifest records the
- gap as `unsupported` for Swift.
-- **Structured `SigningKeyUnavailable` discriminator (both hosts).** The
- signer completion carries a typed `error_code` (rs-sdk-ffi
- `DashSDKSignerErrorCode`), restored as platform-wallet code 31 on both
- hosts. The Rust-internal segment rides the machine prefix
- `signer_error:key_unavailable: ` through `ProtocolError::Generic` (a typed
- rs-dpp variant was rejected for serialization blast radius — accepted
- residual). The Kotlin `MESSAGE_MARKER` text sniff survives ONLY as a
- deprecated fallback for the #4191 merge-order transition (marker-based
- classification predating the typed code) and for conversion paths that
- lose the machine prefix; mixed old-native/new-Kotlin artifacts are
- unsupported outright (the sign-completion JNI arity changed 3→4 args).
- Remove it (and the marker's matcher role) in the next minor release.
-
-## 13. Explicitly out of scope
-
-- Redesigning DIP-13/DIP-15 invitation wire formats.
-- Simultaneous multi-account DashPay contacts; that remains governed by
- `MULTI_ACCOUNT_SPEC.md` and its product gate.
-- Replacing JNI or C FFI with a new binding generator.
-- Making Room and SwiftData schemas structurally identical.
-- Treating example-app visual layout differences as parity failures when capability,
- accessibility contract, and behavior are equivalent.
-
-## 14. Definition of done
-
-This effort is complete when:
-
-1. C1-C3 have regression coverage and both host SDKs use the safe/shared paths.
-2. Android registers all persistence capabilities required by features it advertises.
-3. Android can resume every funded identity/invitation operation already supported
- by iOS.
-4. S1-S4 have one live shared implementation with both host copies removed.
-5. Every supported capability is represented by the executable manifest and named
- tests; no manual parity count contradicts runtime capability.
-6. Cross-platform invitation claim and concurrent-send device smoke tests pass.
diff --git a/docs/sdk/PR3999_WALLET_LIFECYCLE_HARDENING_SPEC.md b/docs/sdk/PR3999_WALLET_LIFECYCLE_HARDENING_SPEC.md
deleted file mode 100644
index 2a9ec4decce..00000000000
--- a/docs/sdk/PR3999_WALLET_LIFECYCLE_HARDENING_SPEC.md
+++ /dev/null
@@ -1,652 +0,0 @@
-# PR #3999 — Kotlin SDK wallet-lifecycle hardening spec
-
-Covers the 4 open blocking review findings on PR #3999
-(`feat/kotlin-sdk-and-example-app`) that were not yet addressed:
-
-1. `PlatformWalletManager.kt:347` — init cleanup starts after fallible
- child constructors.
-2. `PlatformWalletManager.kt:778` — alias cleanup ignores ownership
- recorded only in another wallet's index.
-3. `PlatformWalletManager.kt:783` — identity-key persistence can still
- run after wallet deletion.
-4. `CreateWalletScreen.kt:85` — configuration changes discard the only
- recovery-phrase copy.
-
-Findings 2 and 3 both live in the `removeWallet` / `WalletStorage`
-private-key lifecycle and are designed together (§2). 1 and 4 are
-independent (§1, §3).
-
-## Spec review findings (applied)
-
-Three independent review agents (feasibility, security, scope/simplicity)
-checked a first draft of this spec against the actual source. §1 and
-the overall scope of §2/§3 came back clean; three must-fix issues were
-found and are already folded into the sections below:
-
-1. **Compile-breaking (feasibility):** the original §2 sketch declared
- `isOwnedByAnotherWallet` as a `private` extension function on
- `PrivateKeyExclusion`. It's unreachable from `removeWallet`'s
- `withPrivateKeyExclusion { }` lambda (only the interface's own
- members resolve there) and `private` besides. Fixed: declared on
- the `PrivateKeyExclusion` interface itself, like the existing
- `deleteOwnerIndex`.
-2. **Deadlock-risk (feasibility + security):** the original §2
- `storeIfAbsent` sketch called `derive()` (a Rust FFI call) *inside*
- `privateKeyMutex.withLock`, directly violating `WalletStorage.kt`'s
- own documented invariant that the locked block must never call into
- native code. Fixed: two-phase check → derive-outside-lock →
- re-check-and-store. The security pass also caught that the
- existence check must treat a present-but-undecryptable ciphertext
- blob (a real, already-supported legacy state — see
- `isPrivateKeyDecryptable`) as absent, or the fix would silently
- defeat that re-derive path.
-3. **New risk introduced (security), inconsistent with the rest of the
- screen (feasibility):** the original §3 sketch also moved the
- wallet-creation coroutine's launch from `rememberCoroutineScope()`
- onto `viewModel.viewModelScope`. Feasibility flagged that the
- screen's other captured state (`isCreating`, `error`,
- `navController`) stays composable-scoped, so a `viewModelScope`
- coroutine surviving a config change would mutate/navigate through
- dead references. Security separately flagged that `viewModelScope`
- surviving longer than the composition (e.g. past navigation-away)
- combined with `onCleared()` scrubbing the phrase could wipe the
- *only* copy before the emergency dialog is ever shown, for a
- creation that fails after the user has already left the screen — a
- worse outcome than the bug being fixed. Fixed: keep
- `rememberCoroutineScope()` for the launch; only the *storage* of an
- already-caught phrase moves to the ViewModel.
-
-Additionally, the security pass found one HIGH-severity gap the
-feasibility/scope passes didn't have the angle to catch: the
-tombstone-set design ("a deleted wallet's id is never reused") is
-false — re-importing the same recovery phrase after an accidental
-delete is a real, supported flow, and would permanently brick under
-the original always-on tombstone. Fixed: `createWallet` (the same
-entry point both "new" and "restore from phrase" go through) now
-clears any stale tombstone for that wallet id before storing.
-
-Two lower-severity items were surfaced and are intentionally *not*
-fixed in this pass (see the "Accepted, not fixed" note in §2 and the
-platform-limitation note in §3) — both are pre-existing or
-theoretical, not regressions this spec would introduce, and closing
-them would widen the diff for gaps that either have no live caller
-today or are an accepted platform ceiling.
-
----
-
-## §1. Init cleanup ordering (finding @ `PlatformWalletManager.kt:347`)
-
-### Problem
-
-`PlatformWalletManager`'s primary constructor initializes, in source
-order: `scope` (192) → `teardownGate`/`_syncEvents`/`eventBridge`
-(203-317) → `mnemonicResolver` (325) → `signer` (326-327) →
-`identityKeyDeriver` (337-341) → `persistenceHandler` (343-347) → …
-→ `nativeInitialization = initializePlatformWalletNativeManager(...)`
-(477-490).
-
-`initializePlatformWalletNativeManager` already contains a correct
-cleanup transaction — on failure it cancels `scope` and closes
-`mnemonicResolver`/`signer`/`persistenceHandler` (in that order,
-suppressing secondary failures) before rethrowing. But it only guards
-failures inside *itself* (native bundle create, manager-handle fetch,
-capability reads). It cannot guard `mnemonicResolver`, `signer`, or
-`persistenceHandler`'s own constructors, because those already ran to
-completion (or threw) *before* `nativeInitialization` is reached — a
-throw there aborts the whole primary constructor with no
-`PlatformWalletManager` instance to call `close()` on, so:
-
-- `mnemonicResolver`'s JNI handle (`MnemonicNative.createResolver`)
- leaks if `signer`, `identityKeyDeriver`, or `persistenceHandler`
- throws after it.
-- `signer`'s JNI handle (`SignerNative.createSigner`) leaks if
- `identityKeyDeriver` or `persistenceHandler` throws after it (and
- `database.platformAddressDao()`, evaluated as `signer`'s 4th
- constructor arg, can itself throw before `signer`'s own constructor
- even runs).
-- `persistenceHandler`'s owned single-thread `Executor`
- (`Executors.newSingleThreadExecutor { "dash-persistence" }`) leaks
- if it's the one that throws, or leaks the executor while everything
- *before* it (resolver, signer) also leaks.
-
-`identityKeyDeriver` itself holds no releasable resource (confirmed:
-plain Kotlin object, no native handle, not `AutoCloseable`).
-
-### Chosen approach
-
-Group the four child constructions into one `init`-time block that
-builds each as a local `var`, wraps construction in `try`, and on any
-`Throwable` closes whichever locals were already assigned (in reverse
-construction order) before rethrowing — the same "roll back the
-locals, only adopt on success" shape the Swift equivalent
-(`PlatformWalletManager.swift`'s `configure(...)`) already uses,
-adapted to Kotlin's val-in-constructor idiom via a small holder:
-
-```kotlin
-private class CoreChildren(
- val mnemonicResolver: MnemonicResolverAndPersister,
- val signer: KeystoreSigner,
- val identityKeyDeriver: IdentityKeyPrivateKeyDeriver,
- val persistenceHandler: PlatformWalletPersistenceHandler,
-)
-
-private val coreChildren: CoreChildren = run {
- var mnemonicResolver: MnemonicResolverAndPersister? = null
- var signer: KeystoreSigner? = null
- try {
- val resolver = MnemonicResolverAndPersister(walletStorage)
- .also { mnemonicResolver = it }
- val keySigner = KeystoreSigner(
- walletStorage, network, biometricGate, database.platformAddressDao(),
- ).also { signer = it }
- val deriver = IdentityKeyPrivateKeyDeriver(
- network = network,
- mnemonicResolverHandle = resolver.nativeHandle,
- walletStorage = walletStorage,
- )
- val handler = PlatformWalletPersistenceHandler(
- database = database, privateKeyDeriver = deriver, network = network,
- )
- CoreChildren(resolver, keySigner, deriver, handler)
- } catch (e: Throwable) {
- runCatching { signer?.close() }
- runCatching { mnemonicResolver?.close() }
- scope.cancel()
- throw e
- }
-}
-private val mnemonicResolver get() = coreChildren.mnemonicResolver
-private val signer get() = coreChildren.signer
-private val identityKeyDeriver get() = coreChildren.identityKeyDeriver
-private val persistenceHandler get() = coreChildren.persistenceHandler
-```
-
-- `persistenceHandler` needs no explicit close-on-catch: if its own
- constructor throws, it never assigned itself anywhere, and it's the
- last child built, so there's nothing after it to fail and orphan it.
-- `scope.cancel()` on the failure path mirrors what
- `initializePlatformWalletNativeManager`'s own cleanup already does
- for *its* failures; nothing has been launched on `scope` yet at this
- point (it's created earlier, at line 192, unused until later), so
- cancelling it here is cheap and keeps both failure paths consistent.
-- All downstream references to `mnemonicResolver`, `signer`,
- `identityKeyDeriver`, `persistenceHandler` throughout the file are
- unaffected — they become `private val ... get() = ...` delegating
- properties instead of directly-initialized `private val`s, same
- read-only surface, no call-site changes.
-- `nativeInitialization`'s own cleanup transaction (109-128) is
- untouched — it still guards its own failure window exactly as
- today.
-
-### Alternatives rejected
-
-- **Wrap the whole primary constructor body in try/catch.** Kotlin
- doesn't allow arbitrary try/catch around property initializers
- mixed with the constructor parameter list in a readable way, and it
- would force converting *every* property after this point (including
- `identityRegistration`, `voteCasting`, etc., none of which are
- fallible or hold resources) into part of the guarded region for no
- benefit — larger surface than necessary.
-- **`private constructor` + `companion object { fun create(...) }`
- factory** (the pattern the finding suggests by analogy to Swift).
- Rejected for this specific class: every call site that currently
- does `PlatformWalletManager(...)` (`WalletManagerStore`'s factory
- lambda, tests) would need to change to `PlatformWalletManager.create(...)`,
- and the class already exposes a large public API assuming direct
- construction succeeds or throws synchronously — converting to an
- external factory is a bigger surface change for the same outcome as
- the local-var-holder approach above, which achieves the identical
- rollback guarantee without touching any call site.
-
-### Failure modes covered / not covered
-
-- Covered: any single child's constructor throwing, at any position in
- the four, cleans up every JNI-owning child constructed strictly
- before it.
-- Not covered (explicitly out of scope for this finding, flagged for
- a separate follow-up): `KeystoreSigner.close()` itself does not
- cancel `KeystoreSigner`'s own internal `scope`
- (`CoroutineScope(SupervisorJob() + Dispatchers.IO)`, `KeystoreSigner.kt:53`)
- — that's a pre-existing gap in `KeystoreSigner`'s own `close()`
- logic, orthogonal to *when* cleanup runs (which is what this finding
- is about). Noting it here so it isn't lost, not fixing it in this
- pass to keep the diff surgical.
-
-### Test plan
-
-Add to `PlatformWalletManagerInitializationTest.kt` (or a new
-`PlatformWalletManagerConstructionTest.kt` if constructing a real
-`PlatformWalletManager` needs more fixture setup than that file
-currently has): a test that injects a `walletStorage`/`database`/etc.
-combination where `KeystoreSigner`'s construction throws (e.g. via a
-fake `platformAddressDao()` or a `SignerNative.createSigner` stub that
-throws — check what's fake-able without real JNI, per that file's
-existing lambda-injection pattern for `nativeCreate`/`nativeManagerHandle`/etc.,
-since `PlatformWalletManagerInitializationTest` already fakes native
-calls at that granularity). Assert:
-- The thrown exception propagates (construction still fails).
-- `mnemonicResolver`'s `close()` was invoked (spy/count).
-- No leaked JNI handle assertion is directly measurable from a JVM
- unit test without a real native lib loaded; the practical proof is
- "close() was called on every child constructed before the failing
- one," which is what the test asserts.
-
----
-
-## §2. Cross-wallet private-key ownership (findings @ `:778`, `:783`)
-
-### Shared root cause
-
-`WalletStorage` (`sdk/src/main/kotlin/.../security/WalletStorage.kt`)
-stores private-key ciphertext **globally**, keyed only by pubkey hex
-(`privkey.`, no wallet-id component), because sibling network
-wallets derived from one mnemonic (Testnet/Devnet/Regtest all share
-DIP-9's non-mainnet derivation path) can legitimately derive the same
-pubkey and are expected to share that one ciphertext entry. Ownership
-is tracked **per-wallet** via a separate index (`privkeyowners.`
-→ `Set`), and all wallets share one process-wide
-`WalletStorage` instance and one `privateKeyMutex`.
-
-Two related gaps fall out of that design, both inside `removeWallet`
-(`PlatformWalletManager.kt:718-791`):
-
-**(a) finding @ `:778`.** `aliasesToDelete` (764-770) decides an alias
-is safe to delete when no *other Room `public_keys` row* (a committed,
-on-chain-registered key) references it outside this wallet's
-identities. It never checks whether a *sibling wallet's durable owner
-index* (`privkeyowners.`) already claims the alias —
-so a sibling wallet that pre-stored (but hasn't yet committed a
-`public_keys` row for) the same shared alias loses its ciphertext when
-this wallet is deleted.
-
-**(b) finding @ `:783`.** Two independent gaps, not one:
-- `PlatformWalletPersistenceHandler`'s own persist-callback path
- (`onPersistIdentityKeyUpsert` → `IdentityKeyPrivateKeyDeriver.hasStored`
- then `.deriveAndStore` → `WalletStorage.storePrivateKey`) *is*
- exclusion-fenced (`withCallbackExclusion`), but `hasStored` and the
- eventual `storePrivateKey` inside `deriveAndStore` are two separate
- calls with no single lock spanning both — a sibling caller can store
- between the check and the write.
-- Two **app-level** call sites — `CreateIdentityScreen.kt:219-227`
- and `IdentityKeyAdditionFlow.kt:161-165` — call
- `walletStorage.storePrivateKey(...)` directly from their own
- `scope.launch`/coroutine, entirely outside
- `withPrivateKeyExclusion`/`withCallbackExclusion`/`teardownGate`.
- Neither `WalletStorage` nor `storePrivateKey` has any concept of
- "this wallet was just deleted," so a store that was already
- in-flight when `removeWallet` ran completes anyway, resurrecting the
- just-deleted wallet's owner-index entry with fresh ciphertext.
-
-### Chosen approach
-
-*(Revised after multi-agent spec review — see "Spec review findings"
-below for what changed and why.)*
-
-Three additions to `WalletStorage`, all executed under the existing
-`privateKeyMutex` (via `withPrivateKeyExclusion`/its internal scope),
-so no new lock is introduced:
-
-**1. Cross-wallet ownership query**, used to fix `:778`. Added to the
-`PrivateKeyExclusion` **interface** (not a private extension function
-— extension functions can't see `WalletStorage`'s private members and
-aren't reachable from inside a `withPrivateKeyExclusion { }` lambda,
-where only the interface's own receiver is in scope), implemented in
-`privateKeyExclusionScope` alongside the existing `deletePrivateKeys`/
-`deleteOwnerIndex`:
-
-```kotlin
-interface PrivateKeyExclusion {
- suspend fun deletePrivateKeys(pubkeyHexes: Collection)
- suspend fun deleteOwnerIndex(walletId: ByteArray)
-
- /** True if any wallet OTHER than [excludingWalletId] still claims
- * [pubkeyHex] in its durable owner index. */
- suspend fun isOwnedByAnotherWallet(pubkeyHex: String, excludingWalletId: ByteArray): Boolean
-}
-
-private val privateKeyExclusionScope = object : PrivateKeyExclusion {
- // ...existing overrides...
- override suspend fun isOwnedByAnotherWallet(
- pubkeyHex: String,
- excludingWalletId: ByteArray,
- ): Boolean {
- val excludingHex = excludingWalletId.toHex()
- val prefs = store.data.first()
- return prefs.asMap().any { (key, value) ->
- key.name.startsWith(PRIVKEY_OWNERS_PREFIX) &&
- key.name.removePrefix(PRIVKEY_OWNERS_PREFIX) != excludingHex &&
- (value as? Set<*>)?.contains(pubkeyHex.lowercase()) == true
- }
- }
-}
-```
-
-`removeWallet`'s `aliasesToDelete` computation (`PlatformWalletManager.kt:764-770`)
-becomes:
-
-```kotlin
-val aliasesToDelete = buildList {
- for ((pubkeyHex, publicKeyData) in keysByPubkeyHex) {
- val referencedElsewhere = database.publicKeyDao()
- .countReferencesOutsideIdentities(publicKeyData, ownedIdentityIds) > 0
- val ownedElsewhere = isOwnedByAnotherWallet(pubkeyHex, walletId)
- if (!referencedElsewhere && !ownedElsewhere) add(pubkeyHex)
- }
-}
-```
-
-Already runs inside `walletStorage.withPrivateKeyExclusion { ... }`
-(736) — as an interface method, `isOwnedByAnotherWallet` resolves on
-that lambda's `PrivateKeyExclusion` receiver directly, same as
-`deleteOwnerIndex` does today, keeping it under the same lock as the
-delete that follows — no TOCTOU between the check and
-`deletePrivateKeys`/`deleteOwnerIndex`.
-
-**2. Atomic-enough check-and-store**, used to fix the `hasStored`/
-`deriveAndStore` half of `:783` (and the same race underlies `:778`'s
-"rollback path" note). **Not a single lock-held call** — `WalletStorage.kt`'s
-own documented invariant on `withPrivateKeyExclusion` is explicit: *"must
-also never call into native code (a persistence callback parked on this
-lock can be holding native locks)."* `derive()` is a Rust FFI call
-(`IdentityNative.deriveIdentityPrivateKeyWithResolver`), so it cannot run
-inside `privateKeyMutex.withLock`. Instead, a double-checked pattern —
-lock only for the existence check/write, derive in between, re-check
-before writing:
-
-```kotlin
-/** If [pubkeyHex] has no *usable* stored ciphertext (absent, or present
- * but undecryptable — see [isPrivateKeyDecryptable]), derive it via
- * [derive] and store it; either way record [ownerWalletId] in the
- * owner index. Returns whether a derive+store actually happened. */
-suspend fun storeIfAbsent(
- pubkeyHex: String,
- ownerWalletId: ByteArray,
- derive: suspend () -> ByteArray,
-): Boolean {
- // Fast path: already usable under any owner — just record ownership.
- if (privateKeyMutex.withLock { addOwnerIfUsableLocked(pubkeyHex, ownerWalletId) }) {
- return false
- }
- // Derive OUTSIDE the lock (native call) — another writer may store
- // the same alias while this runs.
- val derived = derive()
- return privateKeyMutex.withLock {
- if (addOwnerIfUsableLocked(pubkeyHex, ownerWalletId)) {
- false // lost the race while deriving; the winner's copy stands
- } else {
- storePrivateKeyLocked(pubkeyHex, derived, ownerWalletId)
- true
- }
- }
-}
-```
-
-`addOwnerIfUsableLocked` treats "present but not
-`isPrivateKeyDecryptable`" the same as absent (so the legacy-blob
-re-derive path this codebase already supports keeps working — see
-"Spec review findings" #2) — only a present-and-decryptable entry
-short-circuits to "just add ownership." `IdentityKeyPrivateKeyDeriver.deriveAndStore`
-calls `storeIfAbsent` instead of the current separate
-`hasStored`/`storePrivateKey` pair; `PlatformWalletPersistenceHandler`'s
-`existedBefore` becomes `!storeIfAbsent(...)`'s result directly,
-removing the `runCatching { hasStored }.getOrDefault(true)` fallback
-entirely (a genuine simplification, not just a safety fix). This isn't
-a single atomic transaction (derivation still happens outside the
-lock, matching `storePrivateKey`'s existing outside-the-lock derive
-pattern today), but it closes the specific gap the finding names: the
-existence check and the eventual write are no longer two calls with an
-unguarded window where a sibling wallet's write is invisible to the
-first check *and* silently overwritten by the second.
-
-**3. Deletion tombstone**, used to fix the app-level-bypass half of
-`:783`:
-
-```kotlin
-// WalletStorage — guarded by privateKeyMutex. Process-lifetime, but
-// explicitly cleared on wallet (re-)creation (see below) — a deleted
-// wallet's id CAN be reused within one process (re-import of the same
-// recovery phrase after an accidental delete is a real, supported
-// flow), so this must not be "set once, never cleared."
-private val tombstonedWalletIds = mutableSetOf()
-```
-
-`removeWallet`'s locked section additionally calls
-`walletStorage.tombstoneWallet(walletId)` (same
-`withPrivateKeyExclusion` block, right alongside `deleteOwnerIndex`,
-so tombstoning happens atomically with the alias cleanup it's
-protecting). `PlatformWalletManager.createWallet` (`:541-...`, the
-single entry point for both "new wallet" and "restore/re-import from
-mnemonic" — same function, both paths pass a mnemonic) calls
-`walletStorage.clearTombstone(walletId)` right after the native create
-succeeds and before `storeMnemonic`, so a re-imported wallet with a
-previously-tombstoned id is immediately usable again, matching
-`storePrivateKey`'s pre-existing "keyed globally by wallet id,
-deterministic from seed+network" contract.
-
-`storePrivateKey` (and the new `storeIfAbsent`, via the same
-lock-held check) reject writes for a tombstoned `ownerWalletId`,
-throwing a new `WalletTombstonedException(walletId)` — the two
-app-level call sites (`CreateIdentityScreen.kt:219-227`,
-`IdentityKeyAdditionFlow.kt:161-165`) need to catch it. **Accepted,
-not fixed, in this pass:** `storePrivateKey`'s `ownerWalletId` param is
-nullable (`= null`); a null-owner call bypasses both the tombstone
-check and the owner-index union `removeWallet` reads. No current
-caller on the derive/register path passes null, so this is a
-theoretical gap, not a live bug — noted rather than closed by, e.g.,
-making the parameter non-nullable (a wider API change touching every
-call site for a path nothing currently exercises). What they *do* on
-catch is a product decision, not purely mechanical:
-
-- `CreateIdentityScreen`: the key was already derived and the identity
- isn't registered yet — the safest behavior is to abort registration
- with a clear "wallet was removed during setup" error, since there's
- no wallet left to register against.
-- `IdentityKeyAdditionFlow`: adding a key to an *existing* identity on
- a now-deleted wallet — same abort-with-error behavior; the identity
- itself is decoupled from the wallet's local state at this point.
-
-**I want to confirm that "abort with a clear error, don't silently
-drop it" is the right call for both sites before implementing** — it's
-the conservative default but it's the one piece of this section that's
-a product/UX decision rather than a pure correctness fix, so flagging
-it explicitly for sync rather than assuming.
-
-### Alternatives rejected
-
-- **Namespace ciphertext storage per-wallet** (so sibling wallets each
- get their own copy instead of sharing one `privkey.` entry).
- Rejected: this is a deliberate existing design choice (DIP-9 sibling
- wallets sharing one mnemonic are expected to share key material by
- construction), not a bug — re-namespacing would be a much larger,
- riskier storage-format migration for a problem the ownership-index
- fix already solves without touching the ciphertext layer.
-- **A single process-wide "wallet lifecycle" lock instead of the
- targeted tombstone check.** Rejected: `privateKeyMutex` already
- serializes every `WalletStorage` mutation; adding a *second*,
- coarser lock spanning arbitrary app-level coroutines
- (registration flows, key-addition flows) would be far more invasive
- and risks new deadlocks between `withCallbackExclusion` (native
- callback path) and app-driven UI coroutines. The tombstone check
- reuses the existing lock and only adds a rejection condition.
-
-### Failure modes
-
-- `isOwnedByAnotherWallet` false positive/negative: a false negative
- (misses a legitimate other-wallet claim) reproduces today's bug; a
- false positive (over-retains an alias nobody else needs) only costs
- a stray ciphertext entry, not a lost key — asymmetric risk correctly
- favors retention.
-- `storeIfAbsent` racing two *concurrent* callers for the *same* new
- pubkey: the mutex serializes them, so the second caller's `derive()`
- result is discarded once `hasPrivateKeyLocked` sees the first
- caller's write — matches today's `deriveAndStore`'s intended
- idempotency, now actually atomic.
-- Tombstone check: a wallet ID is only ever tombstoned by `removeWallet`
- itself, under the same lock as the check, so there's no window where
- a store could race the tombstone write.
-
-### Test plan
-
-New `WalletStorageTest.kt` (currently doesn't exist — flagged as a gap
-by the research pass):
-- `isOwnedByAnotherWallet` returns true when wallet B's owner index
- contains the alias and wallet A is being deleted; false when no
- other wallet claims it.
-- `storeIfAbsent` returns `false` and does not overwrite ciphertext
- when the pubkey already has an entry (from any owner); returns
- `true` and writes on first store; two concurrent calls for the same
- new pubkey result in exactly one ciphertext write.
-- `storePrivateKey`/`storeIfAbsent` throw `WalletTombstonedException`
- for a tombstoned wallet id; a non-tombstoned sibling wallet's calls
- are unaffected.
-
-Extend `removeWallet`'s coverage (no dedicated test file currently
-exercises `removeWallet` at all — another gap flagged by the research
-pass) with a sibling-wallet scenario: wallet A and B share a derived
-pubkey via B's *pending* (not-yet-committed) owner-index entry; delete
-A; assert B's ciphertext and owner-index entry survive.
-
----
-
-## §3. `CreateWalletScreen` mnemonic loss on config change (finding @ `:85`)
-
-### Problem
-
-`unrecoverablePhrase` (the sole surviving copy of a mnemonic when every
-durable store *and* rollback attempt failed, carried by
-`WalletCreateRollbackException.mnemonic`) is held in plain
-`remember { mutableStateOf(null) }` inside the `@Composable`.
-That's deliberately not `rememberSaveable` (so the plaintext never
-serializes into the saved-state `Bundle` — correct instinct) but as a
-result it also doesn't survive activity recreation from rotation,
-locale, or theme changes, which discards the composition and loses the
-only copy — the underlying wallet creation already ran, Room rows
-exist, but they're now permanently seedless with no recovery path.
-
-There is no existing ViewModel for this screen; wallet creation runs
-on `rememberCoroutineScope()`, not `viewModelScope`.
-
-### Chosen approach
-
-*(Revised after multi-agent spec review — the coroutine-scope change
-originally proposed here was dropped; see "Spec review findings"
-below.)*
-
-Add `CreateWalletViewModel : ViewModel()` holding
-`unrecoverablePhrase` as a plain `mutableStateOf` property —
-the same shape `TokenActionViewModel` already establishes elsewhere in
-this app module (plain `ViewModel`-retained `mutableStateOf`, not
-`SavedStateHandle`-backed) — so it survives config-change-driven
-recreation via the normal `viewModel()` retention contract, without
-ever touching `SavedStateHandle`/the Bundle:
-
-```kotlin
-class CreateWalletViewModel : ViewModel() {
- var unrecoverablePhrase by mutableStateOf(null)
- private set
-
- fun recordUnrecoverablePhrase(phrase: String) {
- unrecoverablePhrase = phrase
- }
-
- /** Explicit scrub once the user has acknowledged the backup dialog. */
- fun clearUnrecoverablePhrase() {
- unrecoverablePhrase = null
- }
-
- override fun onCleared() {
- unrecoverablePhrase = null
- super.onCleared()
- }
-}
-```
-
-`CreateWalletScreen` takes `viewModel: CreateWalletViewModel =
-viewModel()` (standard Compose factory). **The wallet-creation
-coroutine itself keeps launching on `rememberCoroutineScope()`, as
-today** — only *where the caught phrase is stored* changes. The
-`WalletCreateRollbackException` catch calls
-`viewModel.recordUnrecoverablePhrase(phrase)` instead of assigning
-local `remember` state; the emergency dialog reads
-`viewModel.unrecoverablePhrase` and calls
-`viewModel.clearUnrecoverablePhrase()` on acknowledgement — same UX,
-same non-dismissable-until-acknowledged shape, just backed by
-ViewModel state instead of composition-scoped `remember`. This is
-deliberately the smallest change that closes the named finding: the
-finding is about the phrase being lost *after* it was already
-successfully caught (a later rotation wipes the composition's
-`remember` state); it is not about the creation coroutine surviving
-mid-flight cancellation, which is a different (and not reported)
-concern. Moving the *launch* itself onto `viewModelScope` was
-considered and rejected — see "Spec review findings" #3.
-
-`String` immutability means true byte-level zeroization isn't possible
-here (same platform limitation the finding implicitly accepts by
-saying "explicitly scrubbed... field," not "zeroized bytes") — the
-`clear`/`onCleared` calls null the reference promptly, which is the
-practical ceiling for a JVM `String`. No existing code in this app
-module holds secrets in a `ViewModel`, so this establishes a new (but
-narrow, single-field) pattern rather than reusing one — noted for
-awareness, not a blocker.
-
-### Alternatives rejected
-
-- **`rememberSaveable` with a custom `Saver` that encrypts before
- serializing.** Rejected: still writes ciphertext into the
- saved-state `Bundle`, which can be included in Android's automatic
- backup / restored on a different security surface than intended;
- the finding explicitly calls for avoiding `SavedStateHandle`/Bundle
- entirely, and `ViewModel` retention already solves the actual
- problem (surviving config change) without that exposure.
-- **Process-level singleton / repository holding the phrase.**
- Rejected: broader lifetime than needed (a `ViewModel` scoped to this
- screen's `NavBackStackEntry` already outlives config changes and is
- cleared on real navigation-away, which is the right lifetime — a
- singleton would need its own manual clearing discipline for no
- benefit).
-
-### Test plan
-
-Kotlin unit test (Robolectric not required — pure `ViewModel` logic,
-no Room/Compose): `CreateWalletViewModelTest.kt` —
-`recordUnrecoverablePhrase` sets the field;
-`clearUnrecoverablePhrase`/`onCleared()` (via
-`ViewModelStore.clear()`) null it. This is a pure-JVM test unlike the
-findings in §1/§2, so it's the one item in this spec I can actually
-compile-check locally now that JDK 17 + the Android SDK were located
-on this machine — I'll run `./gradlew :app:testDebugUnitTest` for it
-alongside the others.
-
----
-
-## Cross-cutting test plan
-
-All four items get real `./gradlew :sdk:testDebugUnitTest
-:app:testDebugUnitTest` runs before this is called done — the earlier
-report that I couldn't test Kotlin locally was wrong; JDK 17
-(`openjdk@17` via Homebrew) and the Android SDK (`platforms/android-35`,
-`build-tools/35.0.0`) are both present on this machine, just not on
-the default `java_home`/`ANDROID_HOME` search path this session
-started with.
-
-## Open questions for sync (before coding)
-
-1. §2, tombstone rejection UX: confirm "abort registration/key-add
- with a clear error" is correct for both
- `CreateIdentityScreen`/`IdentityKeyAdditionFlow`, vs. some retry/
- silent-skip behavior.
-2. §2 is the largest, most fund/key-safety-critical piece of this
- spec (touches `WalletStorage`'s locking and storage format
- indirectly via the new tombstone set). Confirm you want it done in
- this pass rather than split into its own follow-up PR after #3999
- lands — it's the one item where "ship it now" vs. "land the other 3
- and follow up" is a real tradeoff given the review-cycle history on
- this branch already (140 review passes, most from automated
- reviewers finding new issues on each push).
-3. §2, accepted gap: `storePrivateKey`'s nullable `ownerWalletId`
- bypasses the tombstone/owner-index union entirely for a null-owner
- call. No current caller passes null on the derive/register path, so
- this is flagged rather than closed (closing it means auditing every
- `storePrivateKey` call site to require an owner). Confirm you're
- fine leaving this as a documented gap rather than widening the API
- change to cover it now.
diff --git a/package.json b/package.json
index fbedbcc4059..5ee76fe1334 100644
--- a/package.json
+++ b/package.json
@@ -1,6 +1,6 @@
{
"name": "@dashevo/platform",
- "version": "4.2.0-beta.4",
+ "version": "4.2.0-beta.6",
"private": true,
"scripts": {
"setup": "yarn install && yarn run build && yarn run configure",
diff --git a/packages/app-connect-contract/package.json b/packages/app-connect-contract/package.json
index d43d3d4df05..b9dc62ae9ea 100644
--- a/packages/app-connect-contract/package.json
+++ b/packages/app-connect-contract/package.json
@@ -1,6 +1,6 @@
{
"name": "@dashevo/app-connect-contract",
- "version": "4.2.0-beta.4",
+ "version": "4.2.0-beta.6",
"description": "A system contract for encrypted wallet-to-app login responses",
"scripts": {
"lint": "eslint .",
diff --git a/packages/bench-suite/package.json b/packages/bench-suite/package.json
index fda6ab3e6d6..ec2e7f25e35 100644
--- a/packages/bench-suite/package.json
+++ b/packages/bench-suite/package.json
@@ -1,7 +1,7 @@
{
"name": "@dashevo/bench-suite",
"private": true,
- "version": "4.2.0-beta.4",
+ "version": "4.2.0-beta.6",
"description": "Dash Platform benchmark tool",
"scripts": {
"bench": "node ./bin/bench.js",
diff --git a/packages/dapi-grpc/.npmignore b/packages/dapi-grpc/.npmignore
index 2b3ec03d4f9..11b735e7df2 100644
--- a/packages/dapi-grpc/.npmignore
+++ b/packages/dapi-grpc/.npmignore
@@ -6,3 +6,7 @@ node_modules
# Ultra runner build cache
.ultra.cache.json
+
+# Native generator regression tests and local Python bytecode are not runtime files.
+tests/codegen
+**/__pycache__
diff --git a/packages/dapi-grpc/README.md b/packages/dapi-grpc/README.md
index c6df2d5e98c..5f78069832e 100644
--- a/packages/dapi-grpc/README.md
+++ b/packages/dapi-grpc/README.md
@@ -140,3 +140,35 @@ Feel free to dive in! [Open an issue](https://github.com/dashpay/platform/issues
## License
[MIT](LICENSE) © Dash Core Group, Inc.
+
+## Building generated clients
+
+From the Platform monorepo, run `yarn install`, then provision the native
+client generators once:
+
+```sh
+python3 packages/dapi-grpc/scripts/setup-codegen.py --install
+yarn workspace @dashevo/dapi-grpc build
+```
+
+Installation needs Python 3.12 or newer, CMake and a C++ compiler. It builds
+checksum-verified sources into your user cache without sudo or Docker. Linux
+runner images provide the same tools at `/opt/client-codegen`. Set
+`DAPI_GRPC_TOOLCHAIN` to use an explicitly provisioned installation; a missing or
+mismatched installation fails rather than silently selecting a different protoc.
+
+`codegen.json` pins the recipe and native generator versions. The client compiler
+is deliberately separate from the Rust build's protoc 32.0: the existing client
+output uses protobuf 3.18.1, gRPC 1.46.3, gRPC Java 1.42.1 and the Yarn-locked
+`ts-protoc-gen` 0.15.0. Update the recipe, Platform lock and runner requirements
+together, with generated-output compatibility checks. Ordinary builds never
+install system packages or start containers.
+
+Generation stages all languages before replacing `clients/`, so a failed plugin
+preserves the previous output. Java, Objective-C and Python are generated for
+repository consumers; the NPM archive continues to exclude them and ships the
+Node and web clients. Run the generation regressions after installing the tools:
+
+```sh
+yarn workspace @dashevo/dapi-grpc exec python3 -m unittest discover -s tests/codegen -v
+```
diff --git a/packages/dapi-grpc/codegen.json b/packages/dapi-grpc/codegen.json
new file mode 100644
index 00000000000..78ce0fc3e06
--- /dev/null
+++ b/packages/dapi-grpc/codegen.json
@@ -0,0 +1,31 @@
+{
+ "recipe_repository": "dashpay/dash-selfhosted-image",
+ "recipe_revision": "e49e8bc9977f5f961a76ba1d1f7673c72173679f",
+ "recipe_files": {
+ "build.py": "17e260ff1e79e416d9addd7da32a7a04e10d7d4db07e3d919e34dac7e471dd49",
+ "CMakeLists.txt": "814bf56f8efd9d8ddd91e64201050ad8397071a03e21b59d6fbd83f365d137e4",
+ "lock.json": "f67f983d739633cf4632e9b30786a7928f1d526df969c4d4acea8804295232cc"
+ },
+ "toolchain": {
+ "schema": 1,
+ "versions": {
+ "protobuf": "3.18.1",
+ "grpc": "1.46.3",
+ "grpc_java": "1.42.1"
+ },
+ "sources": [
+ {
+ "url": "https://github.com/protocolbuffers/protobuf/releases/download/v3.18.1/protobuf-cpp-3.18.1.tar.gz",
+ "sha256": "6ee35eda3f79e49608d2ace8d866313fdec539d8bb14c6c54e8d2a16fa4e6780"
+ },
+ {
+ "url": "https://github.com/grpc/grpc/archive/refs/tags/v1.46.3.tar.gz",
+ "sha256": "d6cbf22cb5007af71b61c6be316a79397469c58c82a942552a62e708bce60964"
+ },
+ {
+ "url": "https://github.com/grpc/grpc-java/archive/refs/tags/v1.42.1.tar.gz",
+ "sha256": "33775a1ad05974bbba6ff97801cd9b326485b21fab91b08a4e1bc53e500e6326"
+ }
+ ]
+ }
+}
diff --git a/packages/dapi-grpc/package.json b/packages/dapi-grpc/package.json
index 3b278e77b29..6bf80b1d39a 100644
--- a/packages/dapi-grpc/package.json
+++ b/packages/dapi-grpc/package.json
@@ -1,6 +1,6 @@
{
"name": "@dashevo/dapi-grpc",
- "version": "4.2.0-beta.4",
+ "version": "4.2.0-beta.6",
"description": "DAPI GRPC definition file and generated clients",
"browser": "browser.js",
"main": "node.js",
@@ -60,6 +60,7 @@
"mocha": "^11.1.0",
"mocha-sinon": "^2.1.2",
"sinon": "^18.0.1",
- "sinon-chai": "^3.7.0"
+ "sinon-chai": "^3.7.0",
+ "ts-protoc-gen": "0.15.0"
}
}
diff --git a/packages/dapi-grpc/scripts/build.sh b/packages/dapi-grpc/scripts/build.sh
index 148404e9837..5d3f2e6cdb9 100755
--- a/packages/dapi-grpc/scripts/build.sh
+++ b/packages/dapi-grpc/scripts/build.sh
@@ -1,252 +1,66 @@
#!/usr/bin/env bash
-# shellcheck disable=SC2250
-#
+# Generate all published clients with the locked native toolchain.
+set -euo pipefail
-SKIP_GRPC_PROTO_BUILD=${SKIP_GRPC_PROTO_BUILD:-0}
-if [[ "${SKIP_GRPC_PROTO_BUILD}" == "1" ]]; then
- echo WARN: Skipping GRPC protobuf definitions rebuild
+if [[ "${SKIP_GRPC_PROTO_BUILD:-0}" == 1 ]]; then
+ echo 'WARN: Skipping GRPC protobuf definitions rebuild'
exit 0
fi
-PROTOS_PATH="$PWD/protos"
-
-CORE_PROTO_PATH="$PWD/protos/core/v0"
-CORE_CLIENTS_PATH="$PWD/clients/core/v0"
-
-PLATFORM_PROTO_PATH="$PWD/protos/platform/v0"
-PLATFORM_CLIENTS_PATH="$PWD/clients/platform/v0"
-
-DRIVE_PROTO_PATH="$PWD/protos/drive/v0"
-DRIVE_CLIENTS_PATH="$PWD/clients/drive/v0"
-
-CORE_WEB_OUT_PATH="$CORE_CLIENTS_PATH/web"
-PLATFORM_WEB_OUT_PATH="$PLATFORM_CLIENTS_PATH/web"
-DRIVE_WEB_OUT_PATH="$DRIVE_CLIENTS_PATH/web"
-
-CORE_JAVA_OUT_PATH="$CORE_CLIENTS_PATH/java"
-PLATFORM_JAVA_OUT_PATH="$PLATFORM_CLIENTS_PATH/java"
-
-CORE_OBJ_C_OUT_PATH="$CORE_CLIENTS_PATH/objective-c"
-PLATFORM_OBJ_C_OUT_PATH="$PLATFORM_CLIENTS_PATH/objective-c"
-
-CORE_PYTHON_OUT_PATH="$CORE_CLIENTS_PATH/python"
-PLATFORM_PYTHON_OUT_PATH="$PLATFORM_CLIENTS_PATH/python"
-
-PROTOC_IMAGE="rvolosatovs/protoc:4.0.0"
-
-set -ex
-
-#################################################
-# Generate JavaScript client for `Core` service #
-#################################################
-
-rm -rf "${CORE_WEB_OUT_PATH:?}/*" || true
-
-docker run -v "$CORE_PROTO_PATH:$CORE_PROTO_PATH" \
- -v "$CORE_WEB_OUT_PATH:$CORE_WEB_OUT_PATH" \
- --rm \
- "$PROTOC_IMAGE" \
- --js_out="import_style=commonjs:$CORE_WEB_OUT_PATH" \
- --ts_out="service=grpc-web:$CORE_WEB_OUT_PATH" \
- -I="$CORE_PROTO_PATH" \
- "core.proto"
-
-# Clean node message classes
-
-rm -rf "$CORE_CLIENTS_PATH/nodejs/*_protoc.js" || true
-rm -rf "$CORE_CLIENTS_PATH/nodejs/*_pbjs.js" || true
-
-# Copy compiled modules with message classes
-
-cp "$CORE_WEB_OUT_PATH/core_pb.js" "$CORE_CLIENTS_PATH/nodejs/core_protoc.js"
-
-# Generate node message classes
-pbjs \
- -t static-module \
- -w commonjs \
- -r core_root \
- -o "$CORE_CLIENTS_PATH/nodejs/core_pbjs.js" \
- "$CORE_PROTO_PATH/core.proto"
-
-#####################################################
-# Generate JavaScript client for `DriveInternal` service #
-#####################################################
-
-rm -rf "${DRIVE_WEB_OUT_PATH:?}/*" || true
-
-docker run -v "$DRIVE_PROTO_PATH:$DRIVE_PROTO_PATH" \
- -v "$DRIVE_WEB_OUT_PATH:$DRIVE_WEB_OUT_PATH" \
- -v "$PROTOS_PATH:$PROTOS_PATH" \
- --rm \
- "$PROTOC_IMAGE" \
- --js_out="import_style=commonjs:$DRIVE_WEB_OUT_PATH" \
- --ts_out="service=grpc-web:$DRIVE_WEB_OUT_PATH" \
- -I="$DRIVE_PROTO_PATH" \
- -I="$PROTOS_PATH" \
- "drive.proto"
-
-# Clean node message classes
-
-rm -rf "$DRIVE_CLIENTS_PATH/nodejs/*_protoc.js" || true
-rm -rf "$DRIVE_CLIENTS_PATH/nodejs/*_pbjs.js" || true
-
-# Copy compiled modules with message classes
-
-cp "$DRIVE_WEB_OUT_PATH/drive_pb.js" "$DRIVE_CLIENTS_PATH/nodejs/drive_protoc.js"
-
-pbjs \
- -t static-module \
- -w commonjs \
- -r platform_root \
- -p "$PROTOS_PATH" \
- -o "$DRIVE_CLIENTS_PATH/nodejs/drive_pbjs.js" \
- "$DRIVE_PROTO_PATH/drive.proto"
-
-#####################################################
-# Generate JavaScript client for `Platform` service #
-#####################################################
-
-rm -rf "${PLATFORM_WEB_OUT_PATH:?}/*" || true
-
-docker run -v "$PLATFORM_PROTO_PATH:$PLATFORM_PROTO_PATH" \
- -v "$PLATFORM_WEB_OUT_PATH:$PLATFORM_WEB_OUT_PATH" \
- -v "$PROTOS_PATH:$PROTOS_PATH" \
- --rm \
- "$PROTOC_IMAGE" \
- --js_out="import_style=commonjs:$PLATFORM_WEB_OUT_PATH" \
- --ts_out="service=grpc-web:$PLATFORM_WEB_OUT_PATH" \
- -I="$PLATFORM_PROTO_PATH" \
- -I="$PROTOS_PATH" \
- "platform.proto"
-
-# Clean node message classes
-
-rm -rf "$PLATFORM_CLIENTS_PATH/nodejs/*_protoc.js" || true
-rm -rf "$PLATFORM_CLIENTS_PATH/nodejs/*_pbjs.js" || true
-
-# Copy compiled modules with message classes
-
-cp "$PLATFORM_WEB_OUT_PATH/platform_pb.js" "$PLATFORM_CLIENTS_PATH/nodejs/platform_protoc.js"
-
-pbjs \
- -t static-module \
- -w commonjs \
- -r platform_root \
- -o "$PLATFORM_CLIENTS_PATH/nodejs/platform_pbjs.js" \
- -p "$PROTOS_PATH" \
- "$PLATFORM_PROTO_PATH/platform.proto"
-
-###################################
-# Generate Java client for `Core` #
-###################################
-
-rm -rf "${CORE_JAVA_OUT_PATH:?}/*" || true
-
-docker run -v "$CORE_PROTO_PATH:$CORE_PROTO_PATH" \
- -v "$CORE_JAVA_OUT_PATH:$CORE_JAVA_OUT_PATH" \
- -v "$PROTOS_PATH:$PROTOS_PATH" \
- --rm \
- "$PROTOC_IMAGE" \
- --plugin=protoc-gen-grpc=/usr/bin/protoc-gen-grpc-java \
- --grpc-java_out="$CORE_JAVA_OUT_PATH" \
- --proto_path="$CORE_PROTO_PATH" \
- -I="$CORE_PROTO_PATH" \
- -I="$PROTOS_PATH" \
- "core.proto"
-
-#######################################
-# Generate Java client for `Platform` #
-#######################################
-
-rm -rf "${PLATFORM_JAVA_OUT_PATH:?}/*" || true
-
-docker run -v "$PLATFORM_PROTO_PATH:$PLATFORM_PROTO_PATH" \
- -v "$PLATFORM_JAVA_OUT_PATH:$PLATFORM_JAVA_OUT_PATH" \
- -v "$PROTOS_PATH:$PROTOS_PATH" \
- --rm \
- "$PROTOC_IMAGE" \
- --plugin=protoc-gen-grpc=/usr/bin/protoc-gen-grpc-java \
- --grpc-java_out="$PLATFORM_JAVA_OUT_PATH" \
- --proto_path="$PLATFORM_PROTO_PATH" \
- -I="$PLATFORM_PROTO_PATH" \
- -I="$PROTOS_PATH" \
- "platform.proto"
-
-##########################################
-# Generate Objective-C client for `Core` #
-##########################################
-
-rm -rf "${CORE_OBJ_C_OUT_PATH:?}/*" || true
-
-docker run -v "$CORE_PROTO_PATH:$CORE_PROTO_PATH" \
- -v "$CORE_OBJ_C_OUT_PATH:$CORE_OBJ_C_OUT_PATH" \
- -v "$PROTOS_PATH:$PROTOS_PATH" \
- --rm \
- "$PROTOC_IMAGE" \
- --plugin=protoc-gen-grpc=/usr/bin/grpc_objective_c_plugin \
- --objc_out="$CORE_OBJ_C_OUT_PATH" \
- --grpc_out="$CORE_OBJ_C_OUT_PATH" \
- --proto_path="$CORE_PROTO_PATH" \
- -I="$CORE_PROTO_PATH" \
- -I="$PROTOS_PATH" \
- "core.proto"
-
-##############################################
-# Generate Objective-C client for `Platform` #
-##############################################
-
-rm -rf "${PLATFORM_OBJ_C_OUT_PATH:?}/*" || true
-
-docker run -v "$PLATFORM_PROTO_PATH:$PLATFORM_PROTO_PATH" \
- -v "$PLATFORM_OBJ_C_OUT_PATH:$PLATFORM_OBJ_C_OUT_PATH" \
- -v "$PROTOS_PATH:$PROTOS_PATH" \
- --rm \
- "$PROTOC_IMAGE" \
- --plugin=protoc-gen-grpc=/usr/bin/grpc_objective_c_plugin \
- --objc_out="$PLATFORM_OBJ_C_OUT_PATH" \
- --grpc_out="$PLATFORM_OBJ_C_OUT_PATH" \
- --proto_path="$PLATFORM_PROTO_PATH" \
- -I="$PLATFORM_PROTO_PATH" \
- -I="$PROTOS_PATH" \
- "platform.proto"
-
-#####################################
-# Generate Python client for `Core` #
-#####################################
-
-rm -rf "${CORE_PYTHON_OUT_PATH:?}/*" || true
-
-docker run -v "$CORE_PROTO_PATH:$CORE_PROTO_PATH" \
- -v "$CORE_PYTHON_OUT_PATH:$CORE_PYTHON_OUT_PATH" \
- -v "$PROTOS_PATH:$PROTOS_PATH" \
- --rm \
- "$PROTOC_IMAGE" \
- --plugin=protoc-gen-grpc=/usr/bin/grpc_python_plugin \
- --python_out="$CORE_PYTHON_OUT_PATH" \
- --grpc_out="$CORE_PYTHON_OUT_PATH" \
- --proto_path="$CORE_PROTO_PATH" \
- -I="$CORE_PROTO_PATH" \
- -I="$PROTOS_PATH" \
- "core.proto"
-
-#########################################
-# Generate Python client for `Platform` #
-#########################################
-
-rm -rf "${PLATFORM_PYTHON_OUT_PATH:?}/*" || true
-
-docker run -v "$PLATFORM_PROTO_PATH:$PLATFORM_PROTO_PATH" \
- -v "$PLATFORM_PYTHON_OUT_PATH:$PLATFORM_PYTHON_OUT_PATH" \
- -v "$PROTOS_PATH:$PROTOS_PATH" \
- --rm \
- "$PROTOC_IMAGE" \
- --plugin=protoc-gen-grpc=/usr/bin/grpc_python_plugin \
- --python_out="$PLATFORM_PYTHON_OUT_PATH" \
- --grpc_out="$PLATFORM_PYTHON_OUT_PATH" \
- --proto_path="$PLATFORM_PROTO_PATH" \
- -I="$PLATFORM_PROTO_PATH" \
- -I="$PROTOS_PATH" \
- "platform.proto"
-
-# Patch generated protobuf files
-exec "${PWD}/scripts/patch-protobuf-js.sh"
+PACKAGE_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
+TOOLCHAIN=$(python3 "$PACKAGE_DIR/scripts/setup-codegen.py")
+PROTOC="$TOOLCHAIN/bin/protoc"
+TS_PLUGIN=$(command -v protoc-gen-ts)
+command -v pbjs >/dev/null
+
+# Keep the existing clients intact when any generator fails. Stage beneath the
+# package so replacement stays on the same filesystem and Yarn PnP still works.
+STAGING=$(mktemp -d "$PACKAGE_DIR/.clients-build.XXXXXX")
+cleanup() {
+ # Restore the previous tree if interrupted between the two renames.
+ if [[ ! -e "$PACKAGE_DIR/clients" && -d "$STAGING/previous" ]]; then
+ mv "$STAGING/previous" "$PACKAGE_DIR/clients"
+ fi
+ rm -rf "$STAGING"
+}
+trap cleanup EXIT
+trap 'exit 130' INT
+trap 'exit 143' TERM
+cp -R "$PACKAGE_DIR/clients" "$STAGING/clients"
+
+for service in core drive platform; do
+ output="$STAGING/clients/$service/v0"
+ schema="$PACKAGE_DIR/protos/$service/v0/$service.proto"
+ includes=(-I"$PACKAGE_DIR/protos/$service/v0" -I"$PACKAGE_DIR/protos" -I"$TOOLCHAIN/include")
+ # Preserve handwritten PromiseClient modules and documentation.
+ rm -f "$output/web/${service}_pb"* "$output/nodejs/${service}_protoc.js" "$output/nodejs/${service}_pbjs.js"
+ "$PROTOC" "${includes[@]}" --plugin="protoc-gen-ts=$TS_PLUGIN" \
+ --js_out="import_style=commonjs:$output/web" \
+ --ts_out="service=grpc-web:$output/web" "$schema"
+ cp "$output/web/${service}_pb.js" "$output/nodejs/${service}_protoc.js"
+ root=platform_root
+ if [[ "$service" == core ]]; then root=core_root; fi
+ pbjs -t static-module -w commonjs -r "$root" -p "$PACKAGE_DIR/protos" \
+ -o "$output/nodejs/${service}_pbjs.js" "$schema"
+ if [[ "$service" != drive ]]; then
+ for language in java objective-c python; do
+ rm -rf "${output:?}/$language"
+ mkdir -p "$output/$language"
+ done
+ "$PROTOC" "${includes[@]}" --plugin="protoc-gen-grpc-java=$TOOLCHAIN/bin/protoc-gen-grpc-java" \
+ --grpc-java_out="$output/java" "$schema"
+ "$PROTOC" "${includes[@]}" --plugin="protoc-gen-grpc=$TOOLCHAIN/bin/grpc_objective_c_plugin" \
+ --objc_out="$output/objective-c" --grpc_out="$output/objective-c" "$schema"
+ "$PROTOC" "${includes[@]}" --plugin="protoc-gen-grpc=$TOOLCHAIN/bin/grpc_python_plugin" \
+ --python_out="$output/python" --grpc_out="$output/python" "$schema"
+ fi
+done
+
+(cd "$STAGING" && "$PACKAGE_DIR/scripts/patch-protobuf-js.sh")
+# Generation and patching have all succeeded. Preserve handwritten files by
+# replacing the staged copy of the complete client tree.
+mv "$PACKAGE_DIR/clients" "$STAGING/previous"
+if ! mv "$STAGING/clients" "$PACKAGE_DIR/clients"; then
+ mv "$STAGING/previous" "$PACKAGE_DIR/clients"
+ exit 1
+fi
diff --git a/packages/dapi-grpc/scripts/check-packed-clients.py b/packages/dapi-grpc/scripts/check-packed-clients.py
new file mode 100644
index 00000000000..e46913b4a24
--- /dev/null
+++ b/packages/dapi-grpc/scripts/check-packed-clients.py
@@ -0,0 +1,32 @@
+#!/usr/bin/env python3
+"""Check that packing preserved the freshly generated DAPI clients."""
+import json
+from pathlib import Path
+import sys
+import tarfile
+
+package = Path(__file__).resolve().parent.parent
+expected = {p.relative_to(package).as_posix(): p.read_bytes()
+ for p in (package / 'clients').rglob('*')
+ if p.is_file() and p.relative_to(package).parts[3] in ('web', 'nodejs')}
+required = {f'clients/{service}/v0/{directory}/{service}{suffix}'
+ for service in ('core', 'drive', 'platform')
+ for directory, suffix in (('web', '_pb.js'), ('web', '_pb.d.ts'),
+ ('web', '_pb_service.js'), ('web', '_pb_service.d.ts'),
+ ('nodejs', '_protoc.js'), ('nodejs', '_pbjs.js'))}
+if not required <= expected.keys():
+ raise SystemExit('Missing generated source clients: ' + ', '.join(sorted(required - expected.keys())))
+found = 0
+for archive in Path(sys.argv[1]).glob('*.tgz'):
+ with tarfile.open(archive) as tar:
+ manifest = tar.extractfile('package/package.json')
+ if manifest is None or json.load(manifest)['name'] != '@dashevo/dapi-grpc':
+ continue
+ found += 1
+ for name, content in expected.items():
+ member = tar.extractfile('package/' + name)
+ if member is None or member.read() != content:
+ raise SystemExit(f'Packed client differs from generated source: {name}')
+if found != 1:
+ raise SystemExit(f'Expected one DAPI package archive, found {found}')
+print(f'Verified {len(expected)} packed client files')
diff --git a/packages/dapi-grpc/scripts/patch-protobuf-js.sh b/packages/dapi-grpc/scripts/patch-protobuf-js.sh
index d046c513ff6..2c6fd34dc16 100755
--- a/packages/dapi-grpc/scripts/patch-protobuf-js.sh
+++ b/packages/dapi-grpc/scripts/patch-protobuf-js.sh
@@ -1,8 +1,8 @@
#!/bin/bash
# shellcheck disable=SC2250
-set -e
+set -euo pipefail
-files=$(find "$PWD/clients/core/v0/web" "$PWD/clients/core/v0/nodejs" "$PWD/clients/platform/v0/web" "$PWD/clients/platform/v0/nodejs" "$PWD/clients/drive/v0/web" "$PWD/clients/drive/v0/nodejs" -name "*_pb.js" -o -name "*_protoc.js") OS=$(uname)
+OS=$(uname)
function replace_in_file() {
if [[ "$OS" = 'Darwin' ]]; then
@@ -15,18 +15,23 @@ function replace_in_file() {
}
# Loop over the files
-for file in $files; do
+while IFS= read -r -d '' file; do
replace_in_file 's/var global = Function('\''return this'\'')();/const proto = {};/g' "$file"
if grep -qrE "[^a-zA-Z]Function\(" "$file"; then
- echo "Error: Function( still present"
+ echo "Error: Function( still present in $file" >&2
+ exit 1
fi
replace_in_file 's/, global);/, { proto });/g' "$file"
if grep -qrE '(^|[^a-zA-Z."])global([^a-zA-Z]|$)' "$file"; then
- echo "Error: global still present"
+ echo "Error: global still present in $file" >&2
+ exit 1
fi
replace_in_file 's/require('\''.\/platform\/v0\/platform_pb.js'\'')/require('\''..\/..\/..\/platform\/v0\/web\/platform_pb.js'\'')/g' "$file"
-done
+done < <(find "$PWD/clients/core/v0/web" "$PWD/clients/core/v0/nodejs" \
+ "$PWD/clients/platform/v0/web" "$PWD/clients/platform/v0/nodejs" \
+ "$PWD/clients/drive/v0/web" "$PWD/clients/drive/v0/nodejs" \
+ \( -name "*_pb.js" -o -name "*_protoc.js" \) -print0)
diff --git a/packages/dapi-grpc/scripts/setup-codegen.py b/packages/dapi-grpc/scripts/setup-codegen.py
new file mode 100644
index 00000000000..957c82d9319
--- /dev/null
+++ b/packages/dapi-grpc/scripts/setup-codegen.py
@@ -0,0 +1,74 @@
+#!/usr/bin/env python3
+"""Verify or install the pinned native DAPI generators (no root or Docker)."""
+import argparse
+import hashlib
+import json
+import os
+from pathlib import Path
+import platform
+import subprocess
+import sys
+import tempfile
+import urllib.request
+
+PACKAGE = Path(__file__).resolve().parent.parent
+CONFIG = PACKAGE / 'codegen.json'
+BINARIES = ('protoc', 'protoc-gen-grpc-java', 'grpc_objective_c_plugin', 'grpc_python_plugin')
+
+
+def verify(destination, config):
+ if json.loads((destination / 'lock.json').read_text()) != config['toolchain']:
+ raise ValueError(f'Client generator versions differ at {destination}')
+ for binary in BINARIES:
+ if not os.access(destination / 'bin' / binary, os.X_OK):
+ raise ValueError(f'Missing client generator: {binary}')
+ version = subprocess.check_output([str(destination / 'bin/protoc'), '--version'], text=True).strip()
+ if version != 'libprotoc ' + config['toolchain']['versions']['protobuf']:
+ raise ValueError(f'Wrong client compiler: {version}')
+
+
+def main():
+ parser = argparse.ArgumentParser(description=__doc__)
+ parser.add_argument('--install', action='store_true', help='Build a user-local toolchain when absent')
+ args = parser.parse_args()
+ config = json.loads(CONFIG.read_text())
+ revision = config['recipe_revision']
+ cache = Path(os.environ.get('XDG_CACHE_HOME', str(Path.home() / '.cache')))
+ local = cache / 'dash/client-codegen' / f'{revision}-{platform.system()}-{platform.machine()}'
+ explicit = os.environ.get('DAPI_GRPC_TOOLCHAIN')
+ if explicit:
+ destination = Path(explicit).resolve()
+ elif Path('/opt/client-codegen').exists():
+ destination = Path('/opt/client-codegen')
+ else:
+ destination = local
+ if destination.exists():
+ verify(destination, config)
+ elif args.install and not explicit:
+ with tempfile.TemporaryDirectory(prefix='client-codegen-recipe-') as tmp:
+ recipe = Path(tmp)
+ for name, expected in config['recipe_files'].items():
+ url = f"https://raw.githubusercontent.com/{config['recipe_repository']}/{revision}/client-codegen/{name}"
+ with urllib.request.urlopen(url, timeout=120) as response:
+ content = response.read()
+ if hashlib.sha256(content).hexdigest() != expected:
+ raise ValueError(f'Recipe checksum mismatch: {name}')
+ (recipe / name).write_bytes(content)
+ if json.loads((recipe / 'lock.json').read_text()) != config['toolchain']:
+ raise ValueError('Recipe and Platform client toolchain locks differ')
+ subprocess.run([sys.executable, str(recipe / 'build.py'), str(destination)],
+ check=True, stdout=sys.stderr)
+ verify(destination, config)
+ else:
+ raise ValueError('Native client generators are missing. Run '
+ '`python3 packages/dapi-grpc/scripts/setup-codegen.py --install` '
+ '(requires Python 3.12+, CMake and a C++ compiler), or provision the matching runner image.')
+ print(destination)
+
+
+if __name__ == '__main__':
+ try:
+ main()
+ except (ValueError, OSError, subprocess.CalledProcessError) as error:
+ print(f'Client codegen: {error}', file=sys.stderr)
+ sys.exit(1)
diff --git a/packages/dapi-grpc/tests/codegen/test_generation.py b/packages/dapi-grpc/tests/codegen/test_generation.py
new file mode 100644
index 00000000000..1743421dbd0
--- /dev/null
+++ b/packages/dapi-grpc/tests/codegen/test_generation.py
@@ -0,0 +1,78 @@
+"""Integration regressions; run with `yarn workspace @dashevo/dapi-grpc exec python3 -m unittest discover -s tests/codegen -v`."""
+import hashlib
+import json
+import os
+from pathlib import Path
+import shutil
+import subprocess
+import tempfile
+import unittest
+
+PACKAGE = Path(__file__).resolve().parents[2]
+
+
+def snapshot(path):
+ return {str(p.relative_to(path)): hashlib.sha256(p.read_bytes()).hexdigest()
+ for p in path.rglob('*') if p.is_file()}
+
+
+class GenerationTests(unittest.TestCase):
+ @classmethod
+ def setUpClass(cls):
+ cls.toolchain = Path(subprocess.check_output(
+ ['python3', str(PACKAGE / 'scripts/setup-codegen.py')], text=True).strip())
+
+ def setUp(self):
+ self.tmp = tempfile.TemporaryDirectory(prefix='client generation with spaces ')
+ self.addCleanup(self.tmp.cleanup)
+ self.package = Path(self.tmp.name) / 'dapi-grpc'
+ shutil.copytree(PACKAGE, self.package, ignore=shutil.ignore_patterns('.clients-build.*'))
+
+ def run_build(self, toolchain=None):
+ env = dict(os.environ, DAPI_GRPC_TOOLCHAIN=str(toolchain or self.toolchain))
+ return subprocess.run(['bash', str(self.package / 'scripts/build.sh')],
+ env=env, capture_output=True, text=True)
+
+ def test_should_generate_all_languages_and_be_idempotent(self):
+ manual = self.package / 'clients/core/v0/web/handwritten.txt'
+ manual.write_text('preserve me')
+ result = self.run_build()
+ self.assertEqual(result.returncode, 0, result.stderr)
+ clients = self.package / 'clients'
+ for name in ('core/v0/java/org/dash/platform/dapi/v0/CoreGrpc.java',
+ 'core/v0/objective-c/Core.pbrpc.h', 'core/v0/python/core_pb2_grpc.py',
+ 'platform/v0/web/platform_pb_service.d.ts', 'drive/v0/nodejs/drive_pbjs.js'):
+ self.assertTrue((clients / name).is_file(), name)
+ first = snapshot(clients)
+ result = self.run_build()
+ self.assertEqual(result.returncode, 0, result.stderr)
+ self.assertEqual(snapshot(clients), first)
+ self.assertEqual(manual.read_text(), 'preserve me')
+
+ def test_should_preserve_clients_when_a_late_generator_fails(self):
+ broken = Path(self.tmp.name) / 'broken-toolchain'
+ (broken / 'bin').mkdir(parents=True)
+ shutil.copy2(self.toolchain / 'lock.json', broken / 'lock.json')
+ for binary in (self.toolchain / 'bin').iterdir():
+ if binary.name != 'grpc_python_plugin':
+ (broken / 'bin' / binary.name).symlink_to(binary)
+ (broken / 'include').symlink_to(self.toolchain / 'include', target_is_directory=True)
+ plugin = broken / 'bin/grpc_python_plugin'
+ plugin.write_text('#!/bin/sh\nexit 23\n')
+ plugin.chmod(0o755)
+ before = snapshot(self.package / 'clients')
+ result = self.run_build(broken)
+ self.assertNotEqual(result.returncode, 0)
+ self.assertIn('Plugin failed', result.stderr)
+ self.assertEqual(snapshot(self.package / 'clients'), before)
+ self.assertEqual(list(self.package.glob('.clients-build.*')), [])
+
+ def test_should_reject_mismatched_toolchain_before_touching_clients(self):
+ broken = Path(self.tmp.name) / 'wrong-version'
+ broken.mkdir()
+ (broken / 'lock.json').write_text(json.dumps({'versions': {'protobuf': '32.0'}}))
+ before = snapshot(self.package / 'clients')
+ result = self.run_build(broken)
+ self.assertNotEqual(result.returncode, 0)
+ self.assertIn('versions differ', result.stderr)
+ self.assertEqual(snapshot(self.package / 'clients'), before)
diff --git a/packages/dapi/package.json b/packages/dapi/package.json
index 658b3f21e6e..cb61fc42735 100644
--- a/packages/dapi/package.json
+++ b/packages/dapi/package.json
@@ -1,7 +1,7 @@
{
"name": "@dashevo/dapi",
"private": true,
- "version": "4.2.0-beta.4",
+ "version": "4.2.0-beta.6",
"description": "A decentralized API for the Dash network",
"scripts": {
"api": "node scripts/api.js",
diff --git a/packages/dash-spv/package.json b/packages/dash-spv/package.json
index 7195da4b0ef..e84c5cf8e5e 100644
--- a/packages/dash-spv/package.json
+++ b/packages/dash-spv/package.json
@@ -1,6 +1,6 @@
{
"name": "@dashevo/dash-spv",
- "version": "5.2.0-beta.4",
+ "version": "5.2.0-beta.6",
"description": "Repository containing SPV functions used by @dashevo",
"main": "index.js",
"scripts": {
diff --git a/packages/dashmate/package.json b/packages/dashmate/package.json
index 85e8f347e12..9c3942c3a75 100644
--- a/packages/dashmate/package.json
+++ b/packages/dashmate/package.json
@@ -1,6 +1,6 @@
{
"name": "dashmate",
- "version": "4.2.0-beta.4",
+ "version": "4.2.0-beta.6",
"description": "Distribution package for Dash node installation",
"scripts": {
"lint": "eslint .",
diff --git a/packages/dashpay-contract/README.md b/packages/dashpay-contract/README.md
index 7e87e8cb45d..8c36e98903a 100644
--- a/packages/dashpay-contract/README.md
+++ b/packages/dashpay-contract/README.md
@@ -59,6 +59,11 @@ To run tests, simply run
npm test
```
+## Design notes
+
+- Count proofs are public. Never add a countable index on `contactRequest` that groups by sender under a recipient, such as `[toUserId, $ownerId]`: it would let anyone list who contacted whom, with counts. A count keyed on `toUserId` alone reveals only a total.
+- Ignoring a sender is local to each device. If it ever syncs across devices, it must be one list the owner encrypts to themselves, not a `contactInfo` document per ignored sender. A `contactInfo` about someone who is not a contact exists publicly, and its creation time lines up with the incoming `contactRequest`, which reveals who was ignored.
+
## Contributing
Feel free to dive in! [Open an issue](https://github.com/dashpay/platform/issues/new/choose) or submit PRs.
diff --git a/packages/dashpay-contract/package.json b/packages/dashpay-contract/package.json
index 7bb8d1619e6..28ecd64b10f 100644
--- a/packages/dashpay-contract/package.json
+++ b/packages/dashpay-contract/package.json
@@ -1,6 +1,6 @@
{
"name": "@dashevo/dashpay-contract",
- "version": "4.2.0-beta.4",
+ "version": "4.2.0-beta.6",
"description": "Reference contract of the DashPay DPA on Dash Evolution",
"scripts": {
"lint": "eslint .",
diff --git a/packages/document-history-contract/package.json b/packages/document-history-contract/package.json
index 138e4a2ecd1..e1e1582d7b4 100644
--- a/packages/document-history-contract/package.json
+++ b/packages/document-history-contract/package.json
@@ -1,6 +1,6 @@
{
"name": "@dashevo/document-history-contract",
- "version": "4.2.0-beta.4",
+ "version": "4.2.0-beta.6",
"description": "The document history contract",
"scripts": {
"lint": "eslint .",
diff --git a/packages/dpns-contract/package.json b/packages/dpns-contract/package.json
index 90d40930aed..b0a72e1db95 100644
--- a/packages/dpns-contract/package.json
+++ b/packages/dpns-contract/package.json
@@ -1,6 +1,6 @@
{
"name": "@dashevo/dpns-contract",
- "version": "4.2.0-beta.4",
+ "version": "4.2.0-beta.6",
"description": "A contract and helper scripts for DPNS DApp",
"scripts": {
"lint": "eslint .",
diff --git a/packages/js-dapi-client/package.json b/packages/js-dapi-client/package.json
index bf7e4233931..5e586c7a5b8 100644
--- a/packages/js-dapi-client/package.json
+++ b/packages/js-dapi-client/package.json
@@ -1,6 +1,6 @@
{
"name": "@dashevo/dapi-client",
- "version": "4.2.0-beta.4",
+ "version": "4.2.0-beta.6",
"description": "Client library used to access Dash DAPI endpoints",
"main": "lib/index.js",
"contributors": [
diff --git a/packages/js-dash-sdk/package.json b/packages/js-dash-sdk/package.json
index 8faee5202bd..5488c3a8ad1 100644
--- a/packages/js-dash-sdk/package.json
+++ b/packages/js-dash-sdk/package.json
@@ -1,6 +1,6 @@
{
"name": "dash",
- "version": "7.2.0-beta.4",
+ "version": "7.2.0-beta.6",
"description": "Dash library for JavaScript/TypeScript ecosystem (Wallet, DAPI, Primitives, BLS, ...)",
"main": "build/index.js",
"unpkg": "dist/dash.min.js",
diff --git a/packages/js-evo-sdk/README.md b/packages/js-evo-sdk/README.md
index 0d58f661fa0..ef5830eb823 100644
--- a/packages/js-evo-sdk/README.md
+++ b/packages/js-evo-sdk/README.md
@@ -19,6 +19,8 @@ Evo SDK provides a high-level, strongly-typed interface for interacting with [Da
- [Building a document create transition by hand](#building-a-document-create-transition-by-hand)
- [Immutable properties (`immutable`)](#immutable-properties-immutable)
- [Property constraints (`propertyConstraints`)](#property-constraints-propertyconstraints)
+- [How a document type is stored (`documentTypeLayout`)](#how-a-document-type-is-stored-documenttypelayout)
+- [What a document costs (`documentCreateCost`)](#what-a-document-costs-documentcreatecost)
- [Chained queries (provable semi-join)](#chained-queries-provable-semi-join)
- [Composite queries (a page plus its sub-queries)](#composite-queries-a-page-plus-its-sub-queries)
- [Contributing](#contributing)
@@ -247,7 +249,13 @@ try {
import { Document, DocumentCreateTransition, BatchTransition } from '@dashevo/evo-sdk';
const document = new Document({ properties, documentTypeName, dataContractId, ownerId });
-const transition = new DocumentCreateTransition({ document, identityContractNonce: nonce });
+// undefined unless the document enters a contest (a DPNS name, a moderation charter)
+const prefundedVotingBalance = await sdk.documents.contestFundToJoin(document);
+const transition = new DocumentCreateTransition({
+ document,
+ identityContractNonce: nonce,
+ prefundedVotingBalance,
+});
const batch = BatchTransition.fromBatchedTransitions([transition.toDocumentTransition()], ownerId, 0); // userFeeIncrease
const stateTransition = batch.toStateTransition();
// sign, then sdk.stateTransitions.broadcast(stateTransition)
@@ -255,6 +263,10 @@ const stateTransition = batch.toStateTransition();
From protocol version 14 the id of a new document commits to the identity contract nonce of its create transition. `new DocumentCreateTransition(...)` derives that id from the document's entropy and `identityContractNonce`, puts it on the transition and writes it back onto `document`, so `document.id` is final once the transition exists and equals `transition.base.id`. Before that the `Document` carries a placeholder. To know the id earlier, `document.setIdForCreation(nonce)` or `Document.generateId(type, owner, contract, entropy, nonce)`, or pass `identityContractNonce` to the `Document` constructor. Pass `platformVersion` (defaults to latest) to any of them for a network on an earlier protocol version. No app needs to reimplement the hash.
+A document whose values fall under a contested index enters a contest, and its create must state the most it pays into the contest's fund: without it, or stating less than the fund to join, Platform refuses the create with error 40114 and still charges its fees. From protocol version 14 that fund doubles once the contest holds 250 contenders and again for every 50 more. `sdk.documents.create` states it itself. For a transition built by hand, `sdk.documents.contestFundToJoin(document)` reads the contest's contenders (one proved query per 100) and returns the `PrefundedVotingBalance` to pass, the contested index's name and the fund to join now, or `undefined` for a document that joins no contest. Without the SDK, pass the document's contract as `dataContract` to `new DocumentCreateTransition(...)`: a contested document then states the contest's fund on its contested index, what joining costs below 250 contenders, and `contestFund` replaces that amount.
+
+From protocol version 14 a create may state more, as headroom for contenders joining before it lands: `new PrefundedVotingBalance({ indexName: prefundedVotingBalance.indexName, credits: 2n * prefundedVotingBalance.credits })`. Platform charges only the fund to join, but the identity must hold what the create states. Before 14 the stated amount must be exactly the contest's fund, and a create stating more is refused.
+
## Encrypted properties (`encryptedFor`)
From protocol version 14 a byte array property can declare how its ciphertext was produced, so a wallet reads the recipe from the contract instead of a side channel: the recipient (an identifier property of the same document type, or `$ownerId` for a message the writer encrypts to themself), the integer properties carrying the recipient's and the sender's key ids, and the scheme. The one scheme today, `ecdh-secp256k1-aes256-cbc`, is the dashpay contact request's: a random 16-byte IV followed by AES-256-CBC with PKCS7 padding under the libsecp256k1 ECDH shared key of the two identities' keys. A fetched contract can be asked what it declares:
@@ -399,7 +411,7 @@ try {
## Property constraints (`propertyConstraints`)
-From protocol version 14 a document type can declare rules its documents' integer properties must meet, each a comparison of two integer expressions built from property paths and integer values:
+From protocol version 14 a document type can declare rules its documents' properties must meet, each a comparison of two integer expressions built from property paths and integer values, an `in` list of values, a comparison of a string property with string constants, a `present` or `absent` test, or `anyOf`, `allOf` or `not` over such conditions:
```json
"propertyConstraints": {
@@ -411,11 +423,21 @@ From protocol version 14 a document type can declare rules its documents' intege
},
"minimumOrder": {
"greaterThanOrEqual": [{ "multiply": ["price", { "ifAbsent": ["quantity", 1] }] }, 100]
+ },
+ "feeWaivedOrAtLeastTen": {
+ "anyOf": [{ "equal": ["fee", 0] }, { "greaterThanOrEqual": ["fee", 10] }]
+ },
+ "discountGivenAboveZero": {
+ "anyOf": [{ "absent": "discount" }, { "greaterThan": ["discount", 0] }]
+ },
+ "tieredFee": { "in": ["fee", [0, 10, 25, 50]] },
+ "closedNeedsClosedAt": {
+ "anyOf": [{ "notEqual": ["status", { "const": "closed" }] }, { "present": "closedAt" }]
}
}
```
-The comparisons are `equal`, `notEqual`, `lessThan`, `lessThanOrEqual`, `greaterThan` and `greaterThanOrEqual`, and the operators `add` and `multiply` (two or more operands) and `subtract`, `divide`, `modulo` and `power` (exactly two). A property the document leaves out counts as 0, or as the value of an `ifAbsent` operand naming it. The arithmetic is exact over 128-bit integers, and `divide` and `modulo` are Euclidean, so a remainder is never negative. The rules are fixed when the document type is created.
+The comparisons are `equal`, `notEqual`, `lessThan`, `lessThanOrEqual`, `greaterThan` and `greaterThanOrEqual`, and the operators `add` and `multiply` (two or more operands) and `subtract`, `divide`, `modulo` and `power` (exactly two). `{ "in": [expression, [values]] }` holds if the expression takes one of two or more distinct integer values. A string property (an enum, say) is compared with `{ "equal": ["status", { "const": "closed" }] }` or `notEqual`, with another string property (`{ "notEqual": ["fromCurrency", "toCurrency"] }`), or listed with `{ "in": ["status", ["open", "pending"]] }`; a constant must be one of the property's `enum` values, and a string the document leaves out equals none, unless `{ "ifAbsent": ["status", "open"] }` gives it a default. Identifier properties compare the same way, with base58 constants: `{ "equal": ["paymentToken", { "const": "" }] }`, `{ "notEqual": ["buyerId", "sellerId"] }`, or `{ "in": ["paymentToken", ["", ""]] }`. `$ownerId`, the document's owner, is an identifier operand as well (`{ "equal": ["authorId", "$ownerId"] }`), and a transfer or purchase that would break such a rule is refused. `{ "startsWith": ["url", { "const": "https://" }] }` and `endsWith` test a string property's start or end, byte for byte, against a constant or another string property. `{ "contains": ["participants", "$ownerId"] }` holds when a typed array property has an element equal to the value, looked for as the array's elements are (an integer expression, a string or an identifier), so `{ "not": { "contains": ["labels", { "const": "used" }] } }` refuses a label; the array is reported as a read of kind `elements`. `anyOf` holds if at least one of two or more conditions holds, `allOf` if every one does, `not` if its one condition does not, `{ "ifThen": [a, b] }` if `b` holds whenever `a` does (evaluating `b` only then), and `{ "ifThenElse": [a, b, c] }` if `b` holds when `a` does and `c` when it does not; `{ "notIn": [expression, [values]] }` is an `in` negated, and `min`, `max` (two or more operands) and `abs` (one) join the arithmetic; conditions are checked in order and `anyOf` stops at the first that holds, so `{ "anyOf": [{ "equal": ["b", 0] }, { "equal": [{ "divide": ["a", "b"] }, 2] }] }` never divides by zero. An operand may read an integer or a boolean property (true as 1, false as 0), or a size: `{ "length": path }` and `{ "byteLength": path }` give the characters and UTF-8 bytes of a string property, and `{ "count": path }` the items of an array or the bytes of a byte array, so `{ "lessThanOrEqual": [{ "count": "tags" }, "maxTags"] }` holds a list to its own limit (a size read is reported with kind `length` or `count`). A type that lists `$createdAt`, `$updatedAt` or `$transferredAt` (or any of them with `BlockHeight` or `CoreBlockHeight` appended) in `required` may read it too: `{ "lessThanOrEqual": [{ "subtract": ["endsAt", "$createdAt"] }, 604800000] }` keeps a listing to a week, and since a price update sets `$updatedAt` and a transfer or purchase `$transferredAt`, each is judged against the rules reading those. `{ "countOf": [type, filter] }` and `{ "sumOf": [type, property, filter] }` read a total from state, how many documents of a type of the same contract match the filter or what an integer property adds up to over them, as the type's count or sum trees keep it once the write is done: `{ "lessThanOrEqual": [{ "countOf": ["listing", { "$ownerId": "$ownerId" }] }, 10] }` on `listing` keeps every owner at ten listings or fewer. The filter maps keys of the counted type (or `$ownerId`) to values read from the document being written, and may be left out for a whole-type total; the type needs `documentsCountable` or `documentsSummable` for a whole-type total, and an index whose properties are exactly the filter's keys (countable, or summing the property) otherwise. A property the document leaves out counts as 0, or as the value of an `ifAbsent` operand naming it; `{ "present": path }` and `{ "absent": path }` tell a property left out from one set to 0, and may name a property of any type. The arithmetic is exact over 128-bit integers, and `divide` and `modulo` are Euclidean, so a remainder is never negative. The rules are fixed when the document type is created.
Consensus checks every rule on each create and replace, and rejects a document that breaks one, or whose rule overflows, divides by zero or raises to a negative power. The code reaches JS as `error.code`, and the message names the rule:
@@ -431,6 +453,62 @@ try {
}
```
+To find a broken rule before paying for a refused transition, a contract lists a document type's rules and checks a document against them with the code consensus runs. The check covers the rules alone, not the JSON schema. It reads the document's owner for `$ownerId`, and the device clock for the times the write will record (`readsSystem` lists the ones a rule reads); a rule reading a block height is not checked, since the height is unknown until the block, and neither is a rule reading a `countOf` or `sumOf` total, which only the platform reads from state (`readsTotals` lists the ones a rule reads, each with its `kind`, `documentType`, the summed `property` of a `sumOf` and the `filter` keys):
+
+```ts
+contract.documentTypePropertyConstraints('offer');
+// [{ name: 'discountBelowPrice', rule: { lessThan: ['discount', 'price'] },
+// reads: [{ path: 'discount', kind: 'value' }, { path: 'price', kind: 'value' }],
+// readsOwner: false, readsSystem: [], readsTotals: [] }, ...]
+
+const broken = contract.checkDocumentPropertyConstraints(document);
+if (broken) {
+ // { rule: 'discountBelowPrice', violation: 'NotMet', message: 'it does not hold' }
+}
+```
+
+Rules come back in name order, the order consensus checks them in; `contract.documentPropertyConstraints` maps every document type that declares rules to its list. The `PropertyConstraintCondition`, `PropertyConstraintExpression` and `PropertyConstraintEqualityOperand` types spell out the rule grammar, and `violation` is one of `NotMet`, `Overflow`, `DivisionByZero`, `NegativeExponent` or `NotAnInteger`, the reason consensus would report.
+
+## How a document type is stored (`documentTypeLayout`)
+
+`documentTypeLayout(contract, documentTypeName, platformVersion)` returns the GroveDB layout of a document type as Drive writes it: the document type tree, the documents by id and, for each index, the property and value trees down to where the index ends. Each layer carries the tree or element type Drive writes there (a count or sum tree, a ranked indexed tree, a reference, an indexOnly item), the wrapper a continuation tree gets under an aggregating value tree, the indexes that use it, and conditions such as the tree a unique index falls back to when a value is null. It runs locally with Drive's own rules, those of protocol version 14 on (an earlier version is refused), so it needs no connection:
+
+```ts
+import { documentTypeLayout, PlatformVersion } from '@dashevo/evo-sdk';
+
+const { root } = documentTypeLayout(contract, 'review', new PlatformVersion(14));
+// root.children: the documents by id ([0]) and one tree per first index property;
+// each node: { key, role, element, wrapper?, rankedAxes, indexes, notes, alternative?, children, structureNode }
+// (wrapper and alternative are left out when there is none)
+```
+
+`structureNode` names the layer of Drive's GroveDB structure description it is an instance of, as the [GroveDB structure viewer](https://dashpay.github.io/grovedb-structure-viewer/) shows it (`#/`).
+
+## What a document costs (`documentCreateCost`)
+
+`documentCreateCost(contract, documentTypeName, options, platformVersion)` returns what creating a document of a type costs, in credits (`creditsPerDash` of them make one Dash), computed locally by Drive from the contract:
+
+- `storage`: the bytes the insert writes and their fee, exact, under two scenarios: `newValues` (the first document with these index values creates their trees) and `knownValues` (a later document with the same values adds only its own entries);
+- `indexes`: per index, the bytes of the layers it shares with other indexes and of its own, so its cost on its own is `sharedBytes + ownBytes`;
+- `processing`: the signature and identity fetch (exact) and the work of the writes (estimated for `existingDocuments` stored documents);
+- `contractCharges`: the create's action fee, token cost and contest fund, when the type has them (a contested create is stored in the vote poll until the contest ends; that storage is not priced);
+- `refund`: what a delete refunds, in the same epoch and a year later;
+- `fields`: how the priced document was filled.
+
+The document is built from sizes, not values: by default each variable-size field is at the middle of its bounds and each optional field is present. Pass `fields` to change that:
+
+```ts
+import { documentCreateCost, PlatformVersion } from '@dashevo/evo-sdk';
+
+const cost = documentCreateCost(contract, 'note', {
+ fields: { text: { length: 200 }, mood: { present: false } },
+ existingDocuments: 10_000,
+}, PlatformVersion.latest());
+const dash = cost.totalCredits.newValues / cost.creditsPerDash;
+```
+
+It follows protocol version 14 on; an earlier version is refused. A type whose documents have a `ttl` is priced by lifetime, with no refund.
+
## Chained queries (provable semi-join)
A `refersTo: permanentDocument` declaration also lights up the read side: a **chained query** answers `SELECT * FROM post WHERE $id IN (SELECT postId FROM like WHERE $ownerId = me)` in one verified round trip. The node returns the inner indexOnly page and the referenced documents under ONE merged proof — a single quorum-signed state root by construction — and the SDK re-derives the outer query itself and checks it against the *proven* inner values — the node cannot substitute, omit, or inject joined documents. For a `permanentDocument` join property a missing referenced document fails verification outright, since such a reference cannot dangle. For a `deletableDocument` join property a referenced document that was deleted since is proven absent: it has no entry in `outerDocuments` (so match the two halves by id, not by position) and its id is listed in `missingOuterIds`, in first-appearance order. The node still cannot pass an existing document off as deleted.
diff --git a/packages/js-evo-sdk/package.json b/packages/js-evo-sdk/package.json
index 77535cc5dd6..b57c406fed3 100644
--- a/packages/js-evo-sdk/package.json
+++ b/packages/js-evo-sdk/package.json
@@ -1,6 +1,6 @@
{
"name": "@dashevo/evo-sdk",
- "version": "4.2.0-beta.4",
+ "version": "4.2.0-beta.6",
"type": "module",
"main": "./dist/evo-sdk.module.js",
"types": "./dist/sdk.d.ts",
diff --git a/packages/js-evo-sdk/src/documents/facade.ts b/packages/js-evo-sdk/src/documents/facade.ts
index 1910b77058b..2a34c3afa37 100644
--- a/packages/js-evo-sdk/src/documents/facade.ts
+++ b/packages/js-evo-sdk/src/documents/facade.ts
@@ -104,18 +104,43 @@ export class DocumentsFacade {
* Creates a document and resolves to the confirmed Document as Platform
* committed it, consensus-populated system fields included — keep this
* instance when you later intend to delete an indexOnly document whose
- * type requires `$createdAt`.
+ * type requires `$createdAt`. A document of a contested index joins a
+ * contest: `options.contestFund` is the most, in credits, it pays into it.
+ * For an indexOnly type the proof shows the document's entry at the proof's
+ * block, not that this create wrote it: no stronger proof exists for one.
*/
async create(options: wasm.DocumentCreateOptions): Promise {
const w = await this.sdk.getWasmSdkConnected();
return w.documentCreate(options);
}
+ /**
+ * The prefunded voting balance a create of `document` states to join the
+ * contest it enters (a DPNS name, a moderation charter): the contested index
+ * and the fund to join it now, which from protocol version 14 doubles once
+ * the contest holds 250 contenders and again for every 50 more. Undefined
+ * when the document joins no contest. {@link create} states it itself; a
+ * transition built by hand passes it as `prefundedVotingBalance` to
+ * `new DocumentCreateTransition`. Its `credits` alone is what the Rust SDK's
+ * `contest_fund_to_join` returns; this is its `prefunded_voting_balance_to_join`.
+ */
+ async contestFundToJoin(
+ document: wasm.Document,
+ ): Promise {
+ const w = await this.sdk.getWasmSdkConnected();
+ return w.getContestFundToJoin(document);
+ }
+
async replace(options: wasm.DocumentReplaceOptions): Promise {
const w = await this.sdk.getWasmSdkConnected();
return w.documentReplace(options);
}
+ /**
+ * Deletes a document and resolves once the proof shows it gone. For an
+ * indexOnly type the proof shows the document's entry gone at the proof's
+ * block, not that this delete removed it: no stronger proof exists for one.
+ */
async delete(options: wasm.DocumentDeleteOptions): Promise {
const w = await this.sdk.getWasmSdkConnected();
return w.documentDelete(options);
diff --git a/packages/js-evo-sdk/src/dpns/facade.ts b/packages/js-evo-sdk/src/dpns/facade.ts
index 0d0a80f1f99..4bc4f49abb6 100644
--- a/packages/js-evo-sdk/src/dpns/facade.ts
+++ b/packages/js-evo-sdk/src/dpns/facade.ts
@@ -33,6 +33,12 @@ export class DpnsFacade {
return w.dpnsResolveName(name);
}
+ /**
+ * Registers a DPNS name. A contested name joins a contest: pass
+ * `options.contestFund` as the most, in credits, the registration pays into
+ * it, or leave it out to state the fund to join read just before the domain
+ * is submitted.
+ */
async registerName(options: wasm.DpnsRegisterNameOptions): Promise {
const w = await this.sdk.getWasmSdkConnected();
return w.dpnsRegisterName(options);
diff --git a/packages/js-evo-sdk/src/voting/facade.ts b/packages/js-evo-sdk/src/voting/facade.ts
index 1626578e90b..eb3f52ec4ca 100644
--- a/packages/js-evo-sdk/src/voting/facade.ts
+++ b/packages/js-evo-sdk/src/voting/facade.ts
@@ -53,7 +53,7 @@ export class VotingFacade {
return w.getVotePollsByEndDateWithProofInfo(query);
}
- async masternodeVote(options: wasm.MasternodeVoteOptions): Promise {
+ async masternodeVote(options: wasm.MasternodeVoteOptions): Promise {
const w = await this.sdk.getWasmSdkConnected();
return w.masternodeVote(options);
}
diff --git a/packages/js-evo-sdk/tests/unit/facades/documents.spec.ts b/packages/js-evo-sdk/tests/unit/facades/documents.spec.ts
index f8ba44ca385..6dca0d6ebb8 100644
--- a/packages/js-evo-sdk/tests/unit/facades/documents.spec.ts
+++ b/packages/js-evo-sdk/tests/unit/facades/documents.spec.ts
@@ -24,6 +24,7 @@ describe('DocumentsFacade', () => {
let getDocumentStub: SinonStub;
let getDocumentWithProofInfoStub: SinonStub;
let documentCreateStub: SinonStub;
+ let getContestFundToJoinStub: SinonStub;
let documentReplaceStub: SinonStub;
let documentDeleteStub: SinonStub;
let documentTransferStub: SinonStub;
@@ -90,6 +91,7 @@ describe('DocumentsFacade', () => {
// Stub transition methods
documentCreateStub = this.sinon.stub(wasmSdk, 'documentCreate').resolves();
+ getContestFundToJoinStub = this.sinon.stub(wasmSdk, 'getContestFundToJoin').resolves(undefined);
documentReplaceStub = this.sinon.stub(wasmSdk, 'documentReplace').resolves();
documentDeleteStub = this.sinon.stub(wasmSdk, 'documentDelete').resolves();
documentTransferStub = this.sinon.stub(wasmSdk, 'documentTransfer').resolves();
@@ -232,6 +234,28 @@ describe('DocumentsFacade', () => {
});
});
+ describe('contestFundToJoin()', () => {
+ it('should resolve to the prefunded voting balance the create states', async () => {
+ const prefundedVotingBalance = new wasmSDKPackage.PrefundedVotingBalance({
+ indexName: 'parentNameAndLabel',
+ credits: BigInt(20000000000),
+ });
+ getContestFundToJoinStub.resolves(prefundedVotingBalance);
+
+ const result = await client.documents.contestFundToJoin(document);
+
+ expect(getContestFundToJoinStub).to.be.calledOnceWithExactly(document);
+ expect(result).to.equal(prefundedVotingBalance);
+ });
+
+ it('should resolve to undefined for a document that joins no contest', async () => {
+ const result = await client.documents.contestFundToJoin(document);
+
+ expect(getContestFundToJoinStub).to.be.calledOnceWithExactly(document);
+ expect(result).to.be.undefined();
+ });
+ });
+
describe('replace()', () => {
it('should replace an existing document', async () => {
const options = {
diff --git a/packages/js-evo-sdk/tests/unit/facades/voting.spec.ts b/packages/js-evo-sdk/tests/unit/facades/voting.spec.ts
index d3fed61c549..6c458327185 100644
--- a/packages/js-evo-sdk/tests/unit/facades/voting.spec.ts
+++ b/packages/js-evo-sdk/tests/unit/facades/voting.spec.ts
@@ -21,6 +21,7 @@ describe('VotingFacade', () => {
let getVotePollsByEndDateStub: SinonStub;
let getVotePollsByEndDateWithProofInfoStub: SinonStub;
let masternodeVoteStub: SinonStub;
+ let recordedVote: wasmSDKPackage.Vote;
beforeEach(async function setup() {
await init();
@@ -60,9 +61,8 @@ describe('VotingFacade', () => {
});
// Stub transition method
- masternodeVoteStub = this.sinon.stub(wasmSdk, 'masternodeVote').resolves({
- success: true,
- });
+ recordedVote = Object.create(wasmSDKPackage.Vote.prototype);
+ masternodeVoteStub = this.sinon.stub(wasmSdk, 'masternodeVote').resolves(recordedVote);
});
describe('contestedResourceVoteState()', () => {
@@ -167,7 +167,7 @@ describe('VotingFacade', () => {
});
describe('masternodeVote()', () => {
- it('should cast a vote on a contested resource', async () => {
+ it('should cast a vote on a contested resource and return the recorded vote', async () => {
const options = {
masternodeProTxHash,
contractId: dataContractId,
@@ -178,9 +178,10 @@ describe('VotingFacade', () => {
signer,
};
- await client.voting.masternodeVote(options);
+ const vote = await client.voting.masternodeVote(options);
expect(masternodeVoteStub).to.be.calledOnceWithExactly(options);
+ expect(vote).to.equal(recordedVote);
});
it('should support abstain vote choice', async () => {
diff --git a/packages/js-grpc-common/package.json b/packages/js-grpc-common/package.json
index 24000661775..c85e7cdc7a9 100644
--- a/packages/js-grpc-common/package.json
+++ b/packages/js-grpc-common/package.json
@@ -1,6 +1,6 @@
{
"name": "@dashevo/grpc-common",
- "version": "4.2.0-beta.4",
+ "version": "4.2.0-beta.6",
"description": "Common GRPC library",
"main": "index.js",
"scripts": {
diff --git a/packages/keyword-search-contract/package.json b/packages/keyword-search-contract/package.json
index 4e61a9c2ec4..b92a6c8bb61 100644
--- a/packages/keyword-search-contract/package.json
+++ b/packages/keyword-search-contract/package.json
@@ -1,6 +1,6 @@
{
"name": "@dashevo/keyword-search-contract",
- "version": "4.2.0-beta.4",
+ "version": "4.2.0-beta.6",
"description": "A contract that allows searching for contracts",
"scripts": {
"lint": "eslint .",
diff --git a/packages/kotlin-sdk/CLAUDE.md b/packages/kotlin-sdk/CLAUDE.md
index 68c2ff5a77b..ad7393bc35f 100644
--- a/packages/kotlin-sdk/CLAUDE.md
+++ b/packages/kotlin-sdk/CLAUDE.md
@@ -32,9 +32,9 @@ are data encrypted under Keystore-wrapped AES keys).
`extern "C"` entry points of the FFI crates **as rlib dependencies**, so
`DashSDKResult` never crosses JNI by value. Errors throw
`org.dashfoundation.dashsdk.ffi.DashSDKException(code, message)`, which
- also carries the consensus code and kind when a platform-wallet result
- reports a consensus rejection (`DashSdkError.consensusError`; branch on
- that, never on the message); panics
+ also carries the consensus code and kind when a platform-wallet result or
+ an rs-sdk-ffi `DashSDKError` reports a consensus rejection
+ (`DashSdkError.consensusError`; branch on that, never on the message); panics
are caught at every export (`support::guard`) — the JNI library must never
abort the app process (workspace profiles `*-android` keep
`panic = "unwind"`).
@@ -46,6 +46,11 @@ are data encrypted under Keystore-wrapped AES keys).
`GlobalRef`s.
- `PlatformWalletManager` is network-locked at construction. Network switch =
destroy + new instance (`WalletManagerStore`), never reconfiguration.
+- Screens read Room `Flow`s or snapshot data copied at the JNI boundary. They
+ never hold a native handle wrapper in composition: `NativeCleaner` can free
+ it in the middle of a read.
+- Bridge an FFI function only when the reference Swift app has a caller for
+ it.
## Building
@@ -84,6 +89,13 @@ export DASH_GRADLE_BUILD_ROOT=/Volumes/DashBuild/gradle-build
- Instrumented tests (`sdk/src/androidTest`): FFI smoke test
(`FfiSmokeTest`) is the A-M1 gate — library loads, version resolves, SDK
handle round-trips.
+- JVM and Robolectric tests never load `libdash_sdk_jni`. JNI symbol names,
+ the hand-written method descriptors in `rs-unified-sdk-jni` and argument
+ marshaling are exercised only by the instrumented tests, and a mismatch
+ fails at runtime, not at compile time. Change a Rust descriptor and its
+ Kotlin signature in the same commit, and extend `FfiSmokeTest`
+ (`persistenceBridgeDescriptorsAllResolve`) and `WalletManagerRoundTripTest`
+ when adding a persistence or callback slot.
- Testnet integration tests are tagged and opt-in (`-Ptestnet=true`).
## Keeping parity with iOS
@@ -92,3 +104,17 @@ The reference implementation is `packages/swift-sdk` + its SwiftExampleApp.
When porting behavior, cite the Swift source file in the KDoc. Reuse iOS
accessibility identifier strings verbatim as Compose `testTag`s for
cross-platform UAT parity.
+
+Parity status lives in `docs/sdk/sdk-parity-manifest.json`. After a
+capability changes, edit the manifest and run
+`python3 scripts/check_sdk_parity_manifest.py --write-summary`; CI rejects a
+stale `PARITY_SUMMARY.md`. Never hand-edit counts in `PARITY.md` or
+`PARITY_SUMMARY.md`. The rules the manifest encodes:
+
+- A capability is `supported` only when it names an automated test or a
+ recorded device or manual gate.
+- Recovery is part of parity: where durable state or on-chain value exists, a
+ flow is ported only when it resumes after process death, so its `restart`
+ must be `tested`.
+- Differences in visual layout are not parity failures when the capability,
+ accessibility identifiers and behavior match.
diff --git a/packages/kotlin-sdk/KotlinExampleApp/TEST_PLAN.md b/packages/kotlin-sdk/KotlinExampleApp/TEST_PLAN.md
index 13784fd4003..1cb02d3fa80 100644
--- a/packages/kotlin-sdk/KotlinExampleApp/TEST_PLAN.md
+++ b/packages/kotlin-sdk/KotlinExampleApp/TEST_PLAN.md
@@ -121,7 +121,7 @@ Most Platform actions have hard preconditions. Establish these fixtures before s
| ID | Action | Layer | Tier | Status | Tags | Entry point & test notes |
|---|---|---|---|---|---|---|
-| ID-01 | Create identity (Core-funded asset lock) | Cross | Essential | ✅ | | `CreateIdentityScreen` / `IdentityRegistrationController` → `platform_wallet_register_identity_with_signer`. New identity + credit balance appear. |
+| ID-01 | Create identity (Core-funded asset lock) | Cross | Essential | ✅ | | `CreateIdentityScreen` / `IdentityRegistrationController` → `platform_wallet_register_identity_with_signer`. New identity + credit balance appear. After a fresh six-key registration, fetch the identity and check that keys 4 and 5 are ECDSA ENCRYPTION / DECRYPTION MEDIUM keys bound to the DashPay contract id and `contactRequest` (the `RegistrationKeys` table). A successful Add Contact does not prove the bounds; no unit test can see what the node stored. |
| ID-02 | Load / discover identity from wallet | Platform | Essential | ✅ | | `LoadIdentityScreen` / `SearchWalletsForIdentitiesScreen` → `platform_wallet_discover_identities`. |
| ID-03 | View identity (info / balance / revision / keys) | Platform | Essential | ✅ | | `IdentityDetailScreen`, `KeysListScreen`, `KeyDetailScreen`. |
| ID-04 | Transfer credits identity → identity | Platform | Essential | ✅ | | `IdentityDetailScreen` → **Transfer Credits** (dialog, `TransferCreditsScreen`) → `platform_wallet_transfer_credits_with_signer` (Keystore-signed). Recipient entered via `RecipientPicker` (local identity / paste base58 id / DPNS name). |
@@ -137,6 +137,7 @@ Most Platform actions have hard preconditions. Establish these fixtures before s
| ID-14 | Credit transfer between two on-device identities (A → B) | Platform | Thorough | ✅ | multiwallet | `IdentityDetailScreen` → **Transfer Credits** (`ID-04`), recipient = wallet B's identity (via `RecipientPicker`). Switch to B; verify credit balance rose. |
| ID-15 | Same identity restored into two wallets (duplicate seed) | Platform | Uncommon | ✅ | multiwallet | Importing the same mnemonic as a second wallet derives the **same** identity; verify consistency. |
| ID-16 | Resume identity top-up from a tracked asset lock | Cross | Manual | 🚧 | | Source UI selects the Rust-tracked exact outpoint, excludes locks bound to another identity, and reaches the compiled tracked-lock list/free and existing-lock registration/top-up JNI bridges. Device gate: interrupt after Core broadcast, restart, resume the same Built transaction/outpoint, and verify foreign/untracked/consumed locks return typed failures without creating a replacement funding transaction. |
+| ID-17 | Signing after biometric re-enrollment | Platform | Manual | ✅ | | Physical device with auth-gated Keystore keys. Re-enroll biometrics (add or remove a fingerprint or face), then sign any identity transition: expect `SigningKeyUnavailable` (code 31). Repair the key from the wallet's key-health sheet (`WalletKeyHealthSheet` → `PlatformWalletManager.repairIdentityKey`); the next sign succeeds. The CI emulator cannot re-enroll biometrics, so the invalidation recovery is pinned only through the fake Keystore at the unit tier. |
### 4.3 Platform Addresses (DIP-17 credit addresses) — `Domain=Address`
@@ -333,7 +334,7 @@ Membership of each feature category across **all** sections (primary section mem
- **Document** — `DOC-01..15`
- **Token** — `TOK-01..20`
- **Shielded** — `SH-01..17`
-- **DashPay** — `DP-01..19` (`DP-12..19` = invitation create, claim, persistence, reclaim; funded evidence 2026-07-23 in `docs/dashpay/KOTLIN_INVITATIONS_SPEC.md` §7)
+- **DashPay** — `DP-01..19` (`DP-12..19` = invitation create, claim, persistence, reclaim; funded run 2026-07-23)
- **System / Diagnostics** — `SYS-01..08`
### Tag index
diff --git a/packages/kotlin-sdk/KotlinExampleApp/app/src/main/java/org/dashfoundation/example/ui/contracts/CreateDocumentScreen.kt b/packages/kotlin-sdk/KotlinExampleApp/app/src/main/java/org/dashfoundation/example/ui/contracts/CreateDocumentScreen.kt
index 137a699e8c7..4f8d025f79c 100644
--- a/packages/kotlin-sdk/KotlinExampleApp/app/src/main/java/org/dashfoundation/example/ui/contracts/CreateDocumentScreen.kt
+++ b/packages/kotlin-sdk/KotlinExampleApp/app/src/main/java/org/dashfoundation/example/ui/contracts/CreateDocumentScreen.kt
@@ -1,5 +1,6 @@
package org.dashfoundation.example.ui.contracts
+import android.util.Log
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
@@ -48,6 +49,7 @@ import kotlinx.serialization.json.jsonObject
import kotlinx.serialization.json.put
import kotlinx.serialization.json.putJsonArray
import org.dashfoundation.dashsdk.persistence.entities.IdentityEntity
+import org.dashfoundation.dashsdk.queries.PropertyConstraintViolation
import org.dashfoundation.example.di.LocalAppContainer
import org.dashfoundation.example.di.LocalAppState
import org.dashfoundation.example.ui.components.AccessiblePicker
@@ -88,6 +90,13 @@ import org.dashfoundation.example.util.truncateMiddle
* decodes them to native bytes. The confirmed canonical JSON the FFI
* returns is shown on success — its `$id` is the on-chain document id a
* DOC-01 browse (chain query) then surfaces.
+ *
+ * Before broadcasting, a document whose type declares `propertyConstraints`
+ * (protocol version 14) is judged by Rust against those rules
+ * (`sdk.contracts.checkPropertyConstraints`, the check consensus runs); a
+ * broken rule is shown by name, violation and reason and nothing is sent. A
+ * check that cannot run (no SDK, no stored contract serialization) is logged
+ * and the create goes ahead, as on iOS.
*/
@OptIn(ExperimentalMaterial3Api::class)
@Composable
@@ -99,6 +108,7 @@ fun CreateDocumentScreen(
val container = LocalAppContainer.current
val appState = LocalAppState.current
val scope = rememberCoroutineScope()
+ val sdk by appState.sdk.collectAsStateWithLifecycle()
val network by appState.currentNetwork.collectAsStateWithLifecycle()
val manager by container.walletManagerStore.activeManager.collectAsStateWithLifecycle()
@@ -129,6 +139,8 @@ fun CreateDocumentScreen(
var isSubmitting by remember { mutableStateOf(false) }
var error by remember { mutableStateOf(null) }
+ // The broken propertyConstraints rule that stopped the last submit.
+ var constraintError by remember { mutableStateOf(null) }
var createdDocId by remember { mutableStateOf(null) }
var createdJson by remember { mutableStateOf(null) }
@@ -283,6 +295,42 @@ fun CreateDocumentScreen(
isSubmitting = true
scope.launch {
try {
+ // Protocol version 14: consensus refuses a document
+ // breaking one of its type's propertyConstraints rules
+ // (error 10422) and still charges for the transition,
+ // so judge it first with the same Rust check.
+ val activeSdk = sdk
+ val check: (suspend (ByteArray) -> PropertyConstraintViolation?)? =
+ if (activeSdk == null) {
+ null
+ } else {
+ { bytes ->
+ activeSdk.contracts.checkPropertyConstraints(
+ serializedContract = bytes,
+ documentType = typeName,
+ propertiesJson = propertiesJson,
+ ownerId = ownerId.identityId,
+ )
+ }
+ }
+ when (
+ val preCheck = propertyConstraintPreCheck(
+ schema,
+ contract?.binarySerialization,
+ check,
+ )
+ ) {
+ is PropertyConstraintPreCheck.Broken -> {
+ constraintError = propertyConstraintViolationAlert(preCheck.violation)
+ return@launch
+ }
+ // Consensus judges the document either way.
+ is PropertyConstraintPreCheck.Skipped -> Log.w(
+ TAG,
+ "propertyConstraints pre-check could not run: ${preCheck.reason}",
+ )
+ PropertyConstraintPreCheck.Passed -> Unit
+ }
val json = mgr.documentTransactions.create(
walletHandle = wallet.handle,
ownerId = ownerId.identityId,
@@ -304,8 +352,15 @@ fun CreateDocumentScreen(
}
ErrorAlertDialog(message = error, onDismiss = { error = null })
+ ErrorAlertDialog(
+ message = constraintError,
+ title = PROPERTY_CONSTRAINT_BROKEN_TITLE,
+ onDismiss = { constraintError = null },
+ )
}
+private const val TAG = "CreateDocumentScreen"
+
/**
* One schema-property editor, dispatched on the JSON-schema `type`. Shared
* by the create form and the DOC-03 replace form ([DocumentActionsScreen]),
diff --git a/packages/kotlin-sdk/KotlinExampleApp/app/src/main/java/org/dashfoundation/example/ui/contracts/DocumentTypeDetailsScreen.kt b/packages/kotlin-sdk/KotlinExampleApp/app/src/main/java/org/dashfoundation/example/ui/contracts/DocumentTypeDetailsScreen.kt
index 77dde84388f..e76d0dc928d 100644
--- a/packages/kotlin-sdk/KotlinExampleApp/app/src/main/java/org/dashfoundation/example/ui/contracts/DocumentTypeDetailsScreen.kt
+++ b/packages/kotlin-sdk/KotlinExampleApp/app/src/main/java/org/dashfoundation/example/ui/contracts/DocumentTypeDetailsScreen.kt
@@ -1,6 +1,7 @@
package org.dashfoundation.example.ui.contracts
import androidx.compose.foundation.clickable
+import androidx.compose.foundation.horizontalScroll
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
@@ -22,6 +23,7 @@ import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.material3.TopAppBar
import androidx.compose.runtime.Composable
+import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
@@ -29,12 +31,15 @@ import androidx.compose.runtime.saveable.rememberSaveable
import androidx.compose.runtime.setValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.testTag
+import androidx.compose.ui.text.font.FontFamily
import androidx.compose.ui.unit.dp
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import androidx.navigation.NavHostController
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
+import org.dashfoundation.dashsdk.queries.DocumentPropertyConstraint
import org.dashfoundation.example.di.LocalAppContainer
+import org.dashfoundation.example.di.LocalAppState
import org.dashfoundation.example.navigation.CountDocuments
import org.dashfoundation.example.navigation.Documents
import org.dashfoundation.example.navigation.NewDocument
@@ -52,6 +57,12 @@ import org.dashfoundation.example.util.hexToBytes
* The "New Document" affordance broadcasts a real create state transition
* via `CreateDocumentScreen` (port of iOS `CreateDocumentView`); the query
* actions (browse / count / sum-average) live alongside it.
+ *
+ * The "Property Constraints" section (protocol version 14) lists the rules
+ * Rust reads from the contract's stored platform serialization
+ * (`sdk.contracts.propertyConstraints`), re-read when the SDK learns the
+ * network's protocol version: the rules are read at that version, and none
+ * exist below 14.
*/
@OptIn(ExperimentalMaterial3Api::class)
@Composable
@@ -61,6 +72,9 @@ fun DocumentTypeDetailsScreen(
navController: NavHostController,
) {
val container = LocalAppContainer.current
+ val appState = LocalAppState.current
+ val sdk by appState.sdk.collectAsStateWithLifecycle()
+ val protocolVersion by appState.platformProtocolVersion.collectAsStateWithLifecycle()
val contractId = remember(contractIdHex) { contractIdHex.hexToBytes() }
val contractFlow = remember(contractIdHex) {
@@ -69,6 +83,9 @@ fun DocumentTypeDetailsScreen(
val contract by contractFlow.collectAsStateWithLifecycle(initialValue = null)
var expandedIndices by rememberSaveable { mutableStateOf(setOf()) }
+ var constraintsSection by remember(contractIdHex, typeName) {
+ mutableStateOf(PropertyConstraintsSection.Hidden)
+ }
Scaffold(
topBar = {
@@ -92,6 +109,18 @@ fun DocumentTypeDetailsScreen(
}
val capabilities = documentTypeCapabilities(schema, contractConfig)
+ LaunchedEffect(sdk, protocolVersion, current.lastUpdated, typeName) {
+ val activeSdk = sdk
+ val read: (suspend (ByteArray) -> List)? =
+ if (activeSdk == null) {
+ null
+ } else {
+ { bytes -> activeSdk.contracts.propertyConstraints(bytes, typeName) }
+ }
+ constraintsSection =
+ loadPropertyConstraintsSection(schema, current.binarySerialization, read)
+ }
+
val properties = schema.objectField("properties") ?: JsonObject(emptyMap())
val indices = schema.arrayField("indices")?.mapNotNull { it as? JsonObject }.orEmpty()
val required = schema.arrayField("required")
@@ -195,6 +224,8 @@ fun DocumentTypeDetailsScreen(
}
}
+ PropertyConstraintsFormSection(constraintsSection)
+
if (indices.isNotEmpty()) {
FormSection(title = "Indices (${indices.size})") {
indices.sortedBy { it.stringField("name").orEmpty() }.forEach { index ->
@@ -321,6 +352,140 @@ fun DocumentTypeDetailsScreen(
}
}
+/**
+ * The protocol-version-14 `propertyConstraints` rules: named conditions every
+ * created or replaced document must meet, checked in name order. Rust reads
+ * them from the stored contract; this section only shows what it reports
+ * (← `propertyConstraintsSection` in DocumentTypeDetailsView.swift).
+ */
+@Composable
+private fun PropertyConstraintsFormSection(section: PropertyConstraintsSection) {
+ when (section) {
+ PropertyConstraintsSection.Hidden -> Unit
+
+ is PropertyConstraintsSection.Rules -> FormSection(
+ title = "Property Constraints (${section.rules.size})",
+ modifier = Modifier.testTag("documentType.propertyConstraints"),
+ ) {
+ section.rules.forEach { rule -> PropertyConstraintRow(rule) }
+ Text(
+ "Every created or replaced document must meet each rule, checked in name " +
+ "order. A document breaking one is refused (error 10422) and the fee " +
+ "is still charged.",
+ style = MaterialTheme.typography.bodySmall,
+ color = MaterialTheme.colorScheme.onSurfaceVariant,
+ )
+ }
+
+ PropertyConstraintsSection.NotEnforced -> FormSection(
+ title = "Property Constraints",
+ modifier = Modifier.testTag("documentType.propertyConstraints"),
+ ) {
+ Text(
+ "The schema declares propertyConstraints, but the network's protocol " +
+ "version does not enforce them (they take effect at protocol version 14).",
+ style = MaterialTheme.typography.bodySmall,
+ color = MaterialTheme.colorScheme.onSurfaceVariant,
+ )
+ }
+
+ is PropertyConstraintsSection.Unavailable -> FormSection(
+ title = "Property Constraints",
+ modifier = Modifier.testTag("documentType.propertyConstraints"),
+ ) {
+ Text(
+ section.reason,
+ style = MaterialTheme.typography.bodySmall,
+ color = MaterialTheme.colorScheme.error,
+ )
+ }
+ }
+}
+
+/**
+ * One `propertyConstraints` rule: its name, the rule as declared, what it
+ * reads, the system times and heights and the totals it reads, and whether an owner change
+ * is judged against it too
+ * (← `PropertyConstraintRowView` in DocumentTypeDetailsView.swift).
+ */
+@Composable
+private fun PropertyConstraintRow(rule: DocumentPropertyConstraint) {
+ val prettyRule = remember(rule) { rule.prettyRuleJson }
+ val reads = remember(rule) { propertyConstraintReadsText(rule) }
+ val systemReads = remember(rule) { propertyConstraintSystemReadsText(rule) }
+ val totals = remember(rule) { propertyConstraintTotalsText(rule) }
+ Column(
+ modifier = Modifier
+ .fillMaxWidth()
+ .padding(vertical = 6.dp)
+ .testTag("documentType.propertyConstraint.${rule.name}"),
+ ) {
+ Row(
+ modifier = Modifier.fillMaxWidth(),
+ horizontalArrangement = Arrangement.SpaceBetween,
+ ) {
+ Text(rule.name, style = MaterialTheme.typography.titleSmall)
+ if (rule.readsOwner) {
+ Text(
+ "\$ownerId",
+ style = MaterialTheme.typography.labelSmall,
+ color = MaterialTheme.colorScheme.tertiary,
+ )
+ }
+ }
+ Text(
+ prettyRule,
+ style = MaterialTheme.typography.bodySmall.copy(fontFamily = FontFamily.Monospace),
+ softWrap = false,
+ modifier = Modifier
+ .fillMaxWidth()
+ .horizontalScroll(rememberScrollState())
+ .padding(vertical = 4.dp),
+ )
+ if (reads.isNotEmpty()) {
+ Text(
+ "Reads: $reads",
+ style = MaterialTheme.typography.bodySmall,
+ color = MaterialTheme.colorScheme.onSurfaceVariant,
+ )
+ }
+ if (systemReads != null) {
+ Text(
+ systemReads,
+ style = MaterialTheme.typography.labelSmall,
+ color = MaterialTheme.colorScheme.tertiary,
+ modifier = Modifier.testTag("documentType.propertyConstraint.${rule.name}.readsSystem"),
+ )
+ Text(
+ PROPERTY_CONSTRAINT_SYSTEM_READS_NOTE,
+ style = MaterialTheme.typography.labelSmall,
+ color = MaterialTheme.colorScheme.tertiary,
+ )
+ }
+ if (totals != null) {
+ Text(
+ totals,
+ style = MaterialTheme.typography.labelSmall,
+ color = MaterialTheme.colorScheme.tertiary,
+ modifier = Modifier.testTag("documentType.propertyConstraint.${rule.name}.readsTotals"),
+ )
+ Text(
+ PROPERTY_CONSTRAINT_TOTALS_NOTE,
+ style = MaterialTheme.typography.labelSmall,
+ color = MaterialTheme.colorScheme.tertiary,
+ )
+ }
+ if (rule.readsOwner) {
+ Text(
+ "Reads \$ownerId, the document's owner: transfers and purchases are judged " +
+ "against this rule too.",
+ style = MaterialTheme.typography.labelSmall,
+ color = MaterialTheme.colorScheme.tertiary,
+ )
+ }
+ }
+}
+
/** One schema property row (← `PropertyRowView` in DocumentTypeDetailsView.swift). */
@Composable
private fun PropertyRow(name: String, property: JsonObject, isRequired: Boolean) {
diff --git a/packages/kotlin-sdk/KotlinExampleApp/app/src/main/java/org/dashfoundation/example/ui/contracts/PropertyConstraints.kt b/packages/kotlin-sdk/KotlinExampleApp/app/src/main/java/org/dashfoundation/example/ui/contracts/PropertyConstraints.kt
new file mode 100644
index 00000000000..afca1f27267
--- /dev/null
+++ b/packages/kotlin-sdk/KotlinExampleApp/app/src/main/java/org/dashfoundation/example/ui/contracts/PropertyConstraints.kt
@@ -0,0 +1,190 @@
+package org.dashfoundation.example.ui.contracts
+
+import kotlinx.coroutines.CancellationException
+import kotlinx.serialization.json.JsonObject
+import org.dashfoundation.dashsdk.queries.DocumentPropertyConstraint
+import org.dashfoundation.dashsdk.queries.PropertyConstraintTotalRead
+import org.dashfoundation.dashsdk.queries.PropertyConstraintViolation
+
+// Screen-side glue for a document type's `propertyConstraints` rules
+// (protocol version 14): named conditions every created or replaced document
+// must meet, refused by consensus with error 10422 while the fee is still
+// charged. Rust parses and judges the rules (`sdk.contracts.propertyConstraints`
+// and `sdk.contracts.checkPropertyConstraints`); nothing here evaluates one.
+// Port of the SwiftExampleApp's `DocumentTypeDetailsView` section and
+// `CreateDocumentView` pre-check.
+
+/**
+ * Whether [schema] carries the `propertyConstraints` keyword. Says nothing
+ * about the rules themselves: those are read by Rust.
+ */
+internal fun documentTypeDeclaresPropertyConstraints(schema: JsonObject?): Boolean =
+ schema?.get("propertyConstraints") != null
+
+/** What the document type screen shows for its rules. */
+internal sealed interface PropertyConstraintsSection {
+ /** Nothing: the schema declares no rules. */
+ data object Hidden : PropertyConstraintsSection
+
+ /** The rules Rust read, in name order (the order consensus checks them in). */
+ data class Rules(val rules: List) : PropertyConstraintsSection
+
+ /**
+ * The schema declares rules, but the SDK's protocol version enforces none
+ * (they take effect at protocol version 14).
+ */
+ data object NotEnforced : PropertyConstraintsSection
+
+ /** The rules could not be read, with why. */
+ data class Unavailable(val reason: String) : PropertyConstraintsSection
+}
+
+/**
+ * Read the rules of a document type with [schema] from its contract's stored
+ * platform serialization, [serializedContract], through [read] (null when no
+ * SDK is connected).
+ */
+internal suspend fun loadPropertyConstraintsSection(
+ schema: JsonObject?,
+ serializedContract: ByteArray?,
+ read: (suspend (serializedContract: ByteArray) -> List)?,
+): PropertyConstraintsSection {
+ if (!documentTypeDeclaresPropertyConstraints(schema)) return PropertyConstraintsSection.Hidden
+ if (read == null) {
+ return PropertyConstraintsSection.Unavailable(
+ "Connect to a network to read the property constraints.",
+ )
+ }
+ if (serializedContract == null || serializedContract.isEmpty()) {
+ return PropertyConstraintsSection.Unavailable(
+ "The data contract has no stored serialization; download it again to read " +
+ "the property constraints.",
+ )
+ }
+ return try {
+ val rules = read(serializedContract)
+ if (rules.isEmpty()) {
+ PropertyConstraintsSection.NotEnforced
+ } else {
+ PropertyConstraintsSection.Rules(rules)
+ }
+ } catch (e: CancellationException) {
+ throw e
+ } catch (e: Exception) {
+ PropertyConstraintsSection.Unavailable("Could not read the property constraints: ${e.message}")
+ } catch (_: UnsatisfiedLinkError) {
+ PropertyConstraintsSection.Unavailable(NATIVE_LIBRARY_PREDATES_RULES)
+ }
+}
+
+/** The outcome of checking a document's rules before it is broadcast. */
+internal sealed interface PropertyConstraintPreCheck {
+ /** The document meets every rule, or its type declares none: send it. */
+ data object Passed : PropertyConstraintPreCheck
+
+ /** The document breaks [violation]'s rule: do not send it. */
+ data class Broken(val violation: PropertyConstraintViolation) : PropertyConstraintPreCheck
+
+ /**
+ * The check could not run, for [reason]. The create goes ahead: consensus
+ * judges the document either way.
+ */
+ data class Skipped(val reason: String) : PropertyConstraintPreCheck
+}
+
+/**
+ * Judge a document to create, of a type with [schema], against its rules
+ * through [check] (null when no SDK is connected), reading the contract's
+ * stored platform serialization [serializedContract].
+ */
+internal suspend fun propertyConstraintPreCheck(
+ schema: JsonObject?,
+ serializedContract: ByteArray?,
+ check: (suspend (serializedContract: ByteArray) -> PropertyConstraintViolation?)?,
+): PropertyConstraintPreCheck {
+ if (!documentTypeDeclaresPropertyConstraints(schema)) return PropertyConstraintPreCheck.Passed
+ if (check == null) return PropertyConstraintPreCheck.Skipped("no SDK is connected")
+ if (serializedContract == null || serializedContract.isEmpty()) {
+ return PropertyConstraintPreCheck.Skipped("the data contract has no stored serialization")
+ }
+ return try {
+ check(serializedContract)
+ ?.let { PropertyConstraintPreCheck.Broken(it) }
+ ?: PropertyConstraintPreCheck.Passed
+ } catch (e: CancellationException) {
+ throw e
+ } catch (e: Exception) {
+ PropertyConstraintPreCheck.Skipped(e.message ?: e.toString())
+ } catch (_: UnsatisfiedLinkError) {
+ PropertyConstraintPreCheck.Skipped(NATIVE_LIBRARY_PREDATES_RULES)
+ }
+}
+
+/**
+ * Why nothing could be read or checked when the bundled native library was
+ * built before the two `propertyConstraints` exports: a stale `.so` must not
+ * crash the screen or block a create.
+ */
+internal const val NATIVE_LIBRARY_PREDATES_RULES =
+ "The native library predates property constraints; rebuild it to read them."
+
+/** Each property [rule] reads with how it reads it, repeats dropped: `price (value), fee (value)`. */
+internal fun propertyConstraintReadsText(rule: DocumentPropertyConstraint): String =
+ rule.reads.distinct().joinToString(", ") { "${it.path} (${it.kind.name})" }
+
+/**
+ * The system times and heights [rule] reads, repeats dropped, as the line
+ * shown under it: `Reads $createdAt, $updatedAt`. `null` for a rule reading
+ * none.
+ */
+internal fun propertyConstraintSystemReadsText(rule: DocumentPropertyConstraint): String? =
+ rule.readsSystem.distinct().takeIf { it.isNotEmpty() }?.joinToString(", ", prefix = "Reads ")
+
+/**
+ * The note under a rule reading a system time or height
+ * ([propertyConstraintSystemReadsText] not `null`): which writes besides a
+ * create or a replace consensus judges against such a rule. Only a reminder
+ * of the protocol's behaviour, the same for every such rule: Rust decides.
+ */
+internal const val PROPERTY_CONSTRAINT_SYSTEM_READS_NOTE =
+ "A price update is judged against the rules reading \$updatedAt or its block heights, " +
+ "and a transfer or purchase against those reading \$transferredAt or its block heights."
+
+/**
+ * The line naming the `countOf` and `sumOf` totals [rule] reads, each once:
+ * `Reads totals: countOf listing by $ownerId; sumOf price of listing by category`.
+ * `null` for a rule reading none. The SwiftExampleApp builds the same text.
+ */
+internal fun propertyConstraintTotalsText(rule: DocumentPropertyConstraint): String? =
+ rule.readsTotals
+ .map(::totalText)
+ .distinct()
+ .takeIf { it.isNotEmpty() }
+ ?.joinToString("; ", prefix = "Reads totals: ")
+
+/**
+ * One total as [propertyConstraintTotalsText] names it: its kind, the property
+ * it sums (a `sumOf` only), its type, and the keys it filters by.
+ */
+private fun totalText(total: PropertyConstraintTotalRead): String = buildString {
+ append(total.kind.name)
+ total.property?.let { append(" $it of") }
+ append(" ${total.documentType}")
+ if (total.filter.isNotEmpty()) append(" by ${total.filter.joinToString(", ")}")
+}
+
+/**
+ * The note under a rule reading a total ([propertyConstraintTotalsText] not
+ * `null`): the check before sending reads no state, so it cannot catch the
+ * rule. The same text as the SwiftExampleApp's, for cross-platform UAT.
+ */
+internal const val PROPERTY_CONSTRAINT_TOTALS_NOTE =
+ "The platform reads these totals when the document is sent; the check before sending " +
+ "does not, so it cannot catch this rule."
+
+/** Title of the alert a broken rule raises instead of a broadcast. */
+internal const val PROPERTY_CONSTRAINT_BROKEN_TITLE = "Not sent: a property constraint is broken"
+
+/** Body of the alert a broken rule raises: the rule, the violation and the reason. */
+internal fun propertyConstraintViolationAlert(violation: PropertyConstraintViolation): String =
+ "Rule: ${violation.rule}\nViolation: ${violation.violation.name}\nReason: ${violation.message}"
diff --git a/packages/kotlin-sdk/KotlinExampleApp/app/src/test/java/org/dashfoundation/example/ui/contracts/PropertyConstraintsTest.kt b/packages/kotlin-sdk/KotlinExampleApp/app/src/test/java/org/dashfoundation/example/ui/contracts/PropertyConstraintsTest.kt
new file mode 100644
index 00000000000..192178a948d
--- /dev/null
+++ b/packages/kotlin-sdk/KotlinExampleApp/app/src/test/java/org/dashfoundation/example/ui/contracts/PropertyConstraintsTest.kt
@@ -0,0 +1,331 @@
+package org.dashfoundation.example.ui.contracts
+
+import kotlinx.coroutines.CancellationException
+import kotlinx.coroutines.test.runTest
+import kotlinx.serialization.json.buildJsonObject
+import kotlinx.serialization.json.put
+import kotlinx.serialization.json.putJsonObject
+import org.dashfoundation.dashsdk.errors.DashSdkError
+import org.dashfoundation.dashsdk.queries.DocumentPropertyConstraint
+import org.dashfoundation.dashsdk.queries.PropertyConstraintRead
+import org.dashfoundation.dashsdk.queries.PropertyConstraintTotalRead
+import org.dashfoundation.dashsdk.queries.PropertyConstraintViolation
+import org.junit.Assert.assertArrayEquals
+import org.junit.Assert.assertEquals
+import org.junit.Assert.assertFalse
+import org.junit.Assert.assertNull
+import org.junit.Assert.assertTrue
+import org.junit.Assert.fail
+import org.junit.Test
+
+/**
+ * Pins the screen-side `propertyConstraints` glue (protocol version 14) behind
+ * the DocumentTypeDetailsScreen section and the CreateDocumentScreen
+ * pre-check. Rust reads and judges the rules; these tests stand in for it
+ * with plain lambdas, so they pin only when the app asks, what it does with
+ * the answer, and that a check which cannot run never blocks a create
+ * (consensus judges the document either way). Pure JVM: no native library.
+ */
+class PropertyConstraintsTest {
+
+ private val schema = buildJsonObject {
+ put("type", "object")
+ putJsonObject("propertyConstraints") {
+ putJsonObject("perUnitFee") { put("present", "fee") }
+ }
+ }
+
+ private val plainSchema = buildJsonObject { put("type", "object") }
+
+ private val stored = byteArrayOf(1, 2, 3)
+
+ private val rule = DocumentPropertyConstraint(
+ name = "sellerIsOwner",
+ ruleJson = """{"anyOf":[{"absent":"sellerId"},{"equal":["sellerId","${'$'}ownerId"]}]}""",
+ reads = listOf(
+ PropertyConstraintRead("sellerId", PropertyConstraintRead.Kind.Presence),
+ PropertyConstraintRead("sellerId", PropertyConstraintRead.Kind.Identifier),
+ PropertyConstraintRead("sellerId", PropertyConstraintRead.Kind.Presence),
+ ),
+ readsOwner = true,
+ )
+
+ /** A rule reading system times and heights, one twice, as Rust lists them. */
+ private val timedRule = DocumentPropertyConstraint(
+ name = "settledAfterTransfer",
+ ruleJson = """{"greaterThan":["${'$'}updatedAt","${'$'}transferredAtCoreBlockHeight"]}""",
+ reads = emptyList(),
+ readsOwner = false,
+ readsSystem = listOf("${'$'}updatedAt", "${'$'}transferredAtCoreBlockHeight", "${'$'}updatedAt"),
+ )
+
+ private val violation = PropertyConstraintViolation(
+ rule = "perUnitFee",
+ violation = PropertyConstraintViolation.Kind.DivisionByZero,
+ message = "it divides by zero",
+ )
+
+ @Test
+ fun `should see the keyword only where the schema declares it`() {
+ assertTrue(documentTypeDeclaresPropertyConstraints(schema))
+ assertFalse(documentTypeDeclaresPropertyConstraints(plainSchema))
+ assertFalse(documentTypeDeclaresPropertyConstraints(null))
+ }
+
+ // Document type screen
+
+ @Test
+ fun `should hide the section without asking Rust when the schema declares no rules`() = runTest {
+ val section = loadPropertyConstraintsSection(plainSchema, stored) {
+ fail("a schema without the keyword must not be read")
+ emptyList()
+ }
+
+ assertEquals(PropertyConstraintsSection.Hidden, section)
+ }
+
+ @Test
+ fun `should list the rules Rust reads from the stored contract`() = runTest {
+ var readBytes: ByteArray? = null
+ val section = loadPropertyConstraintsSection(schema, stored) { bytes ->
+ readBytes = bytes
+ listOf(rule)
+ }
+
+ assertEquals(PropertyConstraintsSection.Rules(listOf(rule)), section)
+ assertArrayEquals(stored, readBytes)
+ }
+
+ /** Below protocol version 14 Rust reports no rules for a type declaring some. */
+ @Test
+ fun `should say the rules are not enforced when Rust reads none`() = runTest {
+ val section = loadPropertyConstraintsSection(schema, stored) { emptyList() }
+
+ assertEquals(PropertyConstraintsSection.NotEnforced, section)
+ }
+
+ @Test
+ fun `should say why the rules cannot be read`() = runTest {
+ val noSdk = loadPropertyConstraintsSection(schema, stored, read = null)
+ assertEquals(
+ PropertyConstraintsSection.Unavailable("Connect to a network to read the property constraints."),
+ noSdk,
+ )
+
+ for (missing in listOf(null, ByteArray(0))) {
+ val noBytes = loadPropertyConstraintsSection(schema, missing) {
+ fail("nothing can be read without the stored serialization")
+ emptyList()
+ }
+ assertTrue(
+ "$noBytes",
+ (noBytes as PropertyConstraintsSection.Unavailable).reason.contains("download it again"),
+ )
+ }
+
+ val failed = loadPropertyConstraintsSection(schema, stored) {
+ throw DashSdkError.SerializationError("Failed to deserialize contract")
+ }
+ assertEquals(
+ PropertyConstraintsSection.Unavailable(
+ "Could not read the property constraints: Failed to deserialize contract",
+ ),
+ failed,
+ )
+ }
+
+ // Create pre-check
+
+ @Test
+ fun `should pass a document whose type declares no rules without asking Rust`() = runTest {
+ val preCheck = propertyConstraintPreCheck(plainSchema, stored) {
+ fail("a schema without the keyword must not be checked")
+ null
+ }
+
+ assertEquals(PropertyConstraintPreCheck.Passed, preCheck)
+ }
+
+ @Test
+ fun `should stop a document that breaks a rule`() = runTest {
+ var checkedBytes: ByteArray? = null
+ val preCheck = propertyConstraintPreCheck(schema, stored) { bytes ->
+ checkedBytes = bytes
+ violation
+ }
+
+ assertEquals(PropertyConstraintPreCheck.Broken(violation), preCheck)
+ assertArrayEquals(stored, checkedBytes)
+ }
+
+ @Test
+ fun `should pass a document that meets every rule`() = runTest {
+ assertEquals(
+ PropertyConstraintPreCheck.Passed,
+ propertyConstraintPreCheck(schema, stored) { null },
+ )
+ }
+
+ @Test
+ fun `should let the create proceed when the check cannot run`() = runTest {
+ assertEquals(
+ PropertyConstraintPreCheck.Skipped("no SDK is connected"),
+ propertyConstraintPreCheck(schema, stored, check = null),
+ )
+ for (missing in listOf(null, ByteArray(0))) {
+ assertEquals(
+ PropertyConstraintPreCheck.Skipped("the data contract has no stored serialization"),
+ propertyConstraintPreCheck(schema, missing) {
+ fail("nothing can be checked without the stored serialization")
+ null
+ },
+ )
+ }
+ assertEquals(
+ PropertyConstraintPreCheck.Skipped("Document type 'offer' not found in the data contract"),
+ propertyConstraintPreCheck(schema, stored) {
+ throw DashSdkError.NotFound("Document type 'offer' not found in the data contract")
+ },
+ )
+ }
+
+ /** An app bundling a native library built before the two exports must neither crash nor block. */
+ @Test
+ fun `should neither crash nor block on a native library without the exports`() = runTest {
+ val missing = UnsatisfiedLinkError("dataContractCheckPropertyConstraints")
+
+ assertEquals(
+ PropertyConstraintPreCheck.Skipped(NATIVE_LIBRARY_PREDATES_RULES),
+ propertyConstraintPreCheck(schema, stored) { throw missing },
+ )
+ assertEquals(
+ PropertyConstraintsSection.Unavailable(NATIVE_LIBRARY_PREDATES_RULES),
+ loadPropertyConstraintsSection(schema, stored) { throw missing },
+ )
+ }
+
+ @Test
+ fun `should not swallow a cancellation`() = runTest {
+ try {
+ propertyConstraintPreCheck(schema, stored) { throw CancellationException("left the screen") }
+ fail("the cancellation must propagate")
+ } catch (e: CancellationException) {
+ assertEquals("left the screen", e.message)
+ }
+ }
+
+ // Display
+
+ @Test
+ fun `should list each read once with its kind`() {
+ assertEquals("sellerId (presence), sellerId (identifier)", propertyConstraintReadsText(rule))
+ assertEquals(
+ "title (length), tags (count), labels (elements)",
+ propertyConstraintReadsText(
+ DocumentPropertyConstraint(
+ name = "sizes",
+ ruleJson = "{}",
+ reads = listOf(
+ PropertyConstraintRead("title", PropertyConstraintRead.Kind.Length),
+ PropertyConstraintRead("tags", PropertyConstraintRead.Kind.Count),
+ PropertyConstraintRead("labels", PropertyConstraintRead.Kind.Elements),
+ ),
+ readsOwner = false,
+ ),
+ ),
+ )
+ }
+
+ @Test
+ fun `should list each system value a rule reads once`() {
+ assertEquals(
+ "Reads ${'$'}updatedAt, ${'$'}transferredAtCoreBlockHeight",
+ propertyConstraintSystemReadsText(timedRule),
+ )
+ }
+
+ @Test
+ fun `should show no system line for a rule reading none`() {
+ assertNull(propertyConstraintSystemReadsText(rule))
+ }
+
+ /** A rule reading a count by owner twice, a whole-type count and a sum by category. */
+ private val totalledRule = DocumentPropertyConstraint(
+ name = "withinLimits",
+ ruleJson = "{}",
+ reads = emptyList(),
+ readsOwner = true,
+ readsTotals = listOf(
+ PropertyConstraintTotalRead(
+ PropertyConstraintTotalRead.Kind.CountOf,
+ "listing",
+ null,
+ listOf("${'$'}ownerId"),
+ ),
+ PropertyConstraintTotalRead(
+ PropertyConstraintTotalRead.Kind.SumOf,
+ "listing",
+ "price",
+ listOf("category", "status"),
+ ),
+ PropertyConstraintTotalRead(
+ PropertyConstraintTotalRead.Kind.CountOf,
+ "listing",
+ null,
+ listOf("${'$'}ownerId"),
+ ),
+ PropertyConstraintTotalRead(
+ PropertyConstraintTotalRead.Kind.CountOf,
+ "listing",
+ null,
+ emptyList(),
+ ),
+ ),
+ )
+
+ /** Each total once, its filter keys after "by"; the SwiftExampleApp builds the same text. */
+ @Test
+ fun `should list each total a rule reads once`() {
+ assertEquals(
+ "Reads totals: countOf listing by ${'$'}ownerId; sumOf price of listing by " +
+ "category, status; countOf listing",
+ propertyConstraintTotalsText(totalledRule),
+ )
+ assertNull(propertyConstraintTotalsText(timedRule))
+ }
+
+ /** The note under a rule reading a total; the SwiftExampleApp shows the same text. */
+ @Test
+ fun `should say the check before sending cannot catch a rule reading a total`() {
+ assertEquals(
+ "The platform reads these totals when the document is sent; the check before " +
+ "sending does not, so it cannot catch this rule.",
+ PROPERTY_CONSTRAINT_TOTALS_NOTE,
+ )
+ }
+
+ /** The note under a rule reading a system value; the SwiftExampleApp shows the same text. */
+ @Test
+ fun `should say which writes the update and transfer times answer to`() {
+ assertEquals(
+ "A price update is judged against the rules reading ${'$'}updatedAt or its block " +
+ "heights, and a transfer or purchase against those reading ${'$'}transferredAt or " +
+ "its block heights.",
+ PROPERTY_CONSTRAINT_SYSTEM_READS_NOTE,
+ )
+ }
+
+ @Test
+ fun `should name the rule the violation and the reason in the alert`() {
+ assertEquals(
+ "Rule: perUnitFee\nViolation: DivisionByZero\nReason: it divides by zero",
+ propertyConstraintViolationAlert(violation),
+ )
+ assertEquals(
+ "Rule: r\nViolation: Later\nReason: m",
+ propertyConstraintViolationAlert(
+ PropertyConstraintViolation("r", PropertyConstraintViolation.Kind.Other("Later"), "m"),
+ ),
+ )
+ }
+}
diff --git a/packages/kotlin-sdk/sdk/src/androidTest/kotlin/org/dashfoundation/dashsdk/PropertyConstraintsFfiTest.kt b/packages/kotlin-sdk/sdk/src/androidTest/kotlin/org/dashfoundation/dashsdk/PropertyConstraintsFfiTest.kt
new file mode 100644
index 00000000000..01b60d620f2
--- /dev/null
+++ b/packages/kotlin-sdk/sdk/src/androidTest/kotlin/org/dashfoundation/dashsdk/PropertyConstraintsFfiTest.kt
@@ -0,0 +1,116 @@
+package org.dashfoundation.dashsdk
+
+import androidx.test.ext.junit.runners.AndroidJUnit4
+import org.dashfoundation.dashsdk.ffi.DashSDKException
+import org.dashfoundation.dashsdk.ffi.NativeLoader
+import org.dashfoundation.dashsdk.ffi.QueriesNative
+import org.dashfoundation.dashsdk.ffi.SdkNative
+import org.dashfoundation.dashsdk.queries.DocumentPropertyConstraint
+import org.dashfoundation.dashsdk.queries.PropertyConstraintViolation
+import org.junit.After
+import org.junit.Assert.assertEquals
+import org.junit.Assert.assertNotEquals
+import org.junit.Assert.assertNull
+import org.junit.Assert.assertThrows
+import org.junit.Before
+import org.junit.Test
+import org.junit.runner.RunWith
+
+/**
+ * Round trips of the two `propertyConstraints` JNI exports through
+ * `rs-sdk-ffi` on a device: the marshalling of the contract bytes, the
+ * strings and the owner id, and the error codes the Kotlin side maps
+ * (`DashSdkError.fromNative`). Rust's own tests cover the rules; the fixture
+ * declares none, so the answers are `[]` and `null` at any protocol version.
+ * No network access: `createTrusted` only builds the client. Kotlin
+ * counterpart of the FFI round trips in the Swift SDK's
+ * `DocumentPropertyConstraintsTests`.
+ */
+@RunWith(AndroidJUnit4::class)
+class PropertyConstraintsFfiTest {
+
+ /**
+ * The platform serialization of a contract created at protocol version
+ * 13, owned by `[7; 32]`, declaring one `note` type with a string
+ * `message` and no rules (the Swift test's fixture, generated by
+ * rs-sdk-ffi's `serialize_to_bytes_with_platform_version`).
+ */
+ private val noteContract = (
+ "013ac40651a4bcb91c3f5c1e458a1c95e48dc2c8259003dd580cab11a5f02ed4850100000000010100000101070707070707" +
+ "07070707070707070707070707070707070707070707070707070001046e6f7465160312047479706512066f626a65637412" +
+ "0a70726f70657274696573160112076d65737361676516031204747970651206737472696e6712096d61784c656e67746805" +
+ "801208706f736974696f6e050012146164646974696f6e616c50726f70657274696573130000000000000000000000"
+ ).chunked(2).map { it.toInt(16).toByte() }.toByteArray()
+
+ private val ownerId = ByteArray(32) { 7 }
+
+ private var handle = 0L
+
+ @Before
+ fun createSdk() {
+ NativeLoader.ensureLoaded()
+ handle = SdkNative.createTrusted(
+ network = 1,
+ dapiAddresses = null,
+ quorumUrl = null,
+ skipAssetLockProofVerification = false,
+ requestRetryCount = 3,
+ requestTimeoutMs = 30_000,
+ platformVersion = 0,
+ )
+ assertNotEquals("createTrusted must return a live handle", 0L, handle)
+ }
+
+ @After
+ fun destroySdk() {
+ SdkNative.destroy(handle)
+ }
+
+ @Test
+ fun shouldReadNoRulesAndNoViolationForATypeDeclaringNone() {
+ val rules = QueriesNative.dataContractGetPropertyConstraints(handle, noteContract, "note")
+ assertEquals(emptyList(), DocumentPropertyConstraint.listFromJson(rules!!))
+
+ val verdict = QueriesNative.dataContractCheckPropertyConstraints(
+ handle,
+ noteContract,
+ "note",
+ """{"message":"hi"}""",
+ ownerId,
+ )
+ assertEquals("null", verdict)
+ assertNull(PropertyConstraintViolation.fromJson(verdict!!))
+ }
+
+ @Test
+ fun shouldKeepTheNativeErrorCodes() {
+ // DashSDKErrorCode: 1 InvalidParameter, 4 SerializationError, 7 NotFound
+ assertCode(7) { QueriesNative.dataContractGetPropertyConstraints(handle, noteContract, "letter") }
+ assertCode(4) {
+ QueriesNative.dataContractGetPropertyConstraints(handle, byteArrayOf(-1, 0, 19), "note")
+ }
+ assertCode(1) { QueriesNative.dataContractGetPropertyConstraints(handle, ByteArray(0), "note") }
+ assertCode(1) {
+ QueriesNative.dataContractCheckPropertyConstraints(handle, noteContract, "note", "[1]", ownerId)
+ }
+ }
+
+ /** The C function reads 32 bytes behind the owner pointer, so the JNI refuses a shorter id. */
+ @Test
+ fun shouldRefuseAnOwnerIdThatIsNot32Bytes() {
+ assertCode(1) {
+ QueriesNative.dataContractCheckPropertyConstraints(
+ handle,
+ noteContract,
+ "note",
+ "{}",
+ ByteArray(20) { 7 },
+ )
+ }
+ }
+
+ private fun assertCode(code: Int, call: () -> Unit) {
+ val error = assertThrows(DashSDKException::class.java) { call() }
+ assertEquals(error.message, code, error.code)
+ }
+}
diff --git a/packages/kotlin-sdk/sdk/src/androidTest/kotlin/org/dashfoundation/dashsdk/wallet/DashPayUnlockAndSyncTest.kt b/packages/kotlin-sdk/sdk/src/androidTest/kotlin/org/dashfoundation/dashsdk/wallet/DashPayUnlockAndSyncTest.kt
index b6f66ea5ca6..9fd179b594c 100644
--- a/packages/kotlin-sdk/sdk/src/androidTest/kotlin/org/dashfoundation/dashsdk/wallet/DashPayUnlockAndSyncTest.kt
+++ b/packages/kotlin-sdk/sdk/src/androidTest/kotlin/org/dashfoundation/dashsdk/wallet/DashPayUnlockAndSyncTest.kt
@@ -22,7 +22,7 @@ import org.junit.runner.RunWith
/**
* Instrumented coverage for the K2 seedless-unlock topology and the
- * DashPay sync-service lifecycle (KOTLIN_MIGRATION_SPEC.md §K2), through
+ * DashPay sync-service lifecycle, through
* the real native lib:
*
* - The **happy-path unlock is the end-to-end proof of the out-buffer
diff --git a/packages/kotlin-sdk/sdk/src/androidTest/kotlin/org/dashfoundation/dashsdk/wallet/WalletManagerRoundTripTest.kt b/packages/kotlin-sdk/sdk/src/androidTest/kotlin/org/dashfoundation/dashsdk/wallet/WalletManagerRoundTripTest.kt
index 34ec9d4ade4..77154af19e8 100644
--- a/packages/kotlin-sdk/sdk/src/androidTest/kotlin/org/dashfoundation/dashsdk/wallet/WalletManagerRoundTripTest.kt
+++ b/packages/kotlin-sdk/sdk/src/androidTest/kotlin/org/dashfoundation/dashsdk/wallet/WalletManagerRoundTripTest.kt
@@ -119,7 +119,7 @@ class WalletManagerRoundTripTest {
*
* Of the K1 getters, `searchDpnsNames` is deliberately untested here:
* it is a live network query and belongs to the `-Ptestnet=true`
- * tier (KOTLIN_MIGRATION_SPEC.md §7.4).
+ * tier.
*/
@Test
fun dashPayRestoreRoundTripsPaymentsContactProfilesAndSyncState() = runBlocking {
diff --git a/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/documents/DocumentTransactions.kt b/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/documents/DocumentTransactions.kt
index 518bff78278..866dfffea82 100644
--- a/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/documents/DocumentTransactions.kt
+++ b/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/documents/DocumentTransactions.kt
@@ -113,6 +113,10 @@ class DocumentTransactions internal constructor(
* @param propertiesJson JSON object keyed by property name (byte-array
* fields as hex, identifier fields as base58); `"{}"` for a document
* type with no required properties.
+ * @param maxContestFund the most, in credits, [ownerId] pays into the
+ * contest a contested document joins, and [ownerId] must hold it;
+ * `null` states the current fund to join, read just before signing. A document that joins no contest
+ * ignores it.
* @return the confirmed document's canonical JSON (now owned by
* [ownerId]; its 32-byte id is the `$id` field).
*/
@@ -123,9 +127,13 @@ class DocumentTransactions internal constructor(
documentType: String,
propertiesJson: String,
signerHandle: Long,
+ maxContestFund: Long? = null,
): String = gate.op {
require(ownerId.size == 32) { "ownerId must be 32 bytes" }
require(contractId.size == 32) { "contractId must be 32 bytes" }
+ require(maxContestFund == null || maxContestFund >= 0) {
+ "maxContestFund must be non-negative, got $maxContestFund"
+ }
mapNativeErrors {
TransactionsNative.documentCreate(
walletHandle,
@@ -133,6 +141,7 @@ class DocumentTransactions internal constructor(
contractId,
documentType,
propertiesJson,
+ maxContestFund ?: 0L,
signerHandle,
)
}
diff --git a/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/errors/PlatformConsensusError.kt b/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/errors/PlatformConsensusError.kt
index 52cb67eb152..7a26cd981b0 100644
--- a/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/errors/PlatformConsensusError.kt
+++ b/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/errors/PlatformConsensusError.kt
@@ -7,10 +7,11 @@ package org.dashfoundation.dashsdk.errors
* to. Branch on these rather than on the error message, whose wording is not
* a contract.
*
- * Read it from [DashSdkError.consensusError]. It is present for the wallet
- * operations whose native error still holds the SDK's consensus verdict (the
- * token state transitions among them) and `null` for every failure that was
- * not a consensus rejection.
+ * Read it from [DashSdkError.consensusError]. It is present for the state
+ * transitions whose native error still holds the SDK's consensus verdict: the
+ * rs-sdk-ffi document, token, data contract and identity transitions, and the
+ * platform-wallet operations (the token state transitions among them). It is
+ * `null` for every failure that was not a consensus rejection.
*/
data class PlatformConsensusError(
val code: Int,
@@ -21,7 +22,9 @@ data class PlatformConsensusError(
* Which of rs-dpp's consensus error families an error belongs to. The native
* layer reports it next to the code, so it is never derived from the number
* on this side. Values mirror `PlatformWalletFFIConsensusErrorKind` in
- * `rs-platform-wallet-ffi/src/error.rs`.
+ * `rs-platform-wallet-ffi/src/error.rs` and `DashSDKConsensusErrorKind` in
+ * `rs-sdk-ffi/src/error.rs`, which a compile-time guard in
+ * `rs-unified-sdk-jni/src/support.rs` holds equal.
*/
enum class ConsensusErrorKind {
/** Structure or version validation failed; the transition was not executed. */
diff --git a/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/ffi/DashSDKException.kt b/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/ffi/DashSDKException.kt
index fe8ab7c976d..ff35b52834f 100644
--- a/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/ffi/DashSDKException.kt
+++ b/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/ffi/DashSDKException.kt
@@ -16,7 +16,7 @@ import org.dashfoundation.dashsdk.errors.PlatformConsensusError
* 8=Timeout, 9=NotImplemented, 10=DriveInternalError, 99=InternalError.
*
* [consensusError] is the consensus rejection behind the failure when the
- * native result carried one, `null` otherwise.
+ * native result or `DashSDKError` carried one, `null` otherwise.
*
* Internal: the public API maps this into the
* [org.dashfoundation.dashsdk.errors.DashSdkError] hierarchy.
@@ -30,7 +30,9 @@ class DashSDKException(
/**
* JNI entry for a failure that is a consensus rejection. [consensusKind]
- * is the `PlatformWalletFFIConsensusErrorKind` discriminant.
+ * is the discriminant of the native kind: `PlatformWalletFFIConsensusErrorKind`
+ * for a platform-wallet result, `DashSDKConsensusErrorKind` for an
+ * rs-sdk-ffi error. The two share their values.
*/
constructor(code: Int, message: String, consensusCode: Int, consensusKind: Int) : this(
code,
diff --git a/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/ffi/IdentityNative.kt b/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/ffi/IdentityNative.kt
index 29be7860bdf..20c31492e1c 100644
--- a/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/ffi/IdentityNative.kt
+++ b/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/ffi/IdentityNative.kt
@@ -219,11 +219,16 @@ internal object IdentityNative {
/**
* Register a DPNS name for [identityId] (32 bytes), signed via
* [signerHandle]. Returns the full domain name (e.g. `"alice.dash"`).
+ *
+ * @param maxContestFund the most, in credits, the identity pays into
+ * the contest a contested name joins; `0` states the current fund to
+ * join, read just before signing. Must be non-negative.
*/
external fun registerDpnsName(
walletHandle: Long,
identityId: ByteArray,
label: String,
+ maxContestFund: Long,
signerHandle: Long,
): String
diff --git a/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/ffi/QueriesNative.kt b/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/ffi/QueriesNative.kt
index 55676165e54..bf961597ad8 100644
--- a/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/ffi/QueriesNative.kt
+++ b/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/ffi/QueriesNative.kt
@@ -72,6 +72,47 @@ internal object QueriesNative {
serializedContracts: Array,
)
+ /**
+ * The `propertyConstraints` rules (protocol version 14) of [documentType]
+ * as a JSON array in name order, each
+ * `{"name", "rule", "reads": [{"path", "kind"}], "readsOwner", "readsSystem", "readsTotals"}`:
+ * `kind` is `value`, `presence`, `text`, `identifier`, `length`, `count`
+ * or `elements`, `readsSystem` names the system times and heights the
+ * rule reads (`$createdAt`, `$updatedAtBlockHeight`, ...), and
+ * `readsTotals` lists its `countOf` and `sumOf` totals as
+ * `{"kind", "documentType", "property" (a sumOf only), "filter"}`.
+ * [serializedContract] is the contract's platform serialization (what
+ * [dataContractFetchWithSerialization] returns), read by Rust at the SDK's
+ * protocol version; no network call. Throws on error (unknown document
+ * type, bytes that are not a contract, empty input).
+ */
+ external fun dataContractGetPropertyConstraints(
+ sdk: Long,
+ serializedContract: ByteArray,
+ documentType: String,
+ ): String?
+
+ /**
+ * The first `propertyConstraints` rule a document to create would break,
+ * as `{"rule", "violation", "message"}`, or the JSON text `null` when it
+ * meets every rule. [propertiesJson] is what the create would send and
+ * [ownerId] the 32-byte owner `$ownerId` reads; [serializedContract] as
+ * for [dataContractGetPropertyConstraints]. The device clock stands in for
+ * the block time the create records (`$createdAt`, `$updatedAt`,
+ * `$transferredAt`), and a rule reading a block height is not judged, the
+ * height being unknown until the block, nor is one reading a `countOf` or
+ * `sumOf` total, which only the platform reads from state. No network call. Throws on
+ * error (as above, plus an owner id that is not 32 bytes or properties
+ * that are not a JSON object).
+ */
+ external fun dataContractCheckPropertyConstraints(
+ sdk: Long,
+ serializedContract: ByteArray,
+ documentType: String,
+ propertiesJson: String,
+ ownerId: ByteArray,
+ ): String?
+
/** JSON array of documents. whereJson/orderByJson may be null. */
external fun documentSearch(
sdk: Long,
diff --git a/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/ffi/TransactionsNative.kt b/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/ffi/TransactionsNative.kt
index 7c3b6e6c6a7..dc0c357cff4 100644
--- a/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/ffi/TransactionsNative.kt
+++ b/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/ffi/TransactionsNative.kt
@@ -111,6 +111,9 @@ internal object TransactionsNative {
* @param propertiesJson JSON object keyed by property name (byte-array
* fields as hex, identifier fields as base58); `"{}"` for a type with
* no required properties.
+ * @param maxContestFund the most, in credits, the owner pays into the
+ * contest a contested document joins; `0` states the current fund to
+ * join, read just before signing. Must be non-negative.
* @return the confirmed document's canonical JSON (its 32-byte id is the
* base58 `$id` field).
*/
@@ -120,6 +123,7 @@ internal object TransactionsNative {
contractId: ByteArray,
documentType: String,
propertiesJson: String,
+ maxContestFund: Long,
signerHandle: Long,
): String
diff --git a/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/identity/IdentityRegistration.kt b/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/identity/IdentityRegistration.kt
index 965b5a14b0c..b8c4cd647f4 100644
--- a/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/identity/IdentityRegistration.kt
+++ b/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/identity/IdentityRegistration.kt
@@ -489,15 +489,30 @@ class IdentityRegistration internal constructor(
/**
* Register a DPNS name for [identityId] (32 bytes), signed via
* [signerHandle]. Returns the full domain name (e.g. `"alice.dash"`).
+ *
+ * @param maxContestFund the most, in credits, the identity pays into
+ * the contest a contested name joins, and the identity must hold it;
+ * `null` states the current fund to join, read just before signing. A name that joins no contest
+ * ignores it. Mirrors Swift `ManagedPlatformWallet.registerDpnsName`.
*/
suspend fun registerDpnsName(
walletHandle: Long,
identityId: ByteArray,
label: String,
signerHandle: Long,
+ maxContestFund: Long? = null,
): String = gate.op {
+ require(maxContestFund == null || maxContestFund >= 0) {
+ "maxContestFund must be non-negative, got $maxContestFund"
+ }
mapNativeErrors {
- IdentityNative.registerDpnsName(walletHandle, identityId, label, signerHandle)
+ IdentityNative.registerDpnsName(
+ walletHandle,
+ identityId,
+ label,
+ maxContestFund ?: 0L,
+ signerHandle,
+ )
}
}
}
diff --git a/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/persistence/DashDatabase.kt b/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/persistence/DashDatabase.kt
index 94ebbb0267f..e74e3b00954 100644
--- a/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/persistence/DashDatabase.kt
+++ b/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/persistence/DashDatabase.kt
@@ -684,6 +684,13 @@ abstract class DashDatabase : RoomDatabase() {
* API 16+; writes go through the persistence handler inside
* `withTransaction`, mirroring the changeset bracketing contract of
* `platform-wallet-ffi`.
+ *
+ * Room runs WAL with `synchronous=NORMAL`: a committed transaction
+ * survives process death, but a power loss can roll back the most
+ * recent commit. That is accepted for every persistence capability
+ * the handler attests (the same class of risk as iOS);
+ * `synchronous=FULL` was deliberately not adopted because it costs
+ * an fsync on every commit.
*/
fun create(context: Context): DashDatabase =
Room.databaseBuilder(context, DashDatabase::class.java, DATABASE_NAME)
diff --git a/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/queries/DocumentPropertyConstraints.kt b/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/queries/DocumentPropertyConstraints.kt
new file mode 100644
index 00000000000..38a868337a3
--- /dev/null
+++ b/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/queries/DocumentPropertyConstraints.kt
@@ -0,0 +1,437 @@
+package org.dashfoundation.dashsdk.queries
+
+import kotlinx.serialization.SerializationException
+import kotlinx.serialization.json.Json
+import kotlinx.serialization.json.JsonArray
+import kotlinx.serialization.json.JsonElement
+import kotlinx.serialization.json.JsonNull
+import kotlinx.serialization.json.JsonObject
+import kotlinx.serialization.json.JsonPrimitive
+import kotlinx.serialization.json.booleanOrNull
+import org.dashfoundation.dashsdk.errors.DashSdkError
+
+/**
+ * A rule of a document type's `propertyConstraints` (protocol version 14): a
+ * named condition every created or replaced document's properties must meet.
+ * Consensus checks every rule, in name order, and refuses a document breaking
+ * one with `DocumentPropertyConstraintViolatedError` (code 10422); a refused
+ * state transition is still paid for. A transfer, a purchase or a price update
+ * is judged against the rules reading what it changes ([readsOwner],
+ * [readsSystem]).
+ *
+ * Rust parses the rules and reports them
+ * (`dash_sdk_data_contract_get_property_constraints`, through
+ * [Contracts.propertyConstraints]); this type only carries what it reports.
+ * The fields mirror wasm-dpp2's `DocumentPropertyConstraint` key for key, and
+ * the Swift SDK's `DocumentPropertyConstraint`
+ * (`SwiftDashSDK/Core/Utils/DocumentPropertyConstraints.swift`).
+ */
+data class DocumentPropertyConstraint(
+ /** The rule's name, its key in `propertyConstraints`. */
+ val name: String,
+ /**
+ * The rule exactly as the document type's schema declares it, as compact
+ * JSON text with sorted keys (every operator object has a single key, so
+ * sorting changes nothing a reader would notice). Among its operators:
+ * sizes (`{ "length": path }`, `{ "byteLength": path }`,
+ * `{ "count": path }`), system times and heights as bare operands
+ * (`"$createdAt"`), `{ "contains": [arrayPath, value] }`,
+ * `{ "startsWith": [a, b] }`, `{ "endsWith": [a, b] }`,
+ * `{ "notIn": [operand, [values]] }`, `{ "min": [a, b, ...] }`,
+ * `{ "max": [a, b, ...] }`, `{ "abs": a }`, `{ "ifThen": [if, then] }`
+ * and `{ "ifThenElse": [if, then, else] }`.
+ */
+ val ruleJson: String,
+ /**
+ * Every property the rule reads, in declared order, a property read twice
+ * listed twice, every branch of an `ifThen` or `ifThenElse` included,
+ * whichever one a document takes. `$ownerId` and the system times and
+ * heights are no properties and are not listed: see [readsOwner] and
+ * [readsSystem], which cover every branch too.
+ */
+ val reads: List,
+ /**
+ * Whether the rule compares the document's owner, `$ownerId`, or reads a
+ * total that depends on it ([readsTotals]): then a transfer or a purchase,
+ * which changes the owner, is judged against it too.
+ */
+ val readsOwner: Boolean,
+ /**
+ * The system times and heights the rule reads, by name, in declared order,
+ * one read twice listed twice: `$createdAt`, `$updatedAt` and
+ * `$transferredAt` (a block time in milliseconds), each also with
+ * `BlockHeight` or `CoreBlockHeight` appended (the Platform or the Core
+ * block height). The names are wasm-dpp2's
+ * `PropertyConstraintSystemProperty`. A rule reads only the ones its
+ * document type records, by listing them in `required`.
+ *
+ * Consensus judges a price update against the rules reading the update's
+ * (`$updatedAt...`), and a transfer or a purchase against those reading
+ * the transfer's (`$transferredAt...`) or the owner ([readsOwner]). This
+ * list only names what a rule reads; Rust decides which writes a rule
+ * answers to.
+ *
+ * Empty for a rule reading none, and for every rule when the native
+ * library predates the field.
+ */
+ val readsSystem: List = emptyList(),
+ /**
+ * The `countOf` and `sumOf` totals the rule reads, in declared order, one
+ * read twice listed twice: how many documents of a type of the same
+ * contract match a filter, or the total of their integer property. The
+ * platform reads them from state when the document is sent; the pre-check
+ * ([Contracts.checkPropertyConstraints]) reads no state and does not judge
+ * a rule reading one.
+ *
+ * Empty for a rule reading none, and for every rule when the native
+ * library predates the field.
+ */
+ val readsTotals: List = emptyList(),
+) {
+ /** [ruleJson] indented for display, or [ruleJson] itself should it not parse back. */
+ val prettyRuleJson: String
+ get() = try {
+ PropertyConstraintJson.pretty(Json.parseToJsonElement(ruleJson))
+ } catch (_: SerializationException) {
+ ruleJson
+ }
+
+ companion object {
+ /**
+ * Decode the JSON array `dash_sdk_data_contract_get_property_constraints`
+ * returns, keeping its order (name order). A rule without
+ * `readsSystem`, from a native library built before it, reads as
+ * reading no system value.
+ *
+ * @throws DashSdkError.SerializationError for text that is not such an array.
+ */
+ fun listFromJson(json: String): List {
+ val entries = PropertyConstraintJson.parse(json) as? JsonArray
+ ?: throw DashSdkError.SerializationError(
+ "propertyConstraints rules are not a JSON array",
+ )
+ return entries.map { entry ->
+ val rule = entry as? JsonObject
+ val name = rule?.get("name")?.jsonStringOrNull()
+ val declaration = rule?.get("rule")
+ val reads = rule?.get("reads") as? JsonArray
+ val readsOwner = rule?.get("readsOwner")?.jsonBooleanOrNull()
+ val readsSystem = rule?.let(::readsSystemOf)
+ val readsTotals = rule?.let(::readsTotalsOf)
+ if (name == null || declaration == null || reads == null || readsOwner == null ||
+ readsSystem == null || readsTotals == null
+ ) {
+ throw DashSdkError.SerializationError("Malformed propertyConstraints rule: $entry")
+ }
+ DocumentPropertyConstraint(
+ name = name,
+ ruleJson = PropertyConstraintJson.compact(declaration),
+ reads = reads.map(PropertyConstraintRead::fromJson),
+ readsOwner = readsOwner,
+ readsSystem = readsSystem,
+ readsTotals = readsTotals,
+ )
+ }
+ }
+
+ /**
+ * The names [rule]'s `readsSystem` lists: empty when the key is
+ * missing, `null` when it is anything but an array of strings.
+ */
+ private fun readsSystemOf(rule: JsonObject): List? {
+ val value = rule["readsSystem"] ?: return emptyList()
+ val names = value as? JsonArray ?: return null
+ return names.map { it.jsonStringOrNull() ?: return null }
+ }
+
+ /**
+ * The totals [rule]'s `readsTotals` lists: empty when the key is
+ * missing, `null` when it or an entry is malformed.
+ */
+ private fun readsTotalsOf(rule: JsonObject): List? {
+ val value = rule["readsTotals"] ?: return emptyList()
+ val entries = value as? JsonArray ?: return null
+ return entries.map { PropertyConstraintTotalRead.fromJsonOrNull(it) ?: return null }
+ }
+ }
+}
+
+/**
+ * A `countOf` or `sumOf` total a `propertyConstraints` rule reads: how many
+ * documents of [documentType], a type of the same contract, match the filter,
+ * or the total of their integer [property] (a `sumOf` only). The fields mirror
+ * wasm-dpp2's `PropertyConstraintTotalRead` and the Swift SDK's.
+ */
+data class PropertyConstraintTotalRead(
+ val kind: Kind,
+ /** The document type the total is over. */
+ val documentType: String,
+ /** The summed integer property of a `sumOf`; `null` for a `countOf`. */
+ val property: String?,
+ /**
+ * The keys the documents are matched by, properties of [documentType] or
+ * `$ownerId`, in the order Rust gives; empty for a total over every
+ * document of the type. The values they must take are in the rule.
+ */
+ val filter: List,
+) {
+ /** What the total counts; the names are the operators'. */
+ sealed interface Kind {
+ /** The kind's name, as Rust reports it. */
+ val name: String
+
+ /** `countOf`: how many documents match. */
+ data object CountOf : Kind {
+ override val name: String get() = "countOf"
+ }
+
+ /** `sumOf`: the total of an integer property over them. */
+ data object SumOf : Kind {
+ override val name: String get() = "sumOf"
+ }
+
+ /** A kind this build does not know, by its name: one a later native library reports. */
+ data class Other(override val name: String) : Kind
+
+ companion object {
+ /** The kind named [name], or [Other] for a name this build does not know. */
+ fun fromName(name: String): Kind = when (name) {
+ CountOf.name -> CountOf
+ SumOf.name -> SumOf
+ else -> Other(name)
+ }
+ }
+ }
+
+ internal companion object {
+ /** The total [entry] describes, or `null` when it is malformed. */
+ fun fromJsonOrNull(entry: JsonElement): PropertyConstraintTotalRead? {
+ val total = entry as? JsonObject ?: return null
+ val kind = total["kind"]?.jsonStringOrNull() ?: return null
+ val documentType = total["documentType"]?.jsonStringOrNull() ?: return null
+ val property = when (val value = total["property"]) {
+ null -> null
+ else -> value.jsonStringOrNull() ?: return null
+ }
+ val keys = total["filter"] as? JsonArray ?: return null
+ val filter = keys.map { it.jsonStringOrNull() ?: return null }
+ return PropertyConstraintTotalRead(Kind.fromName(kind), documentType, property, filter)
+ }
+ }
+}
+
+/** A property a `propertyConstraints` rule reads, and how it reads it. */
+data class PropertyConstraintRead(
+ /** The property's dotted path. */
+ val path: String,
+ val kind: Kind,
+) {
+ /** How a rule reads a property; the names are wasm-dpp2's `PropertyConstraintReadKind`. */
+ sealed interface Kind {
+ /** The kind's name, as Rust reports it. */
+ val name: String
+
+ /** By its value, as an integer operand: an integer or boolean property. */
+ data object Value : Kind {
+ override val name: String get() = "value"
+ }
+
+ /** Only whether the document holds it, in `present` or `absent`. */
+ data object Presence : Kind {
+ override val name: String get() = "presence"
+ }
+
+ /** By its value, compared with strings: a string property. */
+ data object Text : Kind {
+ override val name: String get() = "text"
+ }
+
+ /** By its value, compared with identifiers: an identifier property. */
+ data object Identifier : Kind {
+ override val name: String get() = "identifier"
+ }
+
+ /**
+ * By its size, in a `length` operand (its characters) or a
+ * `byteLength` operand (its UTF-8 bytes): a string property.
+ */
+ data object Length : Kind {
+ override val name: String get() = "length"
+ }
+
+ /**
+ * By its size, in a `count` operand: an array property's items, or a
+ * byte array property's bytes.
+ */
+ data object Count : Kind {
+ override val name: String get() = "count"
+ }
+
+ /** By its elements, which a `contains` looks among: a typed array property. */
+ data object Elements : Kind {
+ override val name: String get() = "elements"
+ }
+
+ /** A kind this build does not know, by its name: one a later native library reports. */
+ data class Other(override val name: String) : Kind
+
+ companion object {
+ /** The kind named [name], or [Other] for a name this build does not know. */
+ fun fromName(name: String): Kind = when (name) {
+ Value.name -> Value
+ Presence.name -> Presence
+ Text.name -> Text
+ Identifier.name -> Identifier
+ Length.name -> Length
+ Count.name -> Count
+ Elements.name -> Elements
+ else -> Other(name)
+ }
+ }
+ }
+
+ internal companion object {
+ fun fromJson(entry: JsonElement): PropertyConstraintRead {
+ val read = entry as? JsonObject
+ val path = read?.get("path")?.jsonStringOrNull()
+ val kind = read?.get("kind")?.jsonStringOrNull()
+ if (path == null || kind == null) {
+ throw DashSdkError.SerializationError("Malformed propertyConstraints read: $entry")
+ }
+ return PropertyConstraintRead(path, Kind.fromName(kind))
+ }
+ }
+}
+
+/**
+ * The first `propertyConstraints` rule a document breaks, as consensus would
+ * report it in `DocumentPropertyConstraintViolatedError` (code 10422).
+ *
+ * Rust judges the document (`dash_sdk_data_contract_check_property_constraints`,
+ * through [Contracts.checkPropertyConstraints]) with the check consensus runs,
+ * the device clock standing in for the times the create records, and a rule
+ * reading a block height or a `countOf` or `sumOf` total left unjudged; this
+ * type only carries the verdict.
+ * The fields mirror wasm-dpp2's `DocumentPropertyConstraintViolation` and the
+ * Swift SDK's `PropertyConstraintViolation`.
+ */
+data class PropertyConstraintViolation(
+ /** The broken rule's name. */
+ val rule: String,
+ val violation: Kind,
+ /** A readable reason, as in the consensus error's message. */
+ val message: String,
+) {
+ /** Why the rule is broken; the names are wasm-dpp2's `PropertyConstraintViolationKind`. */
+ sealed interface Kind {
+ /** The reason's name, as Rust reports it. */
+ val name: String
+
+ /** The rule evaluates without a fault but does not hold. */
+ data object NotMet : Kind {
+ override val name: String get() = "NotMet"
+ }
+
+ /** A value the rule reads or computes does not fit a 128-bit signed integer. */
+ data object Overflow : Kind {
+ override val name: String get() = "Overflow"
+ }
+
+ /** A `divide` or `modulo` by zero. */
+ data object DivisionByZero : Kind {
+ override val name: String get() = "DivisionByZero"
+ }
+
+ /** A `power` with a negative exponent. */
+ data object NegativeExponent : Kind {
+ override val name: String get() = "NegativeExponent"
+ }
+
+ /** A value the rule reads is not an integer. */
+ data object NotAnInteger : Kind {
+ override val name: String get() = "NotAnInteger"
+ }
+
+ /** A reason this build does not know, by its name. */
+ data class Other(override val name: String) : Kind
+
+ companion object {
+ /** The reason named [name], or [Other] for a name this build does not know. */
+ fun fromName(name: String): Kind = when (name) {
+ NotMet.name -> NotMet
+ Overflow.name -> Overflow
+ DivisionByZero.name -> DivisionByZero
+ NegativeExponent.name -> NegativeExponent
+ NotAnInteger.name -> NotAnInteger
+ else -> Other(name)
+ }
+ }
+ }
+
+ /** One sentence naming the rule, the reason and the message (Swift's `errorDescription`). */
+ val description: String
+ get() = "The document breaks the propertyConstraints rule \"$rule\" " +
+ "(${violation.name}): $message."
+
+ companion object {
+ /**
+ * Decode the JSON `dash_sdk_data_contract_check_property_constraints`
+ * returns: `null` for JSON `null`, when the document meets every rule.
+ *
+ * @throws DashSdkError.SerializationError for text that is neither
+ * `null` nor a violation object.
+ */
+ fun fromJson(json: String): PropertyConstraintViolation? {
+ val value = PropertyConstraintJson.parse(json)
+ if (value is JsonNull) return null
+ val violation = value as? JsonObject
+ val rule = violation?.get("rule")?.jsonStringOrNull()
+ val kind = violation?.get("violation")?.jsonStringOrNull()
+ val message = violation?.get("message")?.jsonStringOrNull()
+ if (rule == null || kind == null || message == null) {
+ throw DashSdkError.SerializationError("Malformed propertyConstraints violation: $json")
+ }
+ return PropertyConstraintViolation(rule, Kind.fromName(kind), message)
+ }
+ }
+}
+
+/** JSON readers for the two `propertyConstraints` payloads. */
+internal object PropertyConstraintJson {
+ private val prettyPrinter = Json { prettyPrint = true }
+
+ /** The JSON value [text] holds, [JsonNull] for `null`. */
+ fun parse(text: String): JsonElement = try {
+ Json.parseToJsonElement(text)
+ } catch (e: SerializationException) {
+ throw DashSdkError.SerializationError("Not JSON: $text", e)
+ }
+
+ /** [value] as compact JSON text with sorted keys. */
+ fun compact(value: JsonElement): String = sortedKeys(value).toString()
+
+ /** [value] as indented JSON text with sorted keys. */
+ fun pretty(value: JsonElement): String =
+ prettyPrinter.encodeToString(JsonElement.serializer(), sortedKeys(value))
+
+ private fun sortedKeys(value: JsonElement): JsonElement = when (value) {
+ is JsonObject -> JsonObject(
+ value.entries
+ .sortedBy { it.key }
+ .associateTo(LinkedHashMap()) { (key, element) -> key to sortedKeys(element) },
+ )
+ is JsonArray -> JsonArray(value.map(::sortedKeys))
+ is JsonPrimitive -> value
+ }
+}
+
+/** The string a JSON string holds; `null` for any other JSON value. */
+private fun JsonElement.jsonStringOrNull(): String? =
+ (this as? JsonPrimitive)?.takeIf { it.isString }?.content
+
+/**
+ * The boolean a JSON `true` or `false` holds; `null` for any other JSON value,
+ * the string `"true"` included.
+ */
+private fun JsonElement.jsonBooleanOrNull(): Boolean? =
+ (this as? JsonPrimitive)?.takeIf { !it.isString }?.booleanOrNull
diff --git a/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/queries/PlatformQueries.kt b/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/queries/PlatformQueries.kt
index e019bd7323c..dac3171d26e 100644
--- a/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/queries/PlatformQueries.kt
+++ b/packages/kotlin-sdk/sdk/src/main/kotlin/org/dashfoundation/dashsdk/queries/PlatformQueries.kt
@@ -9,6 +9,7 @@ import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonObject
import org.dashfoundation.dashsdk.Sdk
+import org.dashfoundation.dashsdk.errors.DashSdkError
import org.dashfoundation.dashsdk.errors.mapNativeErrors
import org.dashfoundation.dashsdk.ffi.NativeCleaner
import org.dashfoundation.dashsdk.ffi.QueriesNative
@@ -658,6 +659,84 @@ class Contracts internal constructor(private val sdk: Sdk) {
)
}
}
+
+ /**
+ * The `propertyConstraints` rules (protocol version 14) of [documentType],
+ * in name order (the order consensus checks them in). Empty for a type
+ * declaring none, and for every type while this SDK's protocol version is
+ * below 14. Port of Swift's `SDK.documentPropertyConstraints`.
+ *
+ * Each rule lists the properties it reads and how
+ * ([DocumentPropertyConstraint.reads]), whether it reads the owner
+ * ([DocumentPropertyConstraint.readsOwner]), the system times and
+ * heights it reads ([DocumentPropertyConstraint.readsSystem]) and the
+ * `countOf` and `sumOf` totals it reads ([DocumentPropertyConstraint.readsTotals]).
+ *
+ * [serializedContract] is the contract's platform serialization, the bytes
+ * kept beside a fetched contract ([ContractWithSerialization.binarySerialization],
+ * `DataContractEntity.binarySerialization`). Rust reads it at this SDK's
+ * protocol version; no network call.
+ *
+ * @throws DashSdkError.NotFound for a document type the contract does not declare.
+ * @throws DashSdkError.SerializationError for bytes that are not a contract.
+ * @throws DashSdkError.InvalidParameter for empty contract bytes.
+ */
+ suspend fun propertyConstraints(
+ serializedContract: ByteArray,
+ documentType: String,
+ ): List = sdk.queryGate.op {
+ val json = mapNativeErrors {
+ QueriesNative.dataContractGetPropertyConstraints(
+ sdk.handle,
+ serializedContract,
+ documentType,
+ )
+ } ?: throw DashSdkError.InternalError("No propertyConstraints rules returned")
+ DocumentPropertyConstraint.listFromJson(json)
+ }
+
+ /**
+ * The first `propertyConstraints` rule a document to create would break,
+ * or `null` when it meets them all (always so while this SDK's protocol
+ * version is below 14). Port of Swift's
+ * `SDK.checkDocumentPropertyConstraints`.
+ *
+ * [propertiesJson] is the properties JSON the document would be created
+ * with (what `DocumentTransactions.create` takes) and [ownerId] the 32-byte
+ * identity that would own it, which `$ownerId` reads. Rust builds the
+ * document the create path builds and judges it with the check consensus
+ * runs; nothing but the rules is checked. The device clock stands in for
+ * the block time the create records (`$createdAt`, `$updatedAt`,
+ * `$transferredAt`), so a rule comparing one is judged as of now; a rule
+ * reading a block height (`$createdAtBlockHeight`, ...) is not judged,
+ * since the height is unknown until the block, nor is a rule reading a
+ * `countOf` or `sumOf` total ([DocumentPropertyConstraint.readsTotals]),
+ * which the platform reads from state when the document is sent; consensus
+ * may still refuse the document for either. [serializedContract] is as for
+ * [propertyConstraints]; no network call.
+ *
+ * @throws DashSdkError.InvalidParameter for empty contract bytes, an owner
+ * id that is not 32 bytes, or properties that are not a JSON object.
+ * @throws DashSdkError.NotFound for a document type the contract does not declare.
+ * @throws DashSdkError.SerializationError for bytes that are not a contract.
+ */
+ suspend fun checkPropertyConstraints(
+ serializedContract: ByteArray,
+ documentType: String,
+ propertiesJson: String,
+ ownerId: ByteArray,
+ ): PropertyConstraintViolation? = sdk.queryGate.op {
+ val json = mapNativeErrors {
+ QueriesNative.dataContractCheckPropertyConstraints(
+ sdk.handle,
+ serializedContract,
+ documentType,
+ propertiesJson,
+ ownerId,
+ )
+ } ?: throw DashSdkError.InternalError("No propertyConstraints verdict returned")
+ PropertyConstraintViolation.fromJson(json)
+ }
}
/**
diff --git a/packages/kotlin-sdk/sdk/src/test/kotlin/org/dashfoundation/dashsdk/errors/DashSdkErrorTest.kt b/packages/kotlin-sdk/sdk/src/test/kotlin/org/dashfoundation/dashsdk/errors/DashSdkErrorTest.kt
index 1254c322813..7129c55f633 100644
--- a/packages/kotlin-sdk/sdk/src/test/kotlin/org/dashfoundation/dashsdk/errors/DashSdkErrorTest.kt
+++ b/packages/kotlin-sdk/sdk/src/test/kotlin/org/dashfoundation/dashsdk/errors/DashSdkErrorTest.kt
@@ -521,4 +521,36 @@ class DashSdkErrorTest {
// An SDK error raised on the Kotlin side has no native cause at all.
assertNull(DashSdkError.InvalidParameter("bad argument").consensusError)
}
+
+ @Test
+ fun shouldSurfaceTheConsensusErrorOfAStateTransitionPlatformRejected() {
+ // What the JNI bridge throws for an rs-sdk-ffi DashSDKError Platform
+ // refused under a propertyConstraints rule: ProtocolError (5),
+ // consensus code 10422, kind Basic (1).
+ val native = DashSDKException(
+ 5,
+ "Protocol error: document violates propertyConstraints rule 0",
+ 10422,
+ 1,
+ )
+ val mapped = DashSdkError.fromNative(native)
+
+ // The consensus error rides along; the type and message are what the
+ // code alone maps to.
+ assertTrue(mapped is DashSdkError.ProtocolError)
+ assertEquals(native.message, mapped.message)
+ assertEquals(
+ PlatformConsensusError(10422, ConsensusErrorKind.BASIC),
+ mapped.consensusError,
+ )
+ }
+
+ @Test
+ fun shouldHaveNoConsensusErrorForAnSdkFailureThatWasNotARejection() {
+ val plain = DashSdkError.fromNative(
+ DashSDKException(99, "Internal error: Failed to mint token and wait: timed out"),
+ )
+ assertTrue(plain is DashSdkError.InternalError)
+ assertNull(plain.consensusError)
+ }
}
diff --git a/packages/kotlin-sdk/sdk/src/test/kotlin/org/dashfoundation/dashsdk/queries/DocumentPropertyConstraintsTest.kt b/packages/kotlin-sdk/sdk/src/test/kotlin/org/dashfoundation/dashsdk/queries/DocumentPropertyConstraintsTest.kt
new file mode 100644
index 00000000000..d5354a0ec7e
--- /dev/null
+++ b/packages/kotlin-sdk/sdk/src/test/kotlin/org/dashfoundation/dashsdk/queries/DocumentPropertyConstraintsTest.kt
@@ -0,0 +1,615 @@
+package org.dashfoundation.dashsdk.queries
+
+import kotlinx.serialization.json.Json
+import org.dashfoundation.dashsdk.errors.DashSdkError
+import org.junit.Assert.assertEquals
+import org.junit.Assert.assertNull
+import org.junit.Assert.assertThrows
+import org.junit.Assert.assertTrue
+import org.junit.Test
+
+/**
+ * Coverage for the decoding of what the two protocol-version-14
+ * `propertyConstraints` natives return
+ * (`dash_sdk_data_contract_get_property_constraints` and
+ * `dash_sdk_data_contract_check_property_constraints`).
+ *
+ * Rust parses and evaluates the rules (rs-sdk-ffi's own tests cover every
+ * rule family and violation); Kotlin only decodes, so these tests pin the
+ * JSON shapes, which match wasm-dpp2's and the Swift SDK's key for key
+ * (`DocumentPropertyConstraintsTests.swift` holds the same cases). Pure JVM:
+ * no native library is loaded.
+ */
+class DocumentPropertyConstraintsTest {
+
+ /**
+ * The rules of an `offer` type as the FFI reports them (rs-sdk-ffi's
+ * `should_list_every_rule_in_name_order_with_what_it_reads`), abridged to
+ * three rules.
+ */
+ private val rulesJson = """
+ [
+ {
+ "name": "closedNeedsClosedAt",
+ "readsOwner": false,
+ "reads": [
+ { "kind": "text", "path": "status" },
+ { "kind": "presence", "path": "closedAt" }
+ ],
+ "readsSystem": [],
+ "rule": {
+ "ifThen": [
+ { "equal": ["status", { "const": "closed" }] },
+ { "present": "closedAt" }
+ ]
+ }
+ },
+ {
+ "name": "perUnitFee",
+ "readsOwner": false,
+ "reads": [
+ { "kind": "value", "path": "price" },
+ { "kind": "value", "path": "fee" }
+ ],
+ "readsSystem": [],
+ "rule": { "greaterThanOrEqual": [{ "divide": ["price", "fee"] }, 1] }
+ },
+ {
+ "name": "sellerIsOwner",
+ "readsOwner": true,
+ "reads": [
+ { "kind": "presence", "path": "sellerId" },
+ { "kind": "identifier", "path": "sellerId" }
+ ],
+ "readsSystem": [],
+ "rule": {
+ "anyOf": [{ "absent": "sellerId" }, { "equal": ["sellerId", "${'$'}ownerId"] }]
+ }
+ }
+ ]
+ """.trimIndent()
+
+ /**
+ * Rules reading sizes and the elements of an array, as the FFI reports
+ * them (wasm-dpp2's `DocumentPropertyConstraints.spec.ts` holds the same
+ * rules): a `byteLength` operand reads a string's size, a `count` operand
+ * an array's items, and a `contains` the array it looks in.
+ */
+ private val sizeAndElementRulesJson = """
+ [
+ {
+ "name": "notUsed",
+ "readsOwner": false,
+ "reads": [{ "kind": "elements", "path": "labels" }],
+ "readsSystem": [],
+ "rule": { "not": { "contains": ["labels", { "const": "used" }] } }
+ },
+ {
+ "name": "tagsWithinLimit",
+ "readsOwner": false,
+ "reads": [
+ { "kind": "count", "path": "tags" },
+ { "kind": "value", "path": "maxTags" }
+ ],
+ "readsSystem": [],
+ "rule": { "lessThanOrEqual": [{ "count": "tags" }, "maxTags"] }
+ },
+ {
+ "name": "titleBytes",
+ "readsOwner": false,
+ "reads": [{ "kind": "length", "path": "title" }],
+ "readsSystem": [],
+ "rule": { "lessThanOrEqual": [{ "byteLength": "title" }, 12] }
+ }
+ ]
+ """.trimIndent()
+
+ /**
+ * Rules reading system times and heights (rs-sdk-ffi's
+ * `should_read_the_clock_for_system_times_and_skip_block_heights`), plus
+ * one reading an update time twice and a Core height, in declared order.
+ */
+ private val systemRulesJson = """
+ [
+ {
+ "name": "endsAfterCreation",
+ "readsOwner": false,
+ "reads": [{ "kind": "value", "path": "endsAt" }],
+ "readsSystem": ["${'$'}createdAt"],
+ "rule": { "greaterThan": ["endsAt", "${'$'}createdAt"] }
+ },
+ {
+ "name": "listedAfterHeight10",
+ "readsOwner": false,
+ "reads": [],
+ "readsSystem": ["${'$'}createdAtBlockHeight"],
+ "rule": { "greaterThanOrEqual": ["${'$'}createdAtBlockHeight", 10] }
+ },
+ {
+ "name": "settledAfterTransfer",
+ "readsOwner": false,
+ "reads": [],
+ "readsSystem": [
+ "${'$'}updatedAt",
+ "${'$'}transferredAtCoreBlockHeight",
+ "${'$'}updatedAt"
+ ],
+ "rule": {
+ "allOf": [
+ { "greaterThan": ["${'$'}updatedAt", 0] },
+ { "greaterThan": ["${'$'}transferredAtCoreBlockHeight", 0] },
+ { "lessThan": ["${'$'}updatedAt", 4102444800000] }
+ ]
+ }
+ }
+ ]
+ """.trimIndent()
+
+ // Rules
+
+ @Test
+ fun `should decode the rules in order with what they read`() {
+ val rules = DocumentPropertyConstraint.listFromJson(rulesJson)
+
+ assertEquals(listOf("closedNeedsClosedAt", "perUnitFee", "sellerIsOwner"), rules.map { it.name })
+ assertEquals(
+ listOf(
+ PropertyConstraintRead("status", PropertyConstraintRead.Kind.Text),
+ PropertyConstraintRead("closedAt", PropertyConstraintRead.Kind.Presence),
+ ),
+ rules[0].reads,
+ )
+ assertEquals(
+ listOf(PropertyConstraintRead.Kind.Value, PropertyConstraintRead.Kind.Value),
+ rules[1].reads.map { it.kind },
+ )
+ assertEquals(
+ listOf(PropertyConstraintRead.Kind.Presence, PropertyConstraintRead.Kind.Identifier),
+ rules[2].reads.map { it.kind },
+ )
+ assertEquals(listOf(false, false, true), rules.map { it.readsOwner })
+ assertEquals(List(3) { emptyList() }, rules.map { it.readsSystem })
+ }
+
+ @Test
+ fun `should decode size and element reads as length count and elements`() {
+ val rules = DocumentPropertyConstraint.listFromJson(sizeAndElementRulesJson)
+
+ assertEquals(listOf("notUsed", "tagsWithinLimit", "titleBytes"), rules.map { it.name })
+ assertEquals(
+ listOf(PropertyConstraintRead("labels", PropertyConstraintRead.Kind.Elements)),
+ rules[0].reads,
+ )
+ assertEquals(
+ listOf(
+ PropertyConstraintRead("tags", PropertyConstraintRead.Kind.Count),
+ PropertyConstraintRead("maxTags", PropertyConstraintRead.Kind.Value),
+ ),
+ rules[1].reads,
+ )
+ assertEquals(
+ listOf(PropertyConstraintRead("title", PropertyConstraintRead.Kind.Length)),
+ rules[2].reads,
+ )
+ assertEquals("""{"lessThanOrEqual":[{"byteLength":"title"},12]}""", rules[2].ruleJson)
+ }
+
+ /** The names come through as Rust reports them, in declared order, a repeat kept. */
+ @Test
+ fun `should decode the system times and heights each rule reads`() {
+ val rules = DocumentPropertyConstraint.listFromJson(systemRulesJson)
+
+ assertEquals(
+ listOf(
+ listOf("${'$'}createdAt"),
+ listOf("${'$'}createdAtBlockHeight"),
+ listOf("${'$'}updatedAt", "${'$'}transferredAtCoreBlockHeight", "${'$'}updatedAt"),
+ ),
+ rules.map { it.readsSystem },
+ )
+ assertEquals(listOf(PropertyConstraintRead("endsAt", PropertyConstraintRead.Kind.Value)), rules[0].reads)
+ assertEquals(emptyList(), rules[1].reads)
+ assertEquals(listOf(false, false, false), rules.map { it.readsOwner })
+ }
+
+ /**
+ * An `ifThenElse` reads what every branch reads, whichever a document
+ * takes: the descriptor rs-sdk-ffi's
+ * `should_report_every_branch_of_an_if_then_else_and_judge_the_one_taken`
+ * reports, the owner read in the then branch and the creation time in the
+ * else branch.
+ */
+ @Test
+ fun `should decode what every branch of an ifThenElse reads`() {
+ val rule = DocumentPropertyConstraint.listFromJson(
+ """
+ [
+ {
+ "name": "openEndedSoldByOwner",
+ "readsOwner": true,
+ "reads": [
+ { "kind": "presence", "path": "endsAt" },
+ { "kind": "identifier", "path": "sellerId" },
+ { "kind": "value", "path": "endsAt" }
+ ],
+ "readsSystem": ["${'$'}createdAt"],
+ "rule": {
+ "ifThenElse": [
+ { "absent": "endsAt" },
+ { "equal": ["sellerId", "${'$'}ownerId"] },
+ { "greaterThan": ["endsAt", "${'$'}createdAt"] }
+ ]
+ }
+ }
+ ]
+ """.trimIndent(),
+ ).single()
+
+ assertEquals("openEndedSoldByOwner", rule.name)
+ assertEquals(
+ listOf(
+ PropertyConstraintRead("endsAt", PropertyConstraintRead.Kind.Presence),
+ PropertyConstraintRead("sellerId", PropertyConstraintRead.Kind.Identifier),
+ PropertyConstraintRead("endsAt", PropertyConstraintRead.Kind.Value),
+ ),
+ rule.reads,
+ )
+ assertTrue(rule.readsOwner)
+ assertEquals(listOf("${'$'}createdAt"), rule.readsSystem)
+ assertEquals(
+ """{"ifThenElse":[{"absent":"endsAt"},{"equal":["sellerId","${'$'}ownerId"]},""" +
+ """{"greaterThan":["endsAt","${'$'}createdAt"]}]}""",
+ rule.ruleJson,
+ )
+ }
+
+ /** A native library built before `readsSystem` leaves the key out. */
+ @Test
+ fun `should read a rule without readsSystem as reading no system value`() {
+ val rules = DocumentPropertyConstraint.listFromJson(
+ """[{"name":"r","readsOwner":false,"reads":[],"rule":{"present":"a"}}]""",
+ )
+
+ assertEquals(emptyList(), rules.single().readsSystem)
+ assertEquals(
+ DocumentPropertyConstraint("r", """{"present":"a"}""", emptyList(), readsOwner = false),
+ rules.single(),
+ )
+ }
+
+ /**
+ * The rule is kept as the JSON the schema declares, compact with sorted
+ * keys: array order, and so operand order, is untouched.
+ */
+ @Test
+ fun `should keep each rule as its declared JSON`() {
+ val rules = DocumentPropertyConstraint.listFromJson(rulesJson)
+
+ assertEquals(
+ """{"ifThen":[{"equal":["status",{"const":"closed"}]},{"present":"closedAt"}]}""",
+ rules[0].ruleJson,
+ )
+ assertEquals("""{"greaterThanOrEqual":[{"divide":["price","fee"]},1]}""", rules[1].ruleJson)
+ assertEquals(
+ """{"anyOf":[{"absent":"sellerId"},{"equal":["sellerId","${'$'}ownerId"]}]}""",
+ rules[2].ruleJson,
+ )
+ }
+
+ @Test
+ fun `should sort the keys of a rule declaring several`() {
+ val rules = DocumentPropertyConstraint.listFromJson(
+ """[{"name":"r","readsOwner":false,"reads":[],"rule":{"b":[2],"a":{"d":1,"c":0}}}]""",
+ )
+
+ assertEquals("""{"a":{"c":0,"d":1},"b":[2]}""", rules.single().ruleJson)
+ }
+
+ @Test
+ fun `should indent the same rule for display`() {
+ val rule = DocumentPropertyConstraint.listFromJson(rulesJson).first()
+
+ val pretty = rule.prettyRuleJson
+ assertTrue(pretty, pretty.contains("\n"))
+ assertEquals(Json.parseToJsonElement(rule.ruleJson), Json.parseToJsonElement(pretty))
+ }
+
+ @Test
+ fun `should show rule text that does not parse as it is`() {
+ val rule = DocumentPropertyConstraint("r", "not json", emptyList(), readsOwner = false)
+
+ assertEquals("not json", rule.prettyRuleJson)
+ }
+
+ @Test
+ fun `should decode no rules to an empty list`() {
+ assertEquals(emptyList(), DocumentPropertyConstraint.listFromJson("[]"))
+ }
+
+ /** A kind added by a later protocol version is kept by name rather than failing the whole list. */
+ @Test
+ fun `should keep an unknown read kind by its name`() {
+ val rules = DocumentPropertyConstraint.listFromJson(
+ """
+ [{ "name": "r", "rule": { "present": "a" }, "readsOwner": false,
+ "reads": [{ "path": "a", "kind": "somethingNew" }] }]
+ """.trimIndent(),
+ )
+
+ val kind = rules.single().reads.single().kind
+ assertEquals(PropertyConstraintRead.Kind.Other("somethingNew"), kind)
+ assertEquals("somethingNew", kind.name)
+ }
+
+ @Test
+ fun `should round trip every read kind name`() {
+ val names = listOf("value", "presence", "text", "identifier", "length", "count", "elements")
+ val kinds = listOf(
+ PropertyConstraintRead.Kind.Value,
+ PropertyConstraintRead.Kind.Presence,
+ PropertyConstraintRead.Kind.Text,
+ PropertyConstraintRead.Kind.Identifier,
+ PropertyConstraintRead.Kind.Length,
+ PropertyConstraintRead.Kind.Count,
+ PropertyConstraintRead.Kind.Elements,
+ )
+
+ assertEquals(kinds, names.map(PropertyConstraintRead.Kind::fromName))
+ assertEquals(names, kinds.map { it.name })
+ }
+
+ @Test
+ fun `should refuse malformed rules`() {
+ val malformed = listOf(
+ "not json",
+ """{"name": "r"}""",
+ """[{"name": "r", "rule": {"present": "a"}, "reads": []}]""",
+ // A number is not a boolean
+ """[{"name": "r", "rule": {"present": "a"}, "reads": [], "readsOwner": 1}]""",
+ // Nor is a string
+ """[{"name": "r", "rule": {"present": "a"}, "reads": [], "readsOwner": "true"}]""",
+ // A name must be a string
+ """[{"name": 7, "rule": {"present": "a"}, "reads": [], "readsOwner": false}]""",
+ """[{"name": "r", "rule": {"present": "a"}, "reads": [{"path": "a"}], "readsOwner": false}]""",
+ """[7]""",
+ )
+ for (json in malformed) {
+ assertThrows(json, DashSdkError.SerializationError::class.java) {
+ DocumentPropertyConstraint.listFromJson(json)
+ }
+ }
+ }
+
+ /** Present, `readsSystem` must be an array of strings; only a missing key means none. */
+ @Test
+ fun `should refuse a malformed readsSystem`() {
+ val malformed = listOf(
+ // A single name is not an array
+ "\"\$createdAt\"",
+ "null",
+ "true",
+ """{"${'$'}createdAt": true}""",
+ // Each entry must be a string
+ "[1]",
+ "[null]",
+ """["${'$'}createdAt", 7]""",
+ """[["${'$'}createdAt"]]""",
+ )
+ for (readsSystem in malformed) {
+ val json =
+ """[{"name":"r","readsOwner":false,"reads":[],"readsSystem":$readsSystem,"rule":{"present":"a"}}]"""
+ assertThrows(json, DashSdkError.SerializationError::class.java) {
+ DocumentPropertyConstraint.listFromJson(json)
+ }
+ }
+ }
+
+ /**
+ * The totals rules read, as rs-sdk-ffi's
+ * `should_list_the_totals_a_rule_reads_and_leave_it_unjudged` reports them:
+ * a whole-type count, a count by owner, a sum by category, and none.
+ */
+ private val totalRulesJson = """
+ [
+ {
+ "name": "allListings",
+ "readsOwner": false,
+ "reads": [],
+ "readsSystem": [],
+ "readsTotals": [{ "kind": "countOf", "documentType": "listing", "filter": [] }],
+ "rule": { "lessThan": [{ "countOf": ["listing"] }, 1000] }
+ },
+ {
+ "name": "atMostTwoPerOwner",
+ "readsOwner": true,
+ "reads": [],
+ "readsSystem": [],
+ "readsTotals": [
+ { "kind": "countOf", "documentType": "listing", "filter": ["${'$'}ownerId"] }
+ ],
+ "rule": {
+ "lessThanOrEqual": [{ "countOf": ["listing", { "${'$'}ownerId": "${'$'}ownerId" }] }, 2]
+ }
+ },
+ {
+ "name": "categoryBudget",
+ "readsOwner": false,
+ "reads": [{ "kind": "value", "path": "category" }],
+ "readsSystem": [],
+ "readsTotals": [
+ {
+ "kind": "sumOf",
+ "documentType": "listing",
+ "property": "price",
+ "filter": ["category"]
+ }
+ ],
+ "rule": {
+ "lessThanOrEqual": [{ "sumOf": ["listing", "price", { "category": "category" }] }, 250]
+ }
+ },
+ {
+ "name": "priceCap",
+ "readsOwner": false,
+ "reads": [{ "kind": "value", "path": "price" }],
+ "readsSystem": [],
+ "readsTotals": [],
+ "rule": { "lessThanOrEqual": ["price", 1000] }
+ }
+ ]
+ """.trimIndent()
+
+ // Totals
+
+ @Test
+ fun `should decode the totals each rule reads`() {
+ val rules = DocumentPropertyConstraint.listFromJson(totalRulesJson)
+
+ assertEquals(
+ listOf("allListings", "atMostTwoPerOwner", "categoryBudget", "priceCap"),
+ rules.map { it.name },
+ )
+ assertEquals(
+ listOf(
+ listOf(
+ PropertyConstraintTotalRead(
+ PropertyConstraintTotalRead.Kind.CountOf,
+ "listing",
+ null,
+ emptyList(),
+ ),
+ ),
+ listOf(
+ PropertyConstraintTotalRead(
+ PropertyConstraintTotalRead.Kind.CountOf,
+ "listing",
+ null,
+ listOf("${'$'}ownerId"),
+ ),
+ ),
+ listOf(
+ PropertyConstraintTotalRead(
+ PropertyConstraintTotalRead.Kind.SumOf,
+ "listing",
+ "price",
+ listOf("category"),
+ ),
+ ),
+ emptyList(),
+ ),
+ rules.map { it.readsTotals },
+ )
+ assertEquals(listOf(false, true, false, false), rules.map { it.readsOwner })
+ assertEquals(
+ listOf(PropertyConstraintRead("category", PropertyConstraintRead.Kind.Value)),
+ rules[2].reads,
+ )
+ }
+
+ /** A native library built before `readsTotals` leaves the key out. */
+ @Test
+ fun `should read a rule without readsTotals as reading no total`() {
+ val rules = DocumentPropertyConstraint.listFromJson(
+ """[{"name":"r","readsOwner":false,"reads":[],"readsSystem":[],"rule":{"present":"a"}}]""",
+ )
+
+ assertEquals(emptyList(), rules.single().readsTotals)
+ assertEquals(
+ DocumentPropertyConstraint("r", """{"present":"a"}""", emptyList(), readsOwner = false),
+ rules.single(),
+ )
+ }
+
+ @Test
+ fun `should round trip every total kind name and keep an unknown one`() {
+ for (kind in listOf(PropertyConstraintTotalRead.Kind.CountOf, PropertyConstraintTotalRead.Kind.SumOf)) {
+ assertEquals(kind, PropertyConstraintTotalRead.Kind.fromName(kind.name))
+ }
+ assertEquals(
+ PropertyConstraintTotalRead.Kind.Other("averageOf"),
+ PropertyConstraintTotalRead.Kind.fromName("averageOf"),
+ )
+ }
+
+ @Test
+ fun `should refuse a malformed readsTotals`() {
+ val malformed = listOf(
+ "null",
+ "{}",
+ "[1]",
+ "[null]",
+ // Every total names its kind, type and filter
+ """[{"documentType":"listing","filter":[]}]""",
+ """[{"kind":"countOf","filter":[]}]""",
+ """[{"kind":"countOf","documentType":"listing"}]""",
+ // The filter lists strings, and a property is one
+ """[{"kind":"countOf","documentType":"listing","filter":"${'$'}ownerId"}]""",
+ """[{"kind":"countOf","documentType":"listing","filter":[7]}]""",
+ """[{"kind":"sumOf","documentType":"listing","property":7,"filter":[]}]""",
+ """[{"kind":7,"documentType":"listing","filter":[]}]""",
+ )
+ for (readsTotals in malformed) {
+ val json =
+ """[{"name":"r","readsOwner":false,"reads":[],"readsSystem":[],"readsTotals":$readsTotals,"rule":{"present":"a"}}]"""
+ assertThrows(json, DashSdkError.SerializationError::class.java) {
+ DocumentPropertyConstraint.listFromJson(json)
+ }
+ }
+ }
+
+ // Violations
+
+ @Test
+ fun `should decode a violation`() {
+ val violation = PropertyConstraintViolation.fromJson(
+ """{ "rule": "perUnitFee", "violation": "DivisionByZero", "message": "it divides by zero" }""",
+ )
+
+ assertEquals(
+ PropertyConstraintViolation(
+ rule = "perUnitFee",
+ violation = PropertyConstraintViolation.Kind.DivisionByZero,
+ message = "it divides by zero",
+ ),
+ violation,
+ )
+ assertEquals(
+ "The document breaks the propertyConstraints rule \"perUnitFee\" (DivisionByZero): " +
+ "it divides by zero.",
+ violation?.description,
+ )
+ }
+
+ @Test
+ fun `should round trip every violation name`() {
+ val names = listOf("NotMet", "Overflow", "DivisionByZero", "NegativeExponent", "NotAnInteger")
+ val kinds = listOf(
+ PropertyConstraintViolation.Kind.NotMet,
+ PropertyConstraintViolation.Kind.Overflow,
+ PropertyConstraintViolation.Kind.DivisionByZero,
+ PropertyConstraintViolation.Kind.NegativeExponent,
+ PropertyConstraintViolation.Kind.NotAnInteger,
+ )
+
+ assertEquals(kinds, names.map(PropertyConstraintViolation.Kind::fromName))
+ assertEquals(names, kinds.map { it.name })
+ assertEquals(
+ PropertyConstraintViolation.Kind.Other("Later"),
+ PropertyConstraintViolation.Kind.fromName("Later"),
+ )
+ }
+
+ @Test
+ fun `should read null as every rule holding`() {
+ assertNull(PropertyConstraintViolation.fromJson("null"))
+ }
+
+ @Test
+ fun `should refuse malformed violations`() {
+ for (json in listOf("", "[]", "\"NotMet\"", """{"rule": "r", "violation": "NotMet"}""")) {
+ assertThrows(json, DashSdkError.SerializationError::class.java) {
+ PropertyConstraintViolation.fromJson(json)
+ }
+ }
+ }
+}
diff --git a/packages/masternode-reward-shares-contract/package.json b/packages/masternode-reward-shares-contract/package.json
index 21f0e656b6a..e343b7140c4 100644
--- a/packages/masternode-reward-shares-contract/package.json
+++ b/packages/masternode-reward-shares-contract/package.json
@@ -1,6 +1,6 @@
{
"name": "@dashevo/masternode-reward-shares-contract",
- "version": "4.2.0-beta.4",
+ "version": "4.2.0-beta.6",
"description": "A contract and helper scripts for reward sharing",
"scripts": {
"lint": "eslint .",
diff --git a/packages/moderation-charters-contract/package.json b/packages/moderation-charters-contract/package.json
index 43ddf490133..861ca22b07d 100644
--- a/packages/moderation-charters-contract/package.json
+++ b/packages/moderation-charters-contract/package.json
@@ -1,6 +1,6 @@
{
"name": "@dashevo/moderation-charters-contract",
- "version": "4.2.0-beta.4",
+ "version": "4.2.0-beta.6",
"description": "A system contract for the charters of elected moderation teams",
"scripts": {
"lint": "eslint .",
diff --git a/packages/platform-test-suite/package.json b/packages/platform-test-suite/package.json
index 55c2fc82bcf..1456de275a2 100644
--- a/packages/platform-test-suite/package.json
+++ b/packages/platform-test-suite/package.json
@@ -1,7 +1,7 @@
{
"name": "@dashevo/platform-test-suite",
"private": true,
- "version": "4.2.0-beta.4",
+ "version": "4.2.0-beta.6",
"description": "Dash Network end-to-end tests",
"scripts": {
"test": "yarn exec bin/test.sh",
diff --git a/packages/rs-dapi-client/Cargo.toml b/packages/rs-dapi-client/Cargo.toml
index da97e5b4f98..36043a61304 100644
--- a/packages/rs-dapi-client/Cargo.toml
+++ b/packages/rs-dapi-client/Cargo.toml
@@ -68,7 +68,11 @@ serde_json = { version = "1.0.140", optional = true }
chrono = { version = "0.4.38", features = ["serde"] }
[dev-dependencies]
-tokio = { version = "1.40", features = ["macros"] }
+tokio = { version = "1.40", features = ["macros", "rt", "test-util", "io-util"] }
+# In-memory HTTP/2 server and connector for the stalled-response-body tests.
+h2 = "0.4"
+hyper-util = { version = "0.1", features = ["tokio"] }
+tower = { version = "0.5", features = ["util"] }
[package.metadata.cargo-machete]
diff --git a/packages/rs-dapi-client/src/connection_pool.rs b/packages/rs-dapi-client/src/connection_pool.rs
index b8e7b74775a..bd826cc363f 100644
--- a/packages/rs-dapi-client/src/connection_pool.rs
+++ b/packages/rs-dapi-client/src/connection_pool.rs
@@ -11,13 +11,39 @@ use crate::{
Uri,
};
+/// Default capacity of the [ConnectionPool].
+pub(crate) const DEFAULT_POOL_CAPACITY: usize = 50;
+
/// ConnectionPool represents pool of connections to DAPI nodes.
///
/// It can be cloned and shared between threads.
/// Cloning the pool will create a new reference to the same pool.
#[derive(Debug, Clone)]
pub struct ConnectionPool {
- inner: Arc>>,
+ inner: Arc>,
+}
+
+#[derive(Debug)]
+struct PoolState {
+ connections: LruCache,
+ /// Generation of the connection pooled most recently.
+ generation: u64,
+}
+
+/// A pooled connection and the generation it was pooled at.
+#[derive(Debug)]
+struct Pooled {
+ generation: u64,
+ item: PoolItem,
+}
+
+/// Identity of a pooled connection: the client type, the node, and the
+/// connection-affecting settings (`None` when none were given).
+#[derive(Debug, Clone, PartialEq, Eq, Hash)]
+struct PoolKey {
+ prefix: PoolPrefix,
+ uri: String,
+ connection: Option,
}
impl ConnectionPool {
@@ -29,16 +55,17 @@ impl ConnectionPool {
/// Panics if the capacity is zero.
pub fn new(capacity: usize) -> Self {
Self {
- inner: Arc::new(Mutex::new(LruCache::new(
- capacity.try_into().expect("must be non-zero"),
- ))),
+ inner: Arc::new(Mutex::new(PoolState {
+ connections: LruCache::new(capacity.try_into().expect("must be non-zero")),
+ generation: 0,
+ })),
}
}
}
impl Default for ConnectionPool {
fn default() -> Self {
- Self::new(50)
+ Self::new(DEFAULT_POOL_CAPACITY)
}
}
@@ -56,7 +83,12 @@ impl ConnectionPool {
settings: Option<&AppliedRequestSettings>,
) -> Option {
let key = Self::key(prefix, uri, settings);
- self.inner.lock().expect("must lock").get(&key).cloned()
+ self.inner
+ .lock()
+ .expect("must lock")
+ .connections
+ .get(&key)
+ .map(|pooled| pooled.item.clone())
}
/// Get value from cache or create it using provided closure.
@@ -74,38 +106,110 @@ impl ConnectionPool {
settings: Option<&AppliedRequestSettings>,
create: impl FnOnce() -> Result,
) -> Result {
- if let Some(cli) = self.get(prefix, uri, settings) {
- return Ok(cli);
- }
+ self.get_or_create_with_generation(prefix, uri, settings, create)
+ .map(|(item, _)| item)
+ }
- let cli = create();
- if let Ok(cli) = &cli {
- self.put(uri, settings, cli.clone());
+ /// Like [ConnectionPool::get_or_create], and also returns the generation
+ /// the returned connection was pooled at (see [ConnectionPool::generation]).
+ ///
+ /// The generation is read under the same lock that finds or stores the
+ /// connection, so it is the returned connection's own even when other
+ /// threads replace it right afterwards.
+ pub fn get_or_create_with_generation(
+ &self,
+ prefix: PoolPrefix,
+ uri: &Uri,
+ settings: Option<&AppliedRequestSettings>,
+ create: impl FnOnce() -> Result,
+ ) -> Result<(PoolItem, u64), E> {
+ let key = Self::key(prefix, uri, settings);
+ let cached = self
+ .inner
+ .lock()
+ .expect("must lock")
+ .connections
+ .get(&key)
+ .map(|pooled| (pooled.item.clone(), pooled.generation));
+ if let Some(cached) = cached {
+ return Ok(cached);
}
- cli
+
+ let item = create()?;
+ let generation = self.put_with_generation(uri, settings, item.clone());
+ Ok((item, generation))
}
/// Put item into the pool for the given uri and settings.
pub fn put(&self, uri: &Uri, settings: Option<&AppliedRequestSettings>, value: PoolItem) {
+ self.put_with_generation(uri, settings, value);
+ }
+
+ /// Put item into the pool and return the generation it was pooled at.
+ fn put_with_generation(
+ &self,
+ uri: &Uri,
+ settings: Option<&AppliedRequestSettings>,
+ value: PoolItem,
+ ) -> u64 {
let key = Self::key(&value, uri, settings);
- self.inner.lock().expect("must lock").put(key, value);
+ let mut state = self.inner.lock().expect("must lock");
+ state.generation += 1;
+ let generation = state.generation;
+ state.connections.put(
+ key,
+ Pooled {
+ generation,
+ item: value,
+ },
+ );
+ generation
+ }
+
+ /// Generation of the connection pooled most recently. Every connection
+ /// put into the pool afterwards gets a higher generation.
+ pub fn generation(&self) -> u64 {
+ self.inner.lock().expect("must lock").generation
+ }
+
+ /// Drop every connection to `uri` pooled at or before `generation`,
+ /// whatever its prefix and connection settings.
+ ///
+ /// A request that misses its deadline may have been sent over a half-open
+ /// connection: the network path died after the request left, and nothing
+ /// on the idle channel would ever notice. Keeping it pooled would stall
+ /// the next request sent to the same node, so the executor evicts it and
+ /// the next request dials a fresh connection.
+ ///
+ /// Connections pooled after `generation` stay. They were dialed after the
+ /// timed-out attempt took its connection, for example by a concurrent
+ /// request that already timed out on the same node and reconnected.
+ pub fn remove_uri(&self, uri: &Uri, generation: u64) {
+ let uri = uri.to_string();
+ let mut state = self.inner.lock().expect("must lock");
+ let stale: Vec = state
+ .connections
+ .iter()
+ .filter(|(key, pooled)| key.uri == uri && pooled.generation <= generation)
+ .map(|(key, _)| key.clone())
+ .collect();
+ for key in stale {
+ state.connections.pop(&key);
+ }
}
fn key>(
class: C,
uri: &Uri,
settings: Option<&AppliedRequestSettings>,
- ) -> String {
- let prefix: PoolPrefix = class.into();
+ ) -> PoolKey {
// Only connection-affecting settings participate in the key (see
// `AppliedRequestSettings::connection_key`), so requests differing only
// in per-request knobs (timeout, retries, banning) share a connection.
- // The settings segment is always present (and contains no `:`), so the
- // two branches cannot produce colliding shapes even for a URI whose
- // path mimics a key fragment.
- match settings {
- Some(settings) => format!("{}:{}:{}", prefix, uri, settings.connection_key()),
- None => format!("{}:{}:none", prefix, uri),
+ PoolKey {
+ prefix: class.into(),
+ uri: uri.to_string(),
+ connection: settings.map(AppliedRequestSettings::connection_key),
}
}
}
@@ -161,6 +265,7 @@ impl From for CoreGrpcClient {
}
/// Prefix for the item in the pool. Used to distinguish between Core and Platform clients.
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum PoolPrefix {
Core,
Platform,
@@ -399,6 +504,106 @@ mod tests {
assert!(result.is_some());
}
+ #[tokio::test]
+ async fn should_remove_every_pooled_connection_to_an_uri() {
+ let pool = ConnectionPool::new(10);
+ let uri = test_uri();
+ let longer_port = Uri::from_str("http://127.0.0.1:30001").unwrap();
+ let connect_timeout = RequestSettings {
+ connect_timeout: Some(Duration::from_secs(3)),
+ ..RequestSettings::default()
+ }
+ .finalize();
+
+ pool.put(&uri, None, make_platform_pool_item());
+ pool.put(&uri, None, make_core_pool_item());
+ pool.put(&uri, Some(&connect_timeout), make_platform_pool_item());
+ pool.put(&longer_port, None, make_platform_pool_item());
+
+ pool.remove_uri(&uri, pool.generation());
+
+ assert!(pool.get(PoolPrefix::Platform, &uri, None).is_none());
+ assert!(pool.get(PoolPrefix::Core, &uri, None).is_none());
+ assert!(pool
+ .get(PoolPrefix::Platform, &uri, Some(&connect_timeout))
+ .is_none());
+ assert!(
+ pool.get(PoolPrefix::Platform, &longer_port, None).is_some(),
+ "a URI that merely starts with the evicted one must stay pooled"
+ );
+ }
+
+ #[tokio::test]
+ async fn should_remove_only_the_exact_uri_when_another_extends_it_past_a_colon() {
+ let cases = [
+ ("http://node", "http://node:443"),
+ ("http://node/grpc", "http://node/grpc:8080"),
+ ];
+ for (evicted, kept) in cases {
+ let pool = ConnectionPool::new(10);
+ let evicted = Uri::from_str(evicted).unwrap();
+ let kept = Uri::from_str(kept).unwrap();
+ pool.put(&evicted, None, make_platform_pool_item());
+ pool.put(&kept, None, make_platform_pool_item());
+
+ pool.remove_uri(&evicted, pool.generation());
+
+ assert!(
+ pool.get(PoolPrefix::Platform, &evicted, None).is_none(),
+ "{evicted} must be evicted"
+ );
+ assert!(
+ pool.get(PoolPrefix::Platform, &kept, None).is_some(),
+ "{kept} must stay pooled when {evicted} is evicted"
+ );
+ }
+ }
+
+ #[tokio::test]
+ async fn should_keep_connections_pooled_after_the_given_generation() {
+ let pool = ConnectionPool::new(10);
+ let uri = test_uri();
+ pool.put(&uri, None, make_platform_pool_item());
+ let used_by_attempt = pool.generation();
+ // Replaced after the attempt took its connection.
+ pool.put(&uri, None, make_platform_pool_item());
+
+ pool.remove_uri(&uri, used_by_attempt);
+ assert!(
+ pool.get(PoolPrefix::Platform, &uri, None).is_some(),
+ "a connection pooled after the given generation must stay"
+ );
+
+ pool.remove_uri(&uri, pool.generation());
+ assert!(pool.get(PoolPrefix::Platform, &uri, None).is_none());
+ }
+
+ #[tokio::test]
+ async fn should_return_the_generation_of_the_connection_it_takes() {
+ let pool = ConnectionPool::new(10);
+ let uri = test_uri();
+
+ let (_, created) = pool
+ .get_or_create_with_generation(PoolPrefix::Platform, &uri, None, || {
+ Ok::<_, String>(make_platform_pool_item())
+ })
+ .unwrap();
+ assert_eq!(created, pool.generation());
+
+ let (_, taken) = pool
+ .get_or_create_with_generation(PoolPrefix::Platform, &uri, None, || {
+ Err("the pooled connection must be reused".to_string())
+ })
+ .unwrap();
+ pool.put(&uri, None, make_platform_pool_item());
+
+ assert_eq!(taken, created);
+ assert!(
+ taken < pool.generation(),
+ "a replacement pooled afterwards must get a higher generation"
+ );
+ }
+
#[tokio::test]
async fn test_connection_pool_clone_shares_data() {
let pool = ConnectionPool::new(10);
diff --git a/packages/rs-dapi-client/src/dapi_client.rs b/packages/rs-dapi-client/src/dapi_client.rs
index 6569b1ba4a3..5557af91361 100644
--- a/packages/rs-dapi-client/src/dapi_client.rs
+++ b/packages/rs-dapi-client/src/dapi_client.rs
@@ -4,12 +4,14 @@ use dapi_grpc::mock::Mockable;
use dapi_grpc::tonic::async_trait;
#[cfg(not(target_arch = "wasm32"))]
use dapi_grpc::tonic::transport::Certificate;
+#[cfg(not(target_arch = "wasm32"))]
+use dapi_grpc::tonic::Status;
use std::fmt::{Debug, Display};
use std::time::Duration;
use tracing::Instrument;
use crate::address_list::AddressListError;
-use crate::connection_pool::ConnectionPool;
+use crate::connection_pool::{ConnectionPool, DEFAULT_POOL_CAPACITY};
use crate::request_settings::AppliedRequestSettings;
use crate::transport::{self, TransportError};
use crate::{
@@ -117,14 +119,18 @@ pub struct DapiClient {
impl DapiClient {
/// Initialize new [DapiClient] and optionally override default settings.
+ ///
+ /// `address_list` may be empty; addresses added later to the shared list
+ /// (or a clone of it) are used by this client.
pub fn new(address_list: AddressList, settings: RequestSettings) -> Self {
- // multiply by 3 as we need to store core and platform addresses, and we want some spare capacity just in case
- let address_count = 3 * address_list.len();
+ // multiply by 3 as we need to store core and platform addresses, and we want some spare capacity just in case;
+ // never go below the default, as the list can be empty and addresses can be added later
+ let pool_capacity = (3 * address_list.len()).max(DEFAULT_POOL_CAPACITY);
Self {
address_list,
settings,
- pool: ConnectionPool::new(address_count),
+ pool: ConnectionPool::new(pool_capacity),
#[cfg(feature = "dump")]
dump_dir: None,
#[cfg(not(target_arch = "wasm32"))]
@@ -281,6 +287,25 @@ mod tests {
}
}
+ #[tokio::test]
+ async fn test_new_with_empty_address_list() {
+ let client = DapiClient::new(AddressList::new(), RequestSettings::default());
+ assert!(client.address_list().is_empty());
+
+ let request = dapi_grpc::platform::v0::GetIdentityRequest::default();
+ let err = client
+ .execute(request, RequestSettings::default())
+ .await
+ .expect_err("no addresses to execute the request on");
+ assert!(matches!(err.inner, DapiClientError::NoAvailableAddresses));
+
+ // The address list is shared, so addresses added later are visible to the client.
+ let mut address_list = client.address_list().clone();
+ assert!(address_list.add(mock_address()));
+ assert_eq!(client.get_live_addresses(), vec![mock_address()]);
+ // Execution on the added address: tests/empty_address_list.rs.
+ }
+
#[test]
fn test_can_retry_no_available_addresses() {
let err = DapiClientError::NoAvailableAddresses;
@@ -709,6 +734,325 @@ mod tests {
let display = format!("{}", err);
assert!(display.contains("address list error"));
}
+
+ /// Executor-level coverage for evicting the pooled connection of a node
+ /// whose attempt missed its deadline.
+ #[cfg(not(target_arch = "wasm32"))]
+ mod deadline_pool_eviction {
+ use super::*;
+ use crate::connection_pool::{PoolItem, PoolPrefix};
+ use crate::transport::{BoxFuture, PlatformGrpcClient};
+ use crate::Uri;
+ use dapi_grpc::tonic::transport::Channel;
+ use dapi_grpc::tonic::Code;
+ use std::sync::{Arc, Mutex};
+
+ /// Takes its connection from the executor's pool, as the real gRPC
+ /// clients do, so the pool holds an entry for every node dialed.
+ struct PooledClient {
+ uri: Uri,
+ }
+
+ impl PooledClient {
+ fn pooled(
+ uri: Uri,
+ settings: Option<&AppliedRequestSettings>,
+ pool: &ConnectionPool,
+ ) -> Result {
+ pool.get_or_create(PoolPrefix::Platform, &uri, settings, || {
+ Ok::<_, TransportError>(PoolItem::Platform(PlatformGrpcClient::new(
+ Channel::builder(uri.clone()).connect_lazy(),
+ )))
+ })?;
+ Ok(Self { uri })
+ }
+ }
+
+ impl TransportClient for PooledClient {
+ fn with_uri(uri: Uri, pool: &ConnectionPool) -> Result {
+ Self::pooled(uri, None, pool)
+ }
+
+ fn with_uri_and_settings(
+ uri: Uri,
+ settings: &AppliedRequestSettings,
+ pool: &ConnectionPool,
+ ) -> Result {
+ Self::pooled(uri, Some(settings), pool)
+ }
+ }
+
+ #[derive(Debug)]
+ struct Pong;
+
+ impl Mockable for Pong {}
+
+ /// Never answers on the first node it is sent to; answers at once
+ /// everywhere else.
+ #[derive(Clone, Debug, Default)]
+ struct StallFirstRequest {
+ stalled: Arc>>,
+ }
+
+ impl Mockable for StallFirstRequest {}
+
+ impl TransportRequest for StallFirstRequest {
+ type Client = PooledClient;
+ type Response = Pong;
+
+ const SETTINGS_OVERRIDES: RequestSettings = RequestSettings::default();
+
+ fn method_name(&self) -> &'static str {
+ "stall_first"
+ }
+
+ fn execute_transport<'c>(
+ self,
+ client: &'c mut Self::Client,
+ _settings: &AppliedRequestSettings,
+ ) -> BoxFuture<'c, Result> {
+ let mut stalled = self.stalled.lock().expect("stall lock");
+ let target = stalled.get_or_insert_with(|| client.uri.clone());
+ if *target == client.uri {
+ Box::pin(futures::future::pending())
+ } else {
+ Box::pin(async { Ok(Pong) })
+ }
+ }
+ }
+
+ #[tokio::test(start_paused = true)]
+ async fn should_evict_the_pooled_connection_of_a_node_that_missed_its_deadline() {
+ let request = StallFirstRequest::default();
+ let client = DapiClient::new(
+ "http://127.0.0.1:10001,http://127.0.0.1:10002"
+ .parse()
+ .expect("valid address list"),
+ RequestSettings::default(),
+ );
+
+ let response = client
+ .execute(request.clone(), RequestSettings::default())
+ .await
+ .expect("the other node must answer");
+
+ let stalled = request
+ .stalled
+ .lock()
+ .expect("stall lock")
+ .clone()
+ .expect("a node stalled");
+ // The executor's applied settings for these defaults (no CA
+ // certificate) produce the same pool key.
+ let settings = RequestSettings::default().finalize();
+ assert!(
+ client
+ .pool
+ .get(PoolPrefix::Platform, &stalled, Some(&settings))
+ .is_none(),
+ "the stalled node's connection must be evicted"
+ );
+ assert!(
+ client
+ .pool
+ .get(
+ PoolPrefix::Platform,
+ response.address.uri(),
+ Some(&settings)
+ )
+ .is_some(),
+ "the healthy node's connection must stay pooled"
+ );
+ }
+
+ /// Replaces its node's pooled connection, as a concurrent request
+ /// that timed out on the node and reconnected would, then never
+ /// answers.
+ #[derive(Clone, Debug)]
+ struct ReplaceConnectionThenStall {
+ pool: ConnectionPool,
+ }
+
+ impl Mockable for ReplaceConnectionThenStall {}
+
+ impl TransportRequest for ReplaceConnectionThenStall {
+ type Client = PooledClient;
+ type Response = Pong;
+
+ const SETTINGS_OVERRIDES: RequestSettings = RequestSettings::default();
+
+ fn method_name(&self) -> &'static str {
+ "replace_connection_then_stall"
+ }
+
+ fn execute_transport<'c>(
+ self,
+ client: &'c mut Self::Client,
+ settings: &AppliedRequestSettings,
+ ) -> BoxFuture<'c, Result> {
+ self.pool.put(
+ &client.uri,
+ Some(settings),
+ PoolItem::Platform(PlatformGrpcClient::new(
+ Channel::builder(client.uri.clone()).connect_lazy(),
+ )),
+ );
+ Box::pin(futures::future::pending())
+ }
+ }
+
+ #[tokio::test(start_paused = true)]
+ async fn should_keep_a_connection_pooled_after_the_attempt_took_its_own() {
+ let client = DapiClient::new(
+ "http://127.0.0.1:10001"
+ .parse()
+ .expect("valid address list"),
+ RequestSettings {
+ retries: Some(0),
+ ..RequestSettings::default()
+ },
+ );
+ let request = ReplaceConnectionThenStall {
+ pool: client.pool.clone(),
+ };
+
+ let error = client
+ .execute(request, RequestSettings::default())
+ .await
+ .expect_err("the only node never answers");
+
+ assert!(
+ matches!(
+ &error.inner,
+ DapiClientError::Transport(TransportError::Grpc(status))
+ if status.code() == Code::DeadlineExceeded
+ ),
+ "expected DeadlineExceeded, got {:?}",
+ error.inner
+ );
+ let uri = error.address.expect("the attempted node").uri().clone();
+ let settings = RequestSettings::default().finalize();
+ assert!(
+ client
+ .pool
+ .get(PoolPrefix::Platform, &uri, Some(&settings))
+ .is_some(),
+ "a connection pooled after the attempt took its own must stay pooled"
+ );
+ }
+
+ /// Takes its node's pooled connection, which another worker replaces
+ /// before the constructor returns.
+ struct ReplacedWhileBuildingClient;
+
+ impl ReplacedWhileBuildingClient {
+ fn build(
+ uri: Uri,
+ settings: Option<&AppliedRequestSettings>,
+ pool: &ConnectionPool,
+ ) -> Result<(Self, u64), TransportError> {
+ let connect = || {
+ Ok::<_, TransportError>(PoolItem::Platform(PlatformGrpcClient::new(
+ Channel::builder(uri.clone()).connect_lazy(),
+ )))
+ };
+ let (_, generation) = pool.get_or_create_with_generation(
+ PoolPrefix::Platform,
+ &uri,
+ settings,
+ connect,
+ )?;
+ // What a concurrent request that timed out and reconnected
+ // would pool.
+ pool.put(&uri, settings, connect()?);
+ Ok((Self, generation))
+ }
+ }
+
+ impl TransportClient for ReplacedWhileBuildingClient {
+ fn with_uri(uri: Uri, pool: &ConnectionPool) -> Result