Pipelined sync as the standard driver + Ironwood-first wallet UX + setup-reliability fixes - #7
Merged
Merged
Conversation
…er hook
Establishes the next major update (sync latency) on its own branch.
- docs/SYNC_OPTIMIZATION.md: the concrete roadmap — why we stay on the ECC
(Ironwood-capable) stack rather than swap to Warp/ZKool; what we already have
(tip birthday, subtree roots, spend-before-sync, configurable batch); the three
remaining levers (pipelining, adaptive batch, parallel decryption); the custom
pipelined-driver approach; and the safety gate + testnet validation before it
becomes default.
- SyncOptions { batch_size, pipelined } replaces the bare batch_size arg to
sync_group; command layer builds it from settings.
- Settings.experimental_pipelined_sync (off by default) reserves the opt-in.
Until the pipelined driver lands and is validated against the stock driver on
testnet, the flag safely falls back to zcash_client_backend::sync::run — no
half-built sync loop ever touches fund detection.
Next on this branch: implement the pipelined driver (prefetch download while
scanning) behind the flag, validate against the stock driver, then flip default.
34 core tests + full backend build + tsc green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wire the custom pipelined sync driver that the previous commit left as a tracing-only fallback. `run_pipelined`/`running_pipelined` in wallet.rs faithfully reproduce the upstream `zcash_client_backend::sync::run` control flow (subtree roots -> chain tip -> verify pass -> historic ranges, with reorg/continuity rewind and higher-priority-range restart), changing only how batches are fed: a producer task downloads each batch's compact blocks into memory plus the chain-state anchor and hands them over a bounded channel (capacity 2), so download runs up to two batches ahead of the CPU-bound scan. Same wallet state as the stock driver; only network/CPU overlap is new. Design choices that keep validation a clean equality check: - In-memory `MemBlockSource` per batch instead of the on-disk `FsCache`, so the pipelined path never contends the cache mutex or writes files, and a reorg rewinds only the db (nothing cached to truncate). - Fixed batch units identical to the stock driver (adaptive batch sizing is deferred to a follow-up); `split_scan_range` is unit-tested against the upstream step-7 splitter semantics. - Transparent-UTXO refresh omitted, matching the stock driver: our zcash_client_backend build does not enable `transparent-inputs` (group accounts are Orchard-only view keys), so neither driver performs it. Still gated by `Settings.experimental_pipelined_sync` (default false) and raced against the sync cancellation token; scanning is transactional per batch, so cancellation leaves the db consistent at a batch boundary. Stays off by default until testnet-validated against the stock driver per docs/SYNC_OPTIMIZATION.md. Uses tonic's native `stream.message()` (no futures-util dep) and lets the subtree-root types infer from the `put_*_subtree_roots` calls (no sapling dep). 47 core tests pass; full backend builds; no new clippy warnings. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A concrete testnet acceptance checklist for experimental_pipelined_sync: stock-driver baseline vs pipelined clean-state equality (balance, notes, history, height), incremental sync, cancellation/resume, reorg tolerance, send-after-sync, and the flag-off regression. Sign-off maps to the validation gate in docs/SYNC_OPTIMIZATION.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the memo-v1 "automated poll source" follow-up with the actual governance target: a full rebuild around ValarGroup Shielded Vote (live, ZK-based, on-chain vote chain). Records the user's confirmations — infra is live, FROST-compatible via governance PCZT into ZKP1, Ironwood-supported snapshots, pin production zcash_voting/pir-client at build time — and notes that voting.rs (memo v1) is expected to be scrapped. Includes the wallet-side flow and a de-risking scoping spike as the first step. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A lightwalletd that predates Ironwood (NU6.3) rejects the subtree-roots
request for the Ironwood pool with "invalid shielded protocol value", which
aborts the whole sync at its first step. The raw gRPC error gave no hint that
the cause is a server-capability gap, not a wallet bug.
`annotate_sync_error` now post-processes both sync drivers' results: when the
failure matches that signature it returns an actionable message ("This
lightwalletd server doesn't support Ironwood (NU6.3)… switch to an
Ironwood-capable server in the wallet's network settings"), keeping the raw
server text after an em-dash for debugging. The Groups sync box renders it via
SyncErrorView, showing the actionable headline in bold with the raw detail
dimmed beneath — the friendly message on top of the existing log output.
Unit tests cover the flagged case and pass-through of unrelated errors and
cancellation. Backend builds, tsc clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
wallet_sync cancelled the previous sync's token and immediately opened a new db connection, but cancellation is cooperative: the old sync keeps its connection (and, mid-batch, the SQLite write lock) until it returns at the next batch boundary. The new sync's first writes — put_*_subtree_roots — then raced the old writer and, on a large mainnet batch that held the lock past the 30s busy timeout, failed with "database is locked". The pipelined driver surfaces it more because it keeps the old sync writing more continuously. Add a per-group async lock (AppState.sync_locks) held across the whole sync_group call. A restarting sync cancels the old token, then *waits* on this lock until the cancelled sync fully exits and drops its connection before opening its own — so at most one writer touches a group's db at a time. Different groups still sync in parallel. Status reads already use a read-only (shared-lock) connection, so they don't contend. Driver-agnostic; helps the stock path too. 55 core tests pass; backend builds; no new clippy warnings. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Log per-batch download time (debug) and per-batch scan time + cumulative blocks/s (info) in the pipelined driver, so the download-vs-scan split is visible when diagnosing slow syncs. No behavior change. Confirms whether a slow sync is network-bound (pipelining helps) or CPU-bound in trial decryption + note-commitment tree updates (needs parallel decryption, which no published crate in the current Ironwood cohort provides). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Nothing was subscribing to the app's `tracing` output, so diagnostics (including the new sync timing logs) went nowhere. Add a global subscriber that formats every event to stdout AND an in-memory ring buffer (logbuf.rs, bounded at 3000 lines, honouring RUST_LOG; default info + our crates at debug). The buffer is process-only and never written to disk — wallet logs can contain addresses/amounts. New commands get_logs / clear_logs expose the buffer. Wallet settings gains a "Diagnostics log" card: live auto-refresh (2s), copy-all, refresh, clear, and a scroll-pinned monospace view — so a user can grab and share logs (e.g. the `pipelined scan: … blocks/s` lines) without a terminal. logbuf unit tests cover ring-buffer capping/order and line splitting. Backend builds, tsc clean, no new clippy warnings. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Brings the branch up to current main (active-wallet PR #4, log-viewer PR #5) so it can open a clean PR. Two conflicts resolved: - state.rs / wallet.rs: main added a global `sync_gate` (one wallet syncs at a time) via the active-wallet model; this branch had added a per-group `sync_locks` map to fix the "database is locked" race. The global gate is strictly stronger — only one sync ever runs — so it subsumes the per-group lock. Kept `sync_gate`, dropped `sync_locks` (and its now-unused Arc import). The db-lock fix is preserved: holding the gate across the whole sync_group call means a restarting sync can't race a cancelled one's open db connection. - Kept this branch's `SyncOptions { batch_size, pipelined }` and the experimental pipelined-sync path (the point of the branch); only the locking mechanism changed. The log-viewer commit (cherry-picked to main as PR #5) merges as identical content — no duplicate get_logs/clear_logs. Backend + tsc build clean; clippy unchanged at the 7-warning baseline. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The experimental pipelined sync driver was only togglable by hand-editing settings.json. Add a "Sync" card in Wallet Settings with an "Experimental pipelined sync" checkbox: - New set_experimental_pipelined_sync command (persists the flag; read at the start of each wallet_sync, so it takes effect on the next sync — no restart). - Registered in lib.rs; Settings interface + setExperimentalPipelinedSync IPC binding added. - SyncCard in Wallet.tsx reads the flag from settings and toggles it, with a note that it's off by default and applies on the next sync. - SYNC_PIPELINE_UAT.md: point the toggle steps at the checkbox (settings.json still works) and add A0a to test the checkbox wiring itself. Backend + tsc build clean; clippy unchanged at the 7-warning baseline. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
"Setting up the group's view-only wallet…" could spin forever with nothing in the log viewer: init_group_account emitted no tracing output, and its two unary RPCs (get_latest_block, get_tree_state) had no timeout — a server that accepts the TCP connection but never answers (misconfigured proxy, stalled tree-state for a deep birthday) left setup hung with no error and no clue. - Add tracing::info at each step (connecting → chain tip → fetching tree state → account imported), so the in-app log viewer shows setup progress. Routes through the frost_app_core=debug filter already used by the log buffer. - Bound get_latest_block and get_tree_state with a 30s timeout so a hang surfaces as a "did not respond" connection error (→ the UI's "Couldn't set up the wallet — check the lightwalletd endpoint … Retry" path) instead of an infinite spinner. These are single unary calls, not the long block stream, so a timeout is safe here (the stream deliberately has none). Note: a FROST group wallet is view-only by design — the app holds the group's UFVK, so it receives, shows balance, and builds unsigned txs; spending is via a FROST signing ceremony, not a local spend key. Backend builds clean; clippy unchanged at the 7-warning baseline. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
"Setting up the group's view-only wallet…" could sit forever with nothing in the logs even after the init_group_account timeout/logging fix — because init only runs *after* the wallet-status query returns. If that status read errors (keystore locked, key derivation for the group, or a db problem), status.data stays undefined, the init effect never fires (its guard requires status.data), and the UI's catch-all `!s` branch shows the setup spinner indefinitely with no error and no log line. - Handle status.isError: show the actual error with a Retry (refetch) button. The status read is local (no network), so the message notes it's the keystore or db, not lightwalletd. - Distinguish "Loading wallet…" (status still loading) from "Setting up…" (init actually running). - Show the real init error text in the "Couldn't set up the wallet" branch (it previously hid it behind generic endpoint advice), so the new get_tree_state / get_latest_block timeout messages are visible. tsc clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On testnet the configured lightwalletd often predates Ironwood (NU6.3) and rejects the Ironwood subtree-roots request with "invalid shielded protocol value", which aborted the entire sync at the first step — even though a server that doesn't know the pool has no Ironwood notes to report. The pipelined driver now treats *that specific* rejection on the Ironwood pool as "no Ironwood roots yet": it logs a warning and continues syncing Sapling and Orchard, instead of failing. Any other error still fails the sync, and the Sapling/Orchard root fetches are unchanged. A server that does support Ironwood returns the roots normally. - Extract is_invalid_shielded_protocol() and reuse it in annotate_sync_error (the friendly message for the stock driver, which can't skip) and in the new graceful-skip branch. - Unit test for the detector; existing annotate tests still pass (50 wallet- feature tests green). Note: the graceful skip only applies with experimental pipelined sync ON — the stock zcash_client_backend::sync::run requests Ironwood roots itself and can't be patched, so it still needs an Ironwood-capable server (and shows the friendly message). On such a testnet server there are no Ironwood funds to miss. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Ironwood (NU6.3) support still lives in the zcash_client_backend 0.24 / zcash_client_sqlite 0.22 release-candidate line (no stable 0.24.0/0.22.0 yet), so move the two `=` pins to the newest RCs: - zcash_client_backend =0.24.0-rc.6 -> =0.24.0-rc.7 - zcash_client_sqlite =0.22.0-rc.6 -> =0.22.0-rc.8 Cargo resolved the rest of the cohort via existing caret ranges: - zcash_protocol 0.10.3 -> 0.10.4 (required by sqlite rc.8) - pczt 0.9.1 -> 0.9.3 - orchard 0.15.4 -> 0.15.5 - zcash_pool_migration 0.1.0-rc.5 -> 0.1.0-rc.7 (Orchard->Ironwood turnstile) No source changes needed — the app + core compile against rc.7/rc.8 unchanged; 50 wallet-feature tests pass; clippy unchanged at the 7-warning baseline. Still an RC line: swap these for the stable 0.24.0/0.22.0 (and loosen the `=` pins) once they publish. Re-run Part A of the sync UAT after this bump — a backend/tree dependency change is exactly what that equality test guards. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New DKG groups are Ironwood-only (Orchard is exit-only), so present the wallet
as an Ironwood wallet and stop nagging about mainnet.
1. Orchard -> Ironwood in the UI:
- Balance summary now shows the Ironwood spendable/pending/total (the total
is strictly the Ironwood amount), dropping the always-on Orchard/Ironwood
pool-split line.
- Any legacy Orchard balance is surfaced only when it actually exists, via a
"Move to Ironwood" prompt (auto-hidden for new groups) — so no funds get
stranded.
- Reworded the visible labels/copy: "Group unified address", "Unified
address", "shielded pool" (was "shielded Orchard pool"), "RedPallas"
(dropped "(Orchard)"), DkgWizard "shielded spend authority". Internal
identifiers (isOrchard, groupOrchardKeys, the orchard balance field) are
unchanged — they name the real key type, not user-facing text.
2. Less verbosity: trimmed the wallet intro, transport-security note, welcome
callout, receive-address blurb, viewing-key callout, and the unshield /
migrate explanations down a level.
3. Calmer mainnet UX: removed the passive "you are on mainnet" danger banners
(Wallet page, send form) and the switch-to-mainnet confirm modal. The
transaction page header now carries a small Mainnet/Testnet pill near the
top. Kept: one slimmed send-time confirmation before broadcasting real ZEC,
the address/network mismatch validators, and the branch-mismatch warning
(those are functional, not passive nags).
tsc + vite build clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In the DKG wizard the section titles (Role, Group name, Ciphersuite, Threshold, Server, …) are `<label>`s styled the same dim color/weight as the descriptive text beneath them, so the two blur together. Scope a rule to the wizard form (`.dkg-form`) that renders those titles in the bright text color at weight 600, so each selection reads as a heading. Excludes the participant checkbox rows (`.multiselect-option`), which are options, not section titles. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The pipelined driver is validated and now the default and only sync path, so remove the opt-in machinery: - Removed the "Experimental pipelined sync" checkbox (Wallet Settings), the set_experimental_pipelined_sync command + its registration, the experimental_pipelined_sync setting, and the IPC bindings. - SyncOptions loses its `pipelined` field; sync_group always runs run_pipelined. - Deleted the now-dead stock path: the zcash_client_backend::sync::run call and the on-disk FsBlockDb/FsCache block cache it required (~110 lines) plus their now-unused imports. The pipelined driver streams blocks straight from the network to the scanner in memory, so no disk cache is needed. - The Ironwood pre-NU6.3 graceful-skip (which only ever lived in the pipelined path) is now always in effect — testnet syncs against pre-Ironwood servers without any flag. - Docs: SYNC_OPTIMIZATION.md status updated; SYNC_PIPELINE_UAT.md's on/off toggle and stock-baseline sections marked historical (no in-app comparison is possible now that the stock driver is gone). Backend + tsc build clean; 50 wallet-feature tests pass; clippy unchanged at 7. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes the pipelined sync driver the standard (and only) sync path, hardens Zcash
wallet setup so it can't silently hang, adds Ironwood/NU6.3 robustness (including
syncing against pre-Ironwood testnet servers), refreshes the wallet UX to be
Ironwood-first and less noisy, and bumps the Ironwood crate cohort to the latest
release candidates.
Net diff: ~14 files, mostly
src-tauri/core/src/wallet.rsand the wallet/groupsscreens. Backend +
tscbuild clean; 50 wallet-feature tests pass; clippyunchanged at its 7-warning baseline.
What's in this PR
Sync
download with CPU trial-decryption and streams blocks straight from the network
to the scanner in memory.
Wallet-Settings toggle + command, and
SyncOptions.pipelined.zcash_client_backend::sync::runcall and the on-disk
FsBlockDb/FsCacheblock cache it required (~110 lines).sync's still-open db connection ("database is locked").
Ironwood / NU6.3
rejects the Ironwood subtree-roots request; the driver now skips just that pool
and keeps syncing Sapling/Orchard instead of aborting. Lets testnet sync work
against pre-Ironwood servers (there are no Ironwood notes to miss on them).
the fix (switch to an Ironwood-capable server).
zcash_client_backendrc.6→rc.7,zcash_client_sqliterc.6→rc.8, pulling
zcash_protocol0.10.4,pczt0.9.3,orchard0.15.5, andzcash_pool_migrationrc.7.Wallet setup reliability
init_group_accountwith per-step logs and bounded its twounary RPCs with a timeout, so "Setting up the view-only wallet…" can no longer
spin forever with nothing in the logs.
"Setting up…" spinner (init only runs after status returns), with a Retry.
UX
the always-on Orchard/Ironwood pool split is gone. Any legacy Orchard balance is
surfaced only when it actually exists, with a one-tap "Move to Ironwood" sweep —
so no funds are stranded.
neutral terms; internal identifiers that name the real key type are unchanged.
and the switch-to-mainnet modal; the transaction page header now shows a small
Mainnet/Testnet pill. Kept one slimmed send-time confirmation before
broadcasting real ZEC, plus the address/network-mismatch and branch-mismatch
validators (those are functional, not passive nags).
into the helper text.
Docs
docs/SYNC_OPTIMIZATION.md(pipelined is now standard) anddocs/SYNC_PIPELINE_UAT.md(toggle/stock-baseline sections marked historical).TODO.mdas the real voting target.Behavior changes worth a close look
pipelined driver, the stock
sync::runreference is only in git history.present). Identical for new Ironwood-only groups.
that can't serve it — but such a server has none to track.
Testing
tsc+vite buildclean.cargo test -p frost-app-core --lib --features wallet).docs/SYNC_PIPELINE_UAT.mdfor the testnet acceptance checklist.Not in this PR / follow-ups
0.24.0/0.22.0once published.🤖 Generated with Claude Code