fix(windows): unbreak durable writes, renames, and staged promotions - #445
Merged
Conversation
The Windows path of write_atomic_durable_beneath had never run on Windows before. Three platform bugs broke every write and promotion: 1. The durable temp open set create_new with a custom access_mode but no write flag. std validates the flag, not the mode, so every open failed before touching the disk. Symptom: every xites.json persist logged "creating or truncating a file requires write or append access". 2. open_pinned_directory pinned ancestors with share_mode(READ) only. MoveFileExW with MOVEFILE_WRITE_THROUGH write-opens the parent directory to flush the rename, so every rename failed with a sharing violation (os error 32). Pins now share READ|WRITE and deny only DELETE, matching epix-fs. The sign test also stops holding an append handle across a sign, since the read path deliberately denies write sharing. 3. Raw MoveFileExW calls got plain wide paths, which are MAX_PATH limited. The staging tree nests three 64-char hash directories and crosses 260 chars, so every staged child promotion failed with ERROR_PATH_NOT_FOUND (os error 3) and child commits stayed deferred forever. epix_fs::verbatim_wide_null now converts raw Win32 paths to verbatim form; a regression test writes through a 300+ char path.
spawn_db_warmup rebuilds any served xite db that restore left without
a receipt and logs one summary line per boot. On a healthy warm start
it is a no-op that documents db health ("0 rebuilt, 12 already
present"). It covers any future gap in the restore path and surfaces
slow or refused rebuilds by name.
feedQuery now logs rows, feeds, answered xites, and elapsed time at
DEBUG. The first-start feed latency report was undiagnosable without
knowing whether the dbs answered, timed out, or held no rows.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Summary
The Windows branch of the durable write path added in ce4d96e (instant publishing with staged promotions) had never executed on Windows. On a real Windows node every write, rename, and staged promotion failed. This PR fixes the three platform bugs and adds boot-time db diagnostics.
Bug 1: durable temp open failed validation
write_atomic_durable_beneathopened the temp file withcreate_new(true)and a customaccess_mode(FILE_GENERIC_WRITE | DELETE)but never set.write(true). Rust std validates the boolean flag, not the custom mode, so every open failed before any syscall.Symptom: the node logged
Could not persist served xites: ... creating or truncating a file requires write or append accesson every xite state change.xites.jsonhad been stale for two weeks.Fix: set
.write(true). The custom access mode still controls the actual requested access.Bug 2: directory pins blocked every rename
open_pinned_directorypinned every ancestor withshare_mode(FILE_SHARE_READ).MoveFileExWwithMOVEFILE_WRITE_THROUGHwrite-opens the destination parent to flush the rename, and a pin that denies write sharing fails that open. Every durable rename died with a sharing violation (os error 32). This was verified with a standalone repro: the rename succeeds with the source handle still open once the pin shares write access.Fix: pins now share
READ|WRITEand deny onlyDELETE, matchingpin_windows_directoryin epix-fs. Denying delete is what pins the directory against being swapped. One sign test also stops holding an append handle across a sign, since the read path deliberately denies write sharing.Bug 3: MAX_PATH broke every staged promotion
The staging tree nests three 64-char hash directories (
data/.epix-stage/<ns>/<gns>/<digest>/backups/replaced/...) and crosses the 260-char MAX_PATH limit. Rust std converts long paths to verbatim (\\?\) form internally, but the rawMoveFileExWcalls in epix-fs andcreate_directory_durablereceived plain wide paths. Every child promotion failed withERROR_PATH_NOT_FOUND(os error 3), every childcontent.jsoncommit stayed deferred forever, and the dashboard reported dozens of failed updates.LongPathsEnabled=1does not rescue raw Win32 calls.Fix:
epix_fs::verbatim_wide_nullconverts raw Win32 paths to verbatim form (UNC-aware, rewrites forward slashes since verbatim paths skip separator normalization). A regression test writes, replaces, reads, and deletes through a 300+ char path.Diagnostics: db warm-up and feedQuery timing
spawn_db_warmupruns after restore and rebuilds any served xite db left without a receipt. On a healthy warm start it is a no-op and logs one line (Db warm-up: 0 rebuilt, 12 already present, 12 total in 0.0s) that documents db health per boot. Slow or refused rebuilds are named individually.feedQuerylogs rows, feeds, answered xites, and elapsed time at DEBUG.These were added while chasing a slow-feed-on-start report. Measurement showed the feed answers in under 2s once the promotion pipeline works, so no deeper change (for example persisting the in-memory dbs) is warranted now.
Testing
cargo test -p epix-fs -p epix-xite --lib: 59 passed, 0 failed, including the new MAX_PATH regression test.cargo test -p epix-ui --lib: 337 passed.cancelled_root_commit_registers_a_newly_accepted_stored_childandoptional_limit_persists_and_computes_bytesfail or flake on Windows on clean main too (verified via stash). They are unrelated to this diff.