fix: propagate deletions made while the plugin was not running - #117
Draft
petergaultney wants to merge 11 commits into
Draft
fix: propagate deletions made while the plugin was not running#117petergaultney wants to merge 11 commits into
petergaultney wants to merge 11 commits into
Conversation
A canvas connects its own provider at materialization when it has never been server-synced, and nothing ever released that session: the websocket sync teardown only ran for sessions that started disconnected. With no view open a canvas announces no awareness and receives nothing, so the provider's inactivity watchdog closed the silent socket as an unexpected drop and scheduled a reconnect — every idle canvas in a connected shared folder churned through close/reconnect indefinitely. Background canvas sessions are transient now, matching how markdown documents run their idle sync sessions: once sync completes with no view lock held, the session is released intentionally. Buffered outbound updates flush before the socket closes, and a view attaching during the flush keeps the session. Remote updates for closed canvases continue to arrive over the shared folder's connection, and opening a canvas view reconnects on demand.
completeInitialEnrollmentFromRemote re-checked acceptsRemoteEnrollment before its hash await but not after it. A disk change landing inside the hash parks the machine on the diverged copy, and the enrollment that completed anyway recorded its own hash and mtime as the disk's -- reading the divergence away, settling the document synced, dropping the record that the file had moved on, and letting the queued download write the server's copy over the user's writing. Ask again after the hash and stand the enrollment down instead: the machine keeps the divergence, the download's write-side checks refuse, and the deferral record re-drives the download once the document accepts a remote copy again.
A transient server failure during a file pull ended the transfer for the rest of the session: the pull swallowed the error and reported success, token-fetch failures threw unclassified errors, and the download queue had no retry path. One 5xx left the file missing until plugin reload. - classify token, download-url, and upload-url failures by HTTP status, and wrap network-level transport failures as retryable - retry transient classes inside the read attempt with jittered backoff - re-drive retryable download failures through the queue with backoff, mirroring the sync queue's existing retry path - record terminal failures with their retry class and re-enqueue transient ones from the periodic pass, so even exhausted retries self-heal once the outage passes; permanent classes (auth/permission) are never re-driven - propagate pull failures so they are visible in the file's sync state
Obsidian's Live Preview table widget spawns a full embedded editor inside the cell being edited, seeded with only that cell's text, and instantiates every registered editor extension in it. That editor inherits the host view's editorInfoField, so file identity, document resolution, and source-view DOM ancestry all match the host — and the born-attached bind, which deliberately skips the view-registry identity check so real second views (split panes, popouts) can attach at creation, accepted it. The bind then rendered the whole note into the cell editor's fragment buffer; the widget forwards every transaction it does not recognize into the host note as a user edit, so the entire note was written into one table cell (newlines escaped to <br>, pipes to \|), and the host view's binding captured that as an ordinary edit and synced it. The mis-bind also replaced the engine's editor-view reference, so later document reads returned one cell's text as the note's content. Reject embedded sub-editors with two independent discriminators, either one sufficient. First, owner identity: the owning view resolved through editorInfoField names a different EditorView as its editor. This is available before the sub-editor's DOM is attached; an unresolvable owner leaves the decision open rather than rejecting, because a view under construction has not assigned its editor yet and an in-place editor replacement names the outgoing editor until the owner adopts the new one. Second, container ancestry: the editor's DOM sits inside the widget's .table-cell-wrapper. Ancestry detection is sticky, fully inerts the plugin instance, and is re-checked before the born-attached render dispatches, so nothing is ever dispatched into a cell editor even if it is only identifiable late. Legitimate additional views of the same note still bind born-attached at creation: at that moment their owner editor is either this view or not yet resolvable, so the gate cannot turn them away. Co-authored-by: Peter Gaultney <petergaultney@gmail.com>
…parseable blocks
The Y.Map("frontmatter") mirror could diverge from the document text and
then make the divergence worse: deleting a field never pruned the map
(the sync guard read the smaller key set as corruption) and the repair
path wrote the deleted key back; two clients repairing concurrently each
re-inserted the same reconstructed line and the merge kept both copies;
once duplicate keys made the YAML throw, both mirror directions bailed
out permanently and the dispatch builder treated the broken block as
body, prepending a fresh block on top of it.
Settle the mirror on one rule: the text owns the key set, the map owns
values. Enrollment seeds a structured baseline, and later text edits
publish only values changed by that edit plus removals, so concurrent
edits to unrelated keys remain independent instead of racing as stale
whole-block snapshots. Reconstruction overlays map values only onto keys
the text still carries, while deletion prunes stale map keys rather than
resurrecting them.
Missing or invalid frontmatter stays on the ordinary text-delta path so
stale map entries cannot replace user text. An opt-in feature flag enables
last-wins recovery for duplicate top-level keys; it defaults off so
ordinary clients leave invalid YAML untouched for manual repair.
Harness-Commit: c04928ebe6c9f1f2d6580ea0f3f3b9582fe63019
When a remote deletion arrives, cleanupExtraLocalFiles trashes the local disk file but leaves the in-memory Document alive: the pending-delete mark suppresses the trash's own vault-delete echo, so the deletion handler that would normally destroy the document never runs for this path. Whether the document survives is a race — the suppression token is cleared right after the trash resolves, so an echo that arrives late enough escapes suppression and destroys the document only by luck. A surviving document is a resurrection source. Its next engine write goes through the disk-write path with createIfMissing, re-creating the file at the deleted path; the re-created file has no sync-store entry, so the new-file registration path mints a fresh identity for it and uploads. The deleted file returns for every client under a new identity. The pending-delete write guard only holds while the mark is set, and the mark is cleared as soon as the trash resolves. After the trash resolves, find the live document at the deleted path and run the same teardown a processed vault delete performs: remove it from the file set and index, cleanup(), destroy(), and delete its per-doc persisted state; then refresh the file set so the UI drops the row. The teardown runs before the finally clears the pending-delete mark, so the write guard covers the whole window, and it is idempotent alongside an echo that escapes suppression — the later deletion pass finds nothing. A failed trash skips the teardown: the file survived, so the document must too. This makes delete-beats-dirty-editor deterministic: a remote deletion now always tears down the document, even when an open editor holds unsaved local edits. That is not a new policy — it is what the deletion handler already does whenever the echo escapes suppression today, and the disk file is trashed regardless; the change makes the existing delete-wins semantics deterministic instead of timing-dependent. Co-authored-by: Peter Gaultney <petergaultney@gmail.com>
A pending-upload hold records a local claim on a path until publication commits metadata and clears it. Two paths leak holds. A claim race lost to a committed entry for a content-addressed file delegates identity adoption to the reconciliation sweep, whose hash-match preconditions can fail — and when they do, nothing clears the hold. And a publication whose metadata is already committed identically returns early from the metadata write, before the clear runs. Either way the hold persists in its backing storage indefinitely: the boot-time sweep only removes holds whose file is missing, and these files exist. A leaked hold is armed, not inert. It shields its path from remote-delete cleanup, making the local file immune to deletions made elsewhere; and the moment anyone deletes the path, the next tree sync finds a held path without committed metadata and re-publishes it under the hold's identity. Deleted files silently return. At the end of each tree sync on a converged folder, drop every hold whose path already has committed metadata: a matching identity means the publication completed and the clear was missed; a different identity means the claim lost and adoption has had its chance by the end of a converged sync. Paths with in-flight remaps or active coordinated publication runs are skipped, and holds without committed metadata — genuinely new files awaiting first publication — are untouched. Each sweep logs what it dropped, including whether a live instance was still enrolled under the losing identity. This is a bounded mitigation with two honest limits. It disarms stale holds but does not adopt: a local file still enrolled under the losing identity stays divergent until the reconciliation sweep can re-key it — only the republication is prevented. And there is a one-interval residual window: a hold leaked after one sweep whose path is deleted before the next converged sweep completes can still republish once, because within a tree sync the publication pass runs before the sweep. The sweep bounds a leaked hold's lifetime to one sync interval instead of forever. Co-authored-by: Peter Gaultney <petergaultney@gmail.com>
…creation The file-identity lifecycle leaves no trace at three moments that matter when a deleted file comes back: minting a new identity is silent, a committed claim landing over a different pending-upload hold — the moment a lost race strands its hold — is silent, and re-creating metadata from a lingering legacy docs entry is silent. Add one log line at each. No behavior changes. - SyncStore.new logs each minted identity with its path. When a deleted file returns under a fresh identity, this line is what says which client minted it and when. - SyncStore.set warns when committed metadata lands over a pending- upload hold carrying a different identity. Stranded holds are what later re-publish deleted paths; this is the moment one is stranded. - SyncStore.getMeta warns when it schedules metadata re-creation from a lingering legacy docs entry. The opposite direction — tombstoning meta-without-legacy — already runs silently as the documented convergence path; this direction is the one that resurrects. All three lines use the existing debug-gated, redacted loggers and sit off hot paths: mints are per-new-file behind the registration debounce, and both warns are rare anomaly branches. Co-authored-by: Peter Gaultney <petergaultney@gmail.com>
A file deleted from disk while the plugin is not running never fires a vault delete event, and on the next launch a committed meta path with no local file is indistinguishable from a download that has not happened yet - so the tree sync re-downloads it and the deletion silently reverses. The persisted HSM record is the missing witness: one mapping this guid to the same path (and folder) with disk metadata means this client had the file materialized, so a missing file is a deletion, not a pending download. One-shot scan on the first tree sync after the local folder doc loads collects these paths; the server-create handler propagates the delete for approved paths instead of re-creating the file. Everything after the first sync is covered by live vault events. A mass-disappearance guard (more than 20 files missing and at least 20% of the folder) falls back to the current restore behavior, since a wholesale disappearance looks more like a moved or half-restored vault than deletion intent.
petergaultney
force-pushed
the
fix/offline-delete-scan
branch
from
August 6, 2026 18:54
90db116 to
3f26925
Compare
Contributor
Author
|
As we discussed, I'm not sure this is truly the right shape for what you're doing. I'll leave it for you to close when you've figured out what you actually want. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A file deleted from disk while the plugin is not running never fires a vault delete event. On the next launch, a committed meta path with no local file is indistinguishable from a download that hasn't happened yet, so the tree sync re-downloads it and the deletion silently reverses. Users experience this as deleted files "coming back" whenever they clean up their vault in Finder/a terminal/a script while Obsidian is closed.
This is the last of the resurrection mechanisms from our fleet's zombie-file reports that isn't covered on main —
d044439a(remote-delete teardown) and511f06af(stale-hold sweep) close the others, but neither addresses local deletions the plugin never observed.Approach
The persisted HSM record is the missing witness. A record mapping this guid to the same path (and folder) with disk metadata means this client had the file materialized on disk — so a missing file is a deletion, not a pending download.
prepareOfflineDeleteScan, called aftermergeManager.initialize()): collect committed paths that are missing locally but have a matching persisted record with disk metadata.deleteFiles) instead of re-creating the file. Approvals are valid for one sync pass only.Adds a small
MergeManager.getPersistedStateMeta(guid)accessor (state-meta cache with managed-meta fallback) so the scan can consult records for both documents and managed files.Testing
Two live clients (separate vaults/appIds/IndexedDBs on one machine, same shared folder against our production relay server), both running builds of current main (
5290d2fe), one with this patch. All edits by one author.Baseline (unpatched main): create a note on client A → syncs to B in seconds → quit B → delete the file from B's disk → relaunch B → file resurrects on B's disk ~4s after launch.
Patched (three runs): same procedure → on relaunch, B logs
and the deletion propagates to client A's disk within 2–6s. No resurrection over a subsequent soak. Normal in-app deletes (Obsidian running) still propagate as before (verified on the same pair).
The mass-disappearance guard is exercised only by code review; I didn't simulate a half-restored vault.