Skip to content

feat: Lumiverse .lvbak backup import, proven against a real 4.75 GiB backup - #14

Merged
Coneja-Chibi merged 25 commits into
Coneja-Chibi:Mainstagefrom
Livingooal:feat/lumiverse-archive-import
Aug 8, 2026
Merged

feat: Lumiverse .lvbak backup import, proven against a real 4.75 GiB backup#14
Coneja-Chibi merged 25 commits into
Coneja-Chibi:Mainstagefrom
Livingooal:feat/lumiverse-archive-import

Conversation

@Livingooal

Copy link
Copy Markdown
Contributor

What this is

The Lumiverse .lvbak backup importer, taken from spec to working against a real backup: a 4.75 GiB archive with 32,951 entries that fans out into 1,708 entities (935 characters, 720 lorebooks, 18 presets, 14 personas, 17 regex sets). The branch carries the whole feature: the streaming ZIP64 reader, the lvbak table walk and codecs, the drop filter and import sheet, and the final piece this PR completes - commit by reference, because the naive design could not survive a real backup's size.

The bug the last commit fixes

The import itself worked. The response killed it: serializing all 1,708 entities into one JSON body produced a 3.77 GiB string attempt, past the JS engine's ~2 GiB ceiling, and the resulting RangeError: Out of memory was reported to the user as "not a Lumiverse backup archive" while ~20 GB of RAM burned. Even without the crash, a browser could neither hold those rows nor re-upload them at commit.

Now: entities stage on disk server-side (token-keyed per archive so multi-.lvbak drops work, bounded at 8 stagings with oldest-first eviction, cleared on shutdown), /api/inspect-archive returns summary rows (receipt, kind, parse report, staging ref, knowledgeRefs, entryCount - the real response is 0.94 MiB), and each checked row commits through the new loopback-only POST /api/studio/save-staged into the same saveBundle core, with knowledgeRefs rewritten server-side under the exact keep-both/dropped-ref semantics the entity path had. Unexpected inspect failures now say "something went wrong on Hoplight's side" instead of blaming the backup.

Verification

  • Red tests first, then green: response-shape contract, staged commit collision matrix (client and server), refIds rewrite semantics incl. empty-map drop, multi-archive staging, eviction bound, path-traversal refusals, loopback gate on both untrusted doors.
  • Live-proven twice against the real 4.75 GiB backup (before and after review remediation): HTTP 200 in ~14 s, 0.94 MiB response, zero entity payloads, staged commits landing on the shelf, and a final human pass in the running studio.
  • An independent adversarial review agent ran three passes with its own probes: it found and we fixed multi-archive staging, a dead link caveat, a dead entry-count chip, temp-file cleanup, and a missing gate test; every finding re-proven red-to-green by the reviewer's own probes. Security probes (crafted keys incl. traversal and NUL, token guessing, both remote doors) all refuse correctly.
  • bun run verify:ci: all gates pass. One caveat for local runs only: src/kit/providers/plan-usage.live.test.ts (pre-existing, credential-gated, untouched by this branch) hits Anthropic's live usage API and 429s when the machine is rate-limited; it skips wherever credentials are absent, including CI.

⚠️ Known limitation: the studio UI slows down at real-backup scale

Deliberately out of scope here, but worth a maintainer's eyes before users hit it. After importing the full backup above, on a fast desktop:

  • Initial library load takes several seconds, during which the app looks like it has no assets before pieces load in - the empty-shelf look and the still-loading look need to be different states.
  • Nothing appears cached client-side: switching rooms unloads the library, and coming back re-pays the full ~1,700-piece fetch every time.
  • Import triage has no card preview, so same-name/same-art duplicates cannot be told apart from the sheet.

Happy to file these as separate issues if that's preferred.

Disclosure

AI-assisted: written with Claude Fable 5 (Claude Code), driven and live-tested by @Livingooal against his own real Lumiverse backup. The adversarial review passes were run by an independent Claude agent with fresh context.

Livingooal and others added 25 commits August 4, 2026 15:55
… its own size

A real 4.75 GiB .lvbak imported fine and then died in the reply: 1,708
entities serialized into one 3.77 GiB JSON response, past the engine's
~2 GiB string ceiling (RangeError: Out of memory), reported to the user
as 'not a Lumiverse backup archive'. Entities now stage on disk server
side (token-keyed, bounded, cleared on shutdown), the sheet gets
summary rows carrying receipt, refs, and entry count, and each checked
row commits through POST /api/studio/save-staged into the same
saveBundle core, with knowledgeRefs rewritten server-side under the
exact keep-both and dropped-ref semantics the entity path had.

Unexpected inspect errors now say so honestly instead of blaming the
backup, and the loopback-only gate covers the new route (tested on
both untrusted doors; server-remote tests split at the 500-line cap).

Live-proven against the real backup: response 0.94 MiB in 14.2s, zero
entity payloads, staged commits on the shelf, plus an adversarial
review pass whose findings (multi-archive staging, dead link caveat,
dead entry count, temp cleanup) are all fixed and re-proven.
# Conflicts:
#	docs/generated/summary-queue.json
#	docs/generated/summary-review.json
#	docs/summaries/reference/ui.json
#	docs/summary-reviews/reference/ui.json
#	src/ui/switch/storage-shape-tripwire.test.ts
… make one

Creating a symlink on Windows needs elevation or Developer Mode, so the fixture threw EPERM and
took the suite red for every Windows contributor - a test failing because it could not build its
own setup, which says nothing about the code it guards. CI runs ubuntu, so the assertion still
runs on every push.
@Coneja-Chibi
Coneja-Chibi merged commit 0fc8a76 into Coneja-Chibi:Mainstage Aug 8, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants