feat: one-shot verified, compressed database backup into any directory - #26
Merged
Conversation
`wallmonitor --backup DIR` snapshots the live database with SQLite's online backup API (consistent under WAL, no service stop), runs integrity_check on the copy and refuses to continue unless it is ok, compresses it (gzip default, xz, or none), places it atomically as wallmonitor-<serial>-<UTC stamp>.db.gz, and rotates by the date in the filename (newest per day/week/month for 7/4/12; 0/0/0 keeps all). Only its own files are ever deleted. The snapshot temp lives beside the source, never in DIR, so a sync-watched folder never sees an uncompressed copy or a half-written file. Where DIR is — local disk, NAS mount, a sync client's folder, a directory another machine pulls from — is the user's decision; wallmonitor never talks to anything beyond it, and nothing is scheduled by default. The service installer gains an opt-in --backup-dir that adds a daily timer. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Aug 27, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The database is the history — the charger keeps none, and the degradation watch judges every new charge against it — yet nothing backs it up. The docs say "back up that one file and you have everything" and leave the how to the reader; on the reference install that meant one 1.4 GB file on one NVMe, no copy anywhere. Roadmap #1 "Automated backup helper", scoped down on purpose: the user asked that nothing run on a schedule unless they say so, and that storage location be entirely theirs to choose.
Code touched
wallmonitor/backup.py(new) —run_backup(db_path, dest_dir, compression, keep): SQLite online backup API into a temp file beside the source (never in the destination, which may be a sync-watched folder),PRAGMA integrity_checkon the snapshot with a hard failure if it isn'tok, compress (gzipdefault /xz/none, stdlib only), fsync, atomic rename towallmonitor-<serial>-<UTC stamp>.db.gz, then rotate.rotate()is pure and judged by the timestamp in the filename (sync clients rewrite mtimes): newest per day / ISO week / month for D/W/M, a file kept by any tier is kept,0/0/0rotates nothing, and only names matching its own pattern for this serial are ever returned — a second charger's files or anything foreign in the folder are untouched. Serial comes from thedevice_serialsetting, sanitized,unpinnedon a fresh database. The snapshot inherits WAL mode, so its-wal/-shmsiblings are swept with it.wallmonitor/config.py—--backup DIR(WM_BACKUP_DIR),--backup-compress(WM_BACKUP_COMPRESS),--backup-keep D/W/M(WM_BACKUP_KEEP, validated at parse time).--backupis exempt from the--hostrequirement, like--compactand--discover.wallmonitor/__main__.py— one-shot branch mirroring--compact: run, print sizes and timings and what was rotated out, exit 0.deploy/install-service.sh— opt-in--backup-dir DIRwrites<service>-backup.service(oneshot, same user, sameWM_DB) and a daily.timer(03:30, 15 min jitter,Persistent=true). Without the flag nothing is scheduled;--uninstallremoves both units.docs/running.md— flag rows and a Backup section: what it does, measured numbers, that the directory is the user's decision (four examples, none preferred: second disk, NAS mount, sync-client folder, a directory another machine pulls from), that nothing is scheduled by default, and a restore recipe that removes stale-wal/-shm— an old WAL applied to a restored file corrupts it.docs/recording.mdand README point at it.tests/test_backup.py(new, 7 tests) — snapshot round-trip (integrity, row count, serial, live DB still writable, no temp files left in either directory), unpinned serial andnone/xzcompression, rotation on synthetic names (40 dailies + same-day earlier copy + other-serial + malformed + foreign files) and on disk (foreign file survives, exactly the newest three remain under3/0/0), a corrupt source is refused before the destination is touched, and the flag needs no--hostwhile a bad--backup-keepis a parse error.Risk
finally. A box with less free space than its database cannot back up; the error is a plainOSError, and the destination is never touched.offline). The copy reflects a moment early in the pass — for a telemetry database that is exactly what a snapshot means.0/0/0disables it, but a user who points two chargers with the same pinned serial at one folder (impossible unless one is a clone) would have them rotate against each other.WM_DB; it does not inherit the service's other environment (it doesn't need any). If--dbwas not given at install, the defaultwallmonitor.dbin the monitor directory is used, matching the service.rsyncfrom another machine is a documented example, not a feature.Verification
python -m pytest -q: 120 passed (113 + 7 new).1395.9 MB snapshot (integrity ok, 3.5 s) -> 110.7 MB gzip (14.8 s), 21 s wall. No.wallmonitor-snapshot*left beside the source, no.tmpin the destination, serviceoffline: falsethroughout.rsync -a(6 s on the LAN),gunzip(1.4 s),PRAGMA integrity_check→ok,journal_modewal. Row counts against the live/api/statuscounts taken seconds later:vitals_samples1,549,943 vs 1,549,953 (the ten rows polled after the snapshot),wifi_samples134,375 = 134,375,sessions60 = 60,events2,904 = 2,904,alerts82 = 82; pinned serial present; newest sample timestamp equals the snapshot stamp to the second.xz -6reaches 64 MB in ~4 min; documented as the option for the patient.Deploy:
git pullon an install; no service restart needed (one-shot CLI, service behaviour unchanged). The installer's--backup-dironly matters on a re-run of the installer.