cloud_loader — Google Drive backend for the Slopsmith library

Stream your .sloppak library from Google Drive instead of keeping everything on local disk. Metadata is indexed once; song files download on Play and are removed after the next Play. New files created by other plugins (sloppak converter, retune) are auto-uploaded.

How it works

Bootstrap (indexing, per song):
  A .sloppak is a zip; its metadata (manifest.yaml) and cover art are tiny
  members. So indexing reads ONLY those, over Drive HTTP Range requests —
  no full download. A few KB per song instead of tens/hundreds of MB.
    Drive (range: manifest + cover) → extract_meta() → meta_db.put() → 0-byte stub
  Full song binaries never touch disk until you Play.
  (Fallback: if the ranged read fails on an odd archive, that one song is
  downloaded in full the old way.)

Re-scan (incremental, via the Drive Changes API):
  The first scan mints a change-log page token. Later scans call changes.list
  with that token and get back ONLY what changed since — added, modified,
  moved, or deleted files — so adding one song doesn't re-walk the whole
  folder tree. Each changed file's ancestry is walked up to the configured
  root to confirm it belongs to the library subtree (the change feed is
  drive-wide). A saved token is scoped to its (drive, root); changing either,
  disconnecting, or clicking "Full re-scan" forces a fresh full walk.
  Note: a "Shared with me" root can't be tracked by the change feed — use
  Full re-scan there to pick up new songs the owner added.

Play:
  Frontend wrapper around window.playSong asks /needs_prefetch first.
  If yes → /prefetch starts a background download with progress polling.
           An overlay shows "Downloading from Google Drive · X%".
  When ready → original playSong runs against a real local file.

After play:
  Next prefetch GC-deletes the previous song's local copy.

Auto-upload:
  Background watcher polls cloud-dlc/ every 30 s for new/modified files.
  Uploads to Drive, updates the local index, keeps a stable file_id so
  later edits update the same Drive object.

The server's WebSocket play handler runs unmodified against a real on-disk file. No monkey-patching, no fragile core hooks.

Installation

cd /path/to/slopsmith/plugins
git clone https://github.com/Jafz2001/slopsmith-plugin-cloud-loader.git cloud_loader
pip install -r cloud_loader/requirements.txt
# restart Slopsmith

A new Cloud Library panel appears under Settings once the plugin is loaded. Continue with Setup below to connect your Google account.

Setup (one-time, ~5 min)

1. Create OAuth credentials in Google Cloud Console:
   • Create a project (or reuse one)
   • Enable the Google Drive API for the project
   • Credentials → Create credentials → OAuth client ID → Desktop app
   • Download the JSON
2. In Slopsmith: Settings → Cloud Library
   • Step 1: Upload the downloaded client_secret.json
   • Step 2: Click Connect Google Drive — completes OAuth in your browser
   • Step 3: Browse... and pick the folder containing your library
   • Step 4: Leave Local cache directory empty to reuse your DLC folder
   • Step 5: Start scan — reads metadata + cover art only (a few KB per song). Re-scans are incremental; Full re-scan forces a complete re-walk.

Auth scope

This plugin requests the full drive scope (read + write), not the more restrictive drive.file. Reason: drive.file only sees files the app itself created — it could not read your existing library, forcing a re-upload of everything. If that trade-off matters to you, use a dedicated Google account for the plugin.

What's stored where

Path	Contents
$CONFIG_DIR/cloud_loader/client_secret.json	Your OAuth client credentials
$CONFIG_DIR/cloud_loader/token.json	OAuth refresh token (sensitive)
$CONFIG_DIR/cloud_loader/config.json	Root folder id, dlc dir
$CONFIG_DIR/cloud_loader/index.db	SQLite index of remote files + local state
<dlc_dir>/<song>.sloppak	Transient — present only between Play and the next Play

Token and client_secret are excluded from diagnostics bundles.

Cloud profiles — sync your progress across machines

Beyond the song files, the plugin can sync your player progress (profile, XP, streaks, per-song best scores/accuracy, favorites, loops, quests, wallet, shop, playlists, wanted, tags) between machines and recover it after a reinstall. Find it under Settings → Cloud Library → "6. Progress profiles".

How it works:

• The core app keeps all progress in $CONFIG_DIR/web_library.db. This feature stores a progress-only snapshot of that DB in your Drive, under <root>/.feedback_profiles/<profile_id>/progress.sqlite, with an index at <root>/.feedback_profiles/profiles.json. Only the player-state tables are copied — never the local library index (songs) or derived caches.
• Multiple profiles, one per id. Two people sharing the same Drive library each pick their own profile, so they never overwrite each other. Create a profile, or load an existing one from the list.
• Automatic sync (on by default): does one pull on startup (to pick up progress the other machine left) and pushes your progress every ~5 min. Your own pushes never trigger a self-restore — only a newer push from another machine does.
• Safe restores: before replacing local progress, a timestamped backup of web_library.db is written to $CONFIG_DIR/cloud_loader/profile_backups/ (last 5 kept). Restore copies the column-intersection per table with per-table savepoints, so a snapshot from an older/newer app version restores what it can and skips (logs) what it can't.

Conflict model is last-write-wins per profile — designed for using one machine at a time. If you edit the same profile on two machines nearly at once, the later push wins; the "Guardar ahora ↑" / "Traer de la nube ↓" buttons give you manual control. After loading/pulling a profile, reload the app for the UI to reflect the restored progress.

Endpoints (all under /api/cloud_loader/profiles/): status, list, create, load, push, pull, auto.

Limitations / known issues

• First Play is slow — downloads the whole file before the WebSocket play handler runs. A 200 MB song on 50 Mbps ≈ 30 s.
• No range-request streaming — we fetch whole files rather than partial reads, so a large song downloads in full before it plays.
• Conflict resolution — multi-device editing is last-write-wins. Editing the same file from two devices without restarting can clobber one side's changes when the watcher runs.
• Rate limits — Drive API allows ~1000 requests / 100 s / user. A scan of 500 songs stays well under this.