Skip to content

Latest commit

 

History

History
467 lines (397 loc) · 26.4 KB

File metadata and controls

467 lines (397 loc) · 26.4 KB

Modrix - Architecture

A multi-platform, open-source, Vortex-style mod manager written in Rust, with a GUI, a TUI, and a CLI over one engine. Automated nxm:// download→install, any-game via plugins, Steam/Proton first-class.

Ethos: suckless-in-spirit - minimal, composable, hackable, no telemetry, small dependency surface, single static-ish binaries, everything the GUI does the CLI can do - plus safety-critical discipline (Power of Ten for Rust, §9.3) on the engine that touches users' files.


1. Goals & non-goals

Goals

  • Linux, Windows, macOS. Consistent look and behavior across all three.
  • One engine, three faces: GUI + TUI + CLI.
  • Seamless nxm:// handling: click "Download with Manager" (or our extension's button) → resolved, downloaded, installed to the right game, automatically.
  • Any game via community plugins; Steam + Proton as first-class citizens.
  • Fast, snappy, low-bloat. Not a webview, not Electron, not TypeScript.

Non-goals for v1 (designed-for, not built yet)

  • VFS / overlay deployment (OverlayFS/FUSE/USVFS). Link/copy first.
  • Sites other than Nexus Mods (the client is abstracted, but only Nexus ships).
  • Cloud sync, mobile, telemetry.

2. Locked decisions

Area Choice Rationale
Language Rust Safety where it matters most: the file-deployment engine touches users' game installs. Cargo + the crate ecosystem make 3 frontends cheap.
GUI Iced (MIT, pure-Rust) GPLv2-compatible; one renderer → identical on all 3 OSes; tiny self-contained binary; themed toward a clean macOS-like look.
GUI - ruled out Slint Was the first pick, but its free build is GPLv3-only, incompatible with our GPLv2 goal (see §11). Only modrix-gui links a toolkit, so this stayed a cheap swap.
TUI ratatui Mature, the standard.
CLI clap (derive) Standard, scriptable.
Plugins Two-tier: game.toml + game.lua (mlua, vendored Lua 5.4) Most games are just data; Lua only when logic is needed.
Deployment Link → symlink → copy, transactional manifest Cross-platform, reversible, ships sooner. VFS deferred.
Async/net tokio + hyper + rustls (RustCrypto provider) Concurrent, resumable downloads; Nexus API. Not reqwest: it transitively pulls Apache-2.0-only crates (ring, sync_wrapper), which are GPLv2-incompatible. We assemble a pure-Rust, all-MIT/ISC HTTPS stack instead (see §11).
Storage SQLite via rusqlite mods × profiles × files × conflicts is relational + transactional.
Reliability Power of Ten for Rust forbid unsafe, panic-free, bounded loops, lint-enforced in CI - the deploy engine is safety-critical (§9.3).
Steam steamlocate Parses libraryfolders.vdf / appmanifest_*.acf.
Paths directories XDG / Known Folders / Application Support.
Errors thiserror (libs) / anyhow (binary edges) Typed errors in the engine; ergonomic context at the top.
License GPL-2.0-only Free/open-source; preference of v2 over v3. It's what drives the GUI choice (see §11).

3. Workspace layout

A single Cargo workspace. All logic in modrix-core; frontends are thin. Dependency direction is a hard rule: modrix-core depends on no UI, no site-specific, and no protocol crate; frontends depend on core; no cycles.

modrix/  (cargo workspace)
├── modrix-core        # engine: games, profiles, mod store, deploy, conflicts,
│                      #   manifest, transactions. ZERO UI/network-UI deps.
│                      #   Owns the declarative `game.toml` loader (pure data, no
│                      #   code exec) so the engine can add games without linking
│                      #   the Lua host.
├── modrix-plugin      # mlua host + the sandboxed `modrix` API given to Lua.
│                      #   Also: FOMOD installer. (The `game.toml` loader lives in
│                      #   core; this crate adds the Lua tier on top of it.)
├── modrix-registry    # community plugin registry client: fetch/verify/install
│                      #   game-support plugins (and their agent skill files)
│                      #   from the curated modrix-plugins repository.
├── modrix-service     # the embedded hand-off service every frontend hosts
│                      #   (engine + downloads + IPC listener).
├── modrix-mcp         # MCP server (stdio JSON-RPC): the full engine surface
│                      #   as tools for AI agents, run via `modrix mcp`.
├── modrix-download    # segmented, resumable download engine (aria2/Motrix-style,
│                      #   no aria2 code). Fed by the browser extension's hand-off,
│                      #   NOT a site API. Retains the nxm:// identity parser.
├── modrix-ipc         # single-instance guard + loopback listener. The one
│                      #   ingress for both the OS protocol handler and the
│                      #   browser extension.
├── modrix-protocol    # tiny binary the OS launches for nxm://; forwards the
│                      #   URL to the running instance (or starts a headless one).
├── modrix-cli         # thin: clap over core. The scriptable surface.
├── modrix-tui         # thin: ratatui over core.
├── modrix-gui         # thin: Iced over core. The only crate that links a GUI toolkit.
└── extension/         # userscript (v1) + WebExtension (later). JS - unavoidable
                       #   in a browser.

Licensing: the project ships GPL-2.0-only (free/open-source; preference of v2 over v3). Every dependency is MIT/BSD/LGPL-2.1 or dual-licensed with an MIT arm, all GPLv2-compatible, enforced by cargo-deny in CI. See §11 for why the GUI is Iced and not Slint.


4. Core engine (modrix-core)

The correctness-critical heart. Everything else is presentation or I/O. Built to the Power of Ten reliability standard (§9.3): panic-free, unsafe-free, bounded.

4.1 Domain model

  • Game: a resolved install: plugin id, name, install path, store (Steam/…), Steam AppID, mod-staging root, deploy target(s).
  • Profile: a named, switchable set of enabled mods + load order for a game.
  • Mod: a staged archive's extracted contents in the central store, plus provenance (Nexus mod/file id, version, source).
  • Deployment manifest: the exact record of every file we placed into the game, how (hardlink/symlink/copy), and what original (if any) we displaced.

4.2 Deployment algorithm (link/copy, transactional)

Split into a pure planner (no I/O, trivially testable) and a transactional applier.

  1. Resolve the virtual file tree from enabled mods in load order (target_path → (mod, source_path); later mod wins conflicts). Conflicts are surfaced, not silently resolved.
  2. Diff the resolved tree against the current manifest → adds / removes / no-op.
  3. Back up any pre-existing, non-ours game file we're about to overwrite into a store, recorded in the manifest.
  4. Apply adds with fallback: hardlink (same filesystem) → symlinkcopy. Record link type + source hash per file.
  5. Apply removes: delete a deployed file only if it still matches the manifest (hash/link check - never clobber a file the user changed); restore any backup.
  6. Commit the manifest transactionally (temp file + atomic rename) and keep a small journal so an interrupted deploy is recoverable on next launch.

Undeploy / profile switch is the reverse walk over the manifest. A --dry-run and a verify pass are first-class. The five engine invariants (reversibility, idempotence, no-silent-clobber, crash-safety, determinism) are specified and tested in crates/modrix-core/src/deploy/apply_tests.rs.

4.3 Storage (SQLite sketch)

games(id, plugin_id, name, install_path, store, steam_appid, staging_root, ...)
profiles(id, game_id, name, is_active)
mods(id, game_id, name, version, source, nexus_mod_id, nexus_file_id,
     archive_path, staged_path, install_state)
profile_mods(profile_id, mod_id, enabled, load_order)      -- ordering lives here
deployed_files(id, profile_id, mod_id, target_path, source_path,
               link_type, source_hash, backup_path, deployed_at)  -- the manifest
downloads(id, source, url, nxm_uri, state, bytes_total, bytes_done, ...)

Game/plugin definitions and per-game config stay as plain files (see §5); SQLite (WAL mode) holds the relational index and the manifest.


5. Plugin system (modrix-plugin)

Two tiers so nobody writes code they don't need to.

5.1 Tier 1 - declarative game.toml

Covers the ~80% case. No code. Since api_version = 2 the definition also carries the game's capabilities - core dispatches on this data and has no game-specific logic of its own:

api_version = 2
id          = "skyrimse"
name        = "Skyrim Special Edition"
steam_appid = 489830                   # + gog_id / epic_id / xbox_id / origin_id / uplay_id
install_probe = ["steam", "gog", "registry", "path-hint"]
required_files = ["SkyrimSE.exe"]      # a resolved/entered dir must hold these to be accepted
mod_base      = "install"              # install | documents | local_appdata | roaming_appdata
mod_root      = "Data"                 # relative to mod_base (here, the install dir)
deploy        = "link"                 # link | copy
content_dirs  = ["meshes", "textures", ...]  # archive-root dirs that ARE content
base_files    = ["skyrim.esm", "cc*"]  # what the base game ships (never foreign)

[load_order]                           # omit entirely for BepInEx-style games
strategy    = "plugins_txt"
appdata_dir = "Skyrim Special Edition"

[[external_scan]]                      # how hand-installed mods appear on disk
kind = "file"; label = "plugin"; exts = ["esp", "esm", "esl"]; skip_base = true

[health.loader]                        # game-specific health checks, data-driven
plugins_dir = "SKSE/Plugins"; root_prefix = "skse"; message = "…"

[[registry_keys]]                      # Windows vendor keys drive the `registry` probe
hive = "HKLM"; key = "SOFTWARE\\WOW6432Node\\Bethesda Softworks\\Skyrim Special Edition"; value = "Installed Path"

Frontends read Engine::capabilities(game) from this data - a game without a [load_order] shows no Load Order UI at all. v1 definitions still parse; the two strategies that shipped on v1 resolve through a frozen preset table.

5.2 Tier 2 - game.lua (only when logic is required) - implemented

For custom installers, conditional deploy, special load-order formats, etc. Loaded via mlua (vendored Lua 5.4 - no system Lua dependency). Core defines the GameLogic trait + validated StagePlan seam (core::logic); modrix-plugin::lua implements it over a fresh sandboxed VM per callback with instruction (~5M), wall-clock (250ms), and memory (64 MiB) budgets.

Sandbox: plugins get a curated modrix table and no raw io/os/debug/ require. Every filesystem effect goes through the core's transactional layer (a plugin returns an install/stage plan; it never writes files directly). Each plugin call gets a step/time budget so a bad plugin can't hang or loop forever.

-- callbacks a plugin may implement:
function detect(ctx)      ... end   -- return install path or nil
function mod_root(ctx)    ... end   -- where files deploy, relative to install
function install(archive, ctx) ... end  -- drive a custom/FOMOD-like flow
function load_order(mods, ctx) ... end  -- return ordered list / write order file

-- API surface exposed to plugins (all mediated):
modrix.game            -- paths, appid, store, profile
modrix.fs.stage(src, dest)      -- register a file for deployment (goes to manifest)
modrix.fs.exists / read_dir / read_text
modrix.http.get(url)            -- rate-limited, through the client
modrix.log.info / warn / error
modrix.install.choose(step)     -- present a wizard step to the active frontend

API versioning: every plugin declares api_version; the host refuses or shims mismatches. Plugins live in a discovered directory (user data dir + bundled), one folder per game (<id>/game.toml, optional <id>/game.lua, assets). The 86 bundled games (ported from Vortex) live under games/<id>/ and are embedded into the binary by crates/modrix-core/build.rs, which scans the tree at build time - adding a game.toml ships it with no code change. An installed plugin or user definition with the same id shadows a built-in.

5.3 FOMOD

ModuleConfig.xml / info.xml parsing (via roxmltree) lives in modrix-plugin::fomod so every frontend drives the same install wizard (the GUI's wizard, the CLI's fomod show/fomod apply, MCP tools); Lua plugins can override for bespoke installers. Bounded iteration over the step/condition tree (§9.3).

5.4 The community plugin registry

Plugins are distributed through a curated Git repository (ParkerrDev/modrix-plugins): plugins/<id>/{plugin.toml, game.toml, game.lua?, skills/} plus a generated index.json (id, version, api_version, per-file sha256+size). modrix-registry fetches the index (one request, TTL-cached), verifies every file against its recorded hash before an atomic install into <data>/plugins/<id>/ - exactly where core::defcat discovers definitions, so an installed plugin is immediately a registerable game (and can shadow/update a compiled-in builtin by id). Plugins are fetched on demand, uninstallable, and gc removes any no registered game references. PRs are gated by consistency + index-freshness CI plus modrix plugin validate (definition validity, manifest agreement, game.lua loads under the sandbox).


6. Download pipeline (the "seamless" part)

Corrected (supersedes the earlier API-based design). Modrix does not use the Nexus API, API keys, or any third-party site API. An earlier draft made downloads depend on download_link.json + a personal apikey; that path is removed. The public API refuses free users download links anyway, so it could never serve the common case. Instead Modrix is a download manager fed by a browser extension that hands off the user's own browser download.

Modrix is, at its core, a clean-room Rust reimplementation of the aria2/Motrix download engine (segmented, multi-connection, resumable) with no UI and no aria2 code or binary - a GPLv2-clean rebuild on our hyper + rustls stack. A thin WebExtension captures the browser's real, already-authenticated download and hands it to the engine over loopback; the engine downloads and installs.

6.1 Extension hand-off - the primary path

The Modrix Bridge WebExtension (MV3; Chrome + Firefox) observes downloads via chrome.downloads (primary hook onDeterminingFilename; onCreated fallback on Firefox), plus an explicit "Download with Modrix" context-menu item. When it recognizes a mod download (an archive, or a *.nexus-cdn.com URL) it:

  1. cancels + erases the browser's own download (no duplicate file);
  2. gathers auth context: cookies.getAll({url}) → a Cookie: header, plus navigator.userAgent, the referrer, and the page/tab URL (which carries the Nexus game domain);
  3. POSTs a JSON HandoffJob to http://127.0.0.1:<port>/download with the per-session token.

The signed Nexus CDN URL is self-authenticating (a token + expires + IP binding in its query string), so the engine re-issues it directly - segmented and resumable - provided it runs on the same public IP and before expires. An expired/IP-mismatched URL surfaces a "re-click the download" prompt.

6.2 The engine (modrix-download)

A generic segmented downloader on hyper + rustls: it probes for Range support (206 vs 200), preallocates the target, and drives up to N connections (default 16) issuing Range GETs whose bodies stream to disjoint file offsets on per-worker seek-once handles (no unsafe, no shared cursor). Progress is persisted to a compact .mmdl control file (piece bitfield + ETag validator), giving resumable transfers. Downloads stream to <dest>.part, are checksum-verified when a hash is available (MD5/SHA-256), and are renamed into place only on success (a bad or partial file is never accepted). A FIFO queue with a global concurrency cap schedules multiple downloads; a broadcast event stream feeds the GUI/TUI. On completion the file is routed to a game and handed to Engine::stage - the unchanged transactional install path.

6.3 Install routing - no API

The target game is chosen from what the extension passed: the page/tab URL's Nexus domain (or an nxm:// link's domain, via the retained NxmUri identity helper) → Engine::game_by_nexus_domain; failing that, the download still completes and is parked, and the user is prompted to pick a registered game. A download is never lost because a game is unregistered.

6.4 IPC seam (modrix-ipc)

The loopback HTTP listener is the single-instance mechanism: whoever binds the port is primary; a second launch (or modrix-protocol) detects the bound port and forwards its request instead of starting a duplicate. If nothing is running, a headless engine starts to service the download, so browser clicks work even with the GUI closed. Loopback-only + a per-session token (x-modrix-token), CORS for the extension origin; never bind non-localhost. New JSON endpoints - POST /download, GET /download/<id>, GET /downloads - carry the hand-off and progress.

6.5 nxm:// - dormant, identity-only

Resolving an nxm:// link to a file requires the browser session we deliberately don't automate, so nxm:// is no longer a download mechanism. The OS protocol handler (modrix-protocol) and the NxmUri parser are retained only to extract game/mod/file identity; they are not registered by default. Real downloads always come through the extension hand-off.


7. Frontends (thin, over core)

All three call the same action layer in modrix-core (add mod, enable, reorder, deploy, switch profile, resolve download, run installer). No business logic in a frontend.

  • CLI (clap): the scriptable surface; built first because it proves the engine headless. Everything is reachable here, and --json wraps every command in a stable {"ok":true,"data":…} envelope for agents and scripts.
  • TUI (ratatui): mod list, load-order reorder, conflict view, download queue.
  • GUI (Iced, MIT): the same, themed (switchable specs: Aurora glass / Gold classic, Steam library artwork); the install wizard (FOMOD) renders here.
  • MCP (modrix mcp): the fourth frontend is an AI agent: a hand-rolled stdio JSON-RPC MCP server (modrix-mcp) exposing the full action surface as tools plus observability (downloads, progress, health) and modrix://skill/* resources - built-in usage/plugin-authoring guides and each installed game plugin's skill files - so agents learn per-game modding practice on install.

Because frontends are thin, the GUI toolkit is a localized, swappable choice - which is exactly why the GPLv2 requirement (§11) only cost us a one-crate change.


8. Platform integration

  • Steam: a small bounded VDF parser (core::detect, no steamlocate dep) finds Steam roots + parses libraryfolders.vdf and appmanifest_<appid>.acf for install dirs; the GUI reuses the same roots for library artwork (appcache/librarycache). Cross-platform.
  • Other stores (Windows only): the same core::detect module resolves GOG, Xbox and Uplay from the registry (via winreg) and Epic / Origin from the launchers' on-disk manifests (JSON / query-string). A generic registry probe reads the vendor keys a game.toml declares in registry_keys (Bethesda, CD Projekt, Maxis, …). No store APIs, no network; each store id is declared per game. On non-Windows these resolve to None (only Steam is cross-platform), and winreg is a target-gated dependency.
  • Proton (Linux): games run under Proton have a prefix at steamapps/compatdata/<appid>/pfx/. Deploy must target the right paths and, for some games, into the prefix (e.g. users/steamuser/Documents). This is where we can beat Vortex, which barely handles Linux - Steam Deck is a headline use case. Proton path-mapping is isolated: loadorder.rs resolves the prefix's AppData/Local for Plugins.txt, and roots.rs resolves a game's deploy target for the documents/local_appdata/roaming_appdata mod_base values (The Sims, Baldur's Gate 3, Factorio, …), gated on the prefix existing so a never-launched game is never fabricated.
  • Config/data dirs: directories (XDG on Linux, Known Folders on Windows, Application Support on macOS).
  • Archives: zip + sevenz-rust (pure-Rust 7z) for the common cases. RAR: the unrar license has a field-of-use restriction that is not GPL-compatible (Debian classes it non-free), so we don't bundle it - RAR support comes via libarchive (BSD, GPL-compatible) or as an optional/external-tool feature. Most Nexus mods are zip/7z anyway.

9. Security, safety & reliability

9.1 Security

  • The loopback listener is localhost-only. A request is authorized by the per-session token (CLI, modrix-protocol) or by a browser-extension Origin (chrome-extension://…, moz-extension://…) - browsers stamp Origin themselves and web pages cannot forge another scheme, so the extension is zero-config while a drive-by website is still refused.
  • Lua plugins are sandboxed (no raw io/os/debug/require; fs/http mediated and budgeted).
  • No telemetry.

9.2 File-operation safety

  • File operations are transactional, hash-checked, backed-up, and dry-runnable - we never overwrite a user-modified file silently, and every deploy is reversible and crash-recoverable via the manifest + journal.

9.3 Reliability standard - Power of Ten for Rust

The whole codebase follows the Power of Ten discipline (safety-critical Rust, adapted from NASA/JPL). This is an architectural commitment, not a style preference, because the deploy engine manipulates users' game files - a bug there loses saves and installs. It is mechanized so it holds from commit #1:

  • #![forbid(unsafe_code)] workspace-wide. FFI (mlua, libarchive) stays behind vetted safe wrappers. No raw pointers, no memory-unsafety surface.
  • Panic-freedom in library code. No .unwrap() / panic! / todo! / v[i] on fallible paths; propagate with ?/Result. The only justified panic is .expect("… why it cannot fail") on a real invariant.
  • Bounded loops, no recursion over untrusted input (mod trees, archives, FOMOD XML, load orders can be adversarial). Tree walks become explicit worklists with a depth/count cap; every loop has a visible ceiling.
  • Explicit arithmetic. overflow-checks = true in release; checked_* / saturating_* on anything derived from file sizes or counts.
  • Validate at every trust boundary (disk, network, env, plugin input) for semantics (ranges, invariants), not just shape.
  • Short functions (~60 lines), immutable by default (minimize let mut).
  • Enforced mechanically: pedantic clippy + cherry-picked restriction lints, -D warnings, cargo-deny (incl. the GPLv2 license gate), cross-OS CI. The exact [workspace.lints] / clippy.toml / deny.toml live at the repository root and are wired in Phase 0.

10. Key risks (ranked)

  1. Deployment engine correctness - the make-or-break; ~60% of the real work. Rust + transactional manifest + the Power of Ten discipline + invariant tests + dry-run mitigate it.
  2. Nexus free-user flow / rate limits - depends on the nxm key/expires handshake and the API budget; cache aggressively, degrade gracefully.
  3. FOMOD coverage - many real mods need it; parser fidelity matters.
  4. Proton path mapping - high-value, fiddly; isolate in a Steam/Proton module.

11. Licensing - why GPL-2.0, and why that picks the GUI

GPL-2.0-only, because you prefer v2 over v3 and want the manager free and open source. Both are FSF/OSI free-software licenses; the difference is v3's added terms (anti-tivoization, explicit patent grant, anti-DRM). Preferring v2 is a legitimate choice - it's the Linux-kernel license.

The one hard constraint: GPLv2 and GPLv3 are mutually incompatible - you cannot legally combine GPLv2-only code with GPLv3 code and redistribute it. That part is genuinely unavoidable. It bit us only because Slint's free build is GPLv3-only, so choosing Slint would force the whole app to v3. That was the sole thing pushing v3.

The fix is just the GUI. Slint → Iced (MIT). MIT is GPLv2-compatible, so the app is GPL-2.0 cleanly. We give up Slint's free "Cupertino" theme and instead theme Iced toward the macOS-like look - a little more styling, no blocker. Iced is also arguably more suckless: pure Rust, one language, no DSL, no external framework.

Rest of the tree is clean: tokio, rusqlite, ratatui, clap, mlua, serde, directories, steamlocate, zip, sevenz-rust are all MIT or dual-licensed with an MIT arm → GPLv2-compatible. Rule: take the MIT arm on dual-licensed crates, and avoid any Apache-2.0-only dependency (Apache-2.0 is incompatible with GPLv2). unrar is excluded for a related reason (§8). All of this is a mechanical cargo-deny gate.

The HTTP client is where this bit hardest. The obvious choice, reqwest, transitively pulls ring (its rustls crypto backend, Apache-2.0 AND ISC) and sync_wrapper (Apache-2.0-only, via tower-http). Apache-2.0 is one-way-incompatible with GPLv2, so cargo-deny rejects both. We therefore do not use reqwest. Instead we assemble the client from hyper + rustls, with rustls's crypto provided by rustls-rustcrypto (pure-Rust RustCrypto primitives - aes-gcm, chacha20poly1305, p256/p384, rsa, sha2) and roots from the OS trust store via rustls-native-certs (avoiding the CDLA-Permissive-2.0 webpki-roots). Every crate in that stack is MIT/ISC. The TLS state machine is still audited rustls; only the primitives change. Trade-off accepted with the maintainer: rustls-rustcrypto is young (0.0.x), in exchange for a 100% GPLv2-clean tree with no license exception. zip likewise uses the pure-Rust flate2 backend, never zopfli (Apache-2.0-only).

v2-only vs "v2 or later": we ship GPL-2.0-only, which most faithfully honors "v2 over v3." "GPL-2.0-or-later" would be the flexible alternative, but it lets v3 back in as a possibility - so we don't use it.

Escape hatch: if you later decide Slint's free macOS look is worth more than v2, the alternative is GPLv3 + Slint. It's a one-crate change.