[refactor] investigate-performance-issues

android/gradle.properties

@@ -1,3 +1,5 @@
 org.gradle.jvmargs=-Xmx1536M
 android.useAndroidX=true
 android.enableJetifier=false
+android.builtInKotlin=false
+android.newDsl=false

docs/architecture.md

@@ -83,6 +83,7 @@ which is mitigated by overriding `fileSystemProvider` and `httpClientProvider` i
 1. **`StartupPage`** (`/`)
 
    - Decides where to navigate based on persisted state in `SharedPreferences`. See [onboarding-flow.md](onboarding-flow.md).
+   - Loads languages in stages so the spinner isn't blocked by all 34 of them: a cheap `lazyInit()` for every language, a full `init()` for the one or two the first screen renders, the rest in the background. See [routing.md](routing.md).
 
 1. **`/view/<page>/<lang>`**
 
@@ -101,13 +102,14 @@ ViewPage(page, langCode)
        └─ ref.watch(pageContentProvider((name, langCode))).future
               • watches Language(langCode) for the on-disk path
               • reads <path>/<page.fileName> from FileSystem
-              • inlines images by replacing <img src="files/x.png">
-                with <img src="data:image/png;base64,…">
+              • loads all referenced images in parallel and inlines them:
+                <img src="files/x.png"> → <img src="data:image/png;base64,…">
               • throws LanguageNotDownloadedException / PageNotFoundException /
                 LanguageCorruptedException for the matching cases
 
 → HtmlView(content, direction)
        • sanitize(content, isDarkMode)  ← workarounds for flutter_html bugs
+         (cached per content + brightness, so a rebuild doesn't redo it)
        • Html.fromDom(...) with TagWrapExtension({'table'}) +
          TableHtmlExtension; tables get wrapped in a horizontal scroll view
        • onAnchorTap pushes /view<href> so internal worksheet links navigate
@@ -125,18 +127,19 @@ DownloadLanguageButton(langCode).onPressed
                dio.get(htmlZipUrl, responseType: bytes),
                dio.get(pdfZipUrl,  responseType: bytes),
              ])                                  ← github.com/4training/{html,pdf}-<lang>/archive/main.zip
-           – ZipDecoder.decodeBytes(...) → write each entry into .staging via FileSystem
+           – decode each zip in a worker isolate (max 2 at a time process-wide)
+             → write each entry into .staging via FileSystem
            – on any failure: rm -rf .staging and rethrow (prior data untouched)
            – rename assets-<lang> → assets-<lang>.old (if it existed)
            – rename .staging      → assets-<lang>        (atomic swap)
            – best-effort rm -rf .old
        • _load():
+           – stat structure/contents.json → downloadTimestamp (UTC)
            – read structure/contents.json
            – build pages: Map<String,Page>, pageIndex: List<String>,
              images: Map<String,Image>, pdf paths
-           – compute disk usage
-           – read modified timestamp of contents.json (UTC) → downloadTimestamp
            – emit new Language state
+             (disk usage is not part of it — see languageSizeProvider)
   → snackbar, button stops spinning
 ```
 

docs/content-rendering.md

@@ -11,7 +11,7 @@ ViewPage
         Padding → SingleChildScrollView → Column → SelectionArea
           → Directionality(LTR | RTL based on Globals.rtlLanguages)
             → Html.fromDom(
-                document: sanitize(content, isDarkMode),
+                document: _sanitizedDocument(isDarkMode),  // cached sanitize()
                 extensions: [TagWrapExtension({'table'}, …), TableHtmlExtension()],
                 style: { body, td, th, h1, h2, h3, li, p, ul },
                 onAnchorTap: (url, _, __) => Navigator.pushNamed(context, '/view$url'),
@@ -20,6 +20,8 @@ ViewPage
 
 ## `sanitize()` (`lib/widgets/html_view.dart`)
 
+`HtmlView` is a `StatefulWidget` purely so it can cache the result of `sanitize()` for the current `(content, isDarkMode)` pair. Parsing the HTML and running the passes below over the whole document is expensive on a slow device, while `build()` runs again for all sorts of unrelated reasons (a snackbar, the drawer opening, an orientation change). `flutter_html` only re-reads the document when its dependencies change, so recomputing it on every build was pure waste.
+
 The HTML emitted by the upstream `pywikitools` exporter has a few constructs that `flutter_html` can't render correctly. `sanitize()` runs a tree-rewrite step over the parsed DOM to fix them. Each rewrite is documented in-place, but here is the index:
 
 | Pattern in source HTML | What `sanitize()` does | Why |
@@ -83,4 +85,4 @@ A single breadcrumb is emitted via `debugPrint` on the first suppression per app
 
 ## Image handling
 
-Images are inlined as base64 by `pageContentProvider` *before* `HtmlView` ever sees the HTML. This avoids `flutter_html` having to load `file://`-prefixed URIs (which is fragile on Android/iOS). The downside is the rendered HTML can be large — for image-heavy worksheets like "God's Story (five fingers)" the inlined string can be hundreds of KB. So far this hasn't been a problem in practice.
+Images are inlined as base64 by `pageContentProvider` *before* `HtmlView` ever sees the HTML. This avoids `flutter_html` having to load `file://`-prefixed URIs (which is fragile on Android/iOS). The downside is the rendered HTML can be large — for image-heavy worksheets like "God's Story (five fingers)" the inlined string can be hundreds of KB, and every image is base64-encoded here only to be decoded again by `flutter_html`. So far this hasn't been a problem in practice; `pageContentProvider` at least reads and encodes all images of a page in parallel rather than one at a time. Serving images through a custom image extension that reads file paths directly would remove the round-trip altogether if this ever does become a problem.

docs/data-layer.md

@@ -57,7 +57,6 @@ class Language {
   final List<String> pageIndex;     // menu order (subset of pages.keys)
   final Map<String, Image> images;  // by filename
   final String path;                // local path to html-<lang>-main/
-  final int sizeInKB;
   final DateTime downloadTimestamp; // UTC, taken from contents.json mtime
   bool get downloaded => languageCode != '';
 }
@@ -106,25 +105,29 @@ The branch name (`main`) is hardcoded; switching branches would require a code c
    - `await ref.read(languageDownloaderProvider).download(languageCode)`. The downloader handles the full atomic flow internally (see below); on success the on-disk directory at `pathFor(langCode)` is the new content. On failure it throws and the prior on-disk directory (if any) is left untouched.
    - The caller wraps in a `try/catch` to preserve the existing `Future<bool>` shape.
 2. `_load()`:
-   - Recompute `assetsDirAlreadyExists()`.
-   - Sum file sizes recursively → `sizeInKB`.
-   - Read `contents.json` mtime → `downloadTimestamp` (UTC).
+   - A single `stat()` of `structure/contents.json` answers both "is this language on the device?" and "when was it stored there?" (`downloadTimestamp`, UTC). Not found → `return false`.
    - Parse `contents.json` worksheets, build `pages`, `pageIndex`.
    - Scan `pdf-<lang>-main/` for `.pdf` files; match each `worksheet.pdf` → `Page.pdfPath`.
    - Scan `html-<lang>-main/files/` for images; build `Map<String, Image>`.
-   - `_checkConsistency()` warns about HTML files referenced but missing, or HTML files present but unreferenced.
+   - `_checkConsistency()` warns about HTML files referenced but missing, or HTML files present but unreferenced. **Debug builds only** — it is another full directory listing and only produces log output.
    - On any throw: log, call `deleteResources()` (which delegates to `languageDownloader.delete(languageCode)`), reset to empty `Language`, return `false`.
 
-`init()` calls `_load()` only — no network. `lazyInit()` only checks for `contents.json` existence and returns a sparse `Language(languageCode, {}, [], {}, path, 0, timestamp)` without parsing — used by the background isolate which doesn't need page details. Both `lazyInit()` and `_load()` read the path from `ref.read(languageDownloaderProvider).pathFor(languageCode)`.
+`_load()` runs for every language on the UI isolate, so every call in it is asynchronous and it deliberately does *no* disk-usage accounting — see `languageSizeProvider` in [state-management.md](state-management.md).
+
+`init()` calls `_load()` only — no network. `lazyInit()` only checks for `contents.json` existence and returns a sparse `Language(languageCode, {}, [], {}, path, timestamp)` without parsing — used by the background isolate, and by `StartupPage` for all languages before the first frame. Both `lazyInit()` and `_load()` read the path from `ref.read(languageDownloaderProvider).pathFor(languageCode)`.
 
 ### Inside `LanguageDownloaderImpl` (`lib/data/language_downloader.dart`)
 
 The downloader owns the atomicity, concurrency, and crash-recovery guarantees so callers don't need to reason about partial state. One `download(langCode)` call performs:
 
-1. **Serialize** against any in-flight download — at most one zip pair is held in memory at a time (concurrency cap; protects low-end devices from rapid taps on the per-language download buttons).
+1. **Serialize** against any in-flight download of the *same* language (different languages still download in parallel; protects against rapid taps on the per-language download buttons).
 2. **Crash recovery** — `rm -rf <pathFor(lang)>.staging` so a leftover from a prior crashed run never accumulates.
 3. **Fetch concurrently** — `Future.wait` over two `dio.get(..., responseType: bytes)` calls for the HTML and PDF zips.
-4. **Extract into staging** — `ZipDecoder().decodeBytes(...)` over each response, writing every `ArchiveFile` via the injected `FileSystem`. Not `extractArchiveToDisk` (it is tied to `dart:io` and not testable against `MemoryFileSystem`).
+4. **Extract into staging** — `decodeZipEntries()` (a `ZipDecoder().decodeBytes(...)` pass) followed by writing every entry via the injected `FileSystem`. Not `extractArchiveToDisk` (it is tied to `dart:io` and not testable against `MemoryFileSystem`).
+
+   Decoding is pure, synchronous CPU work that takes seconds per archive on a slow device, so by default it runs in a worker isolate (`decodeZipInIsolate`, injectable via the `zipDecoder` constructor parameter). The decoded entries are copied back to this isolate rather than written from inside the worker, which keeps *all* file access going through the injected `FileSystem`; copying a few MB is negligible next to the decoding.
+
+   At most `kMaxParallelZipDecodes` (2) archives decode at the same time, process-wide. Onboarding downloads up to `kMaxParallelLanguageDownloads` (4) languages at once, each with two archives — letting all of those decode simultaneously would hold far too many decompressed archives in memory on a 4 GB device.
 5. **Atomic swap** — rename existing `assets-<lang>` → `assets-<lang>.old` (if any), rename `.staging` → `assets-<lang>` (rename is atomic on a single filesystem), then best-effort `rm -rf .old`.
 6. **On any throw mid-flight** — `rm -rf .staging` and `rethrow`. The prior on-disk directory (if any) is never touched until step 5, so a failed update never destroys offline content.
 
@@ -149,7 +152,7 @@ FutureProvider.family<String, Resource>((ref, page) async { … }, retry: null)
 
 Behaviour:
 - Reads `<lang.path>/<page.fileName>` as a string.
-- Replaces `<img src="files/x.png">` with `<img src="data:image/png;base64,…">` by base64-encoding the local file via `imageContentProvider`. (Inlining is necessary because `flutter_html` can't load arbitrary local file URIs.)
+- Collects every `<img src="files/x.png">` reference, loads all of them at once through `imageContentProvider` (`Future.wait`), then replaces the references with `<img src="data:image/png;base64,…">`. (Inlining is necessary because `flutter_html` can't load arbitrary local file URIs.) Loading them up-front and in parallel keeps the blocking disk reads out of the render path — a worksheet like "God's Story (five fingers)" references five images.
 - Throws:
   - `LanguageNotDownloadedException(langCode)` if the language is gone.
   - `PageNotFoundException(name, langCode)` if the page isn't in `pages`.

docs/features.md

@@ -72,7 +72,7 @@ Header row has the four "all-languages" buttons:
 - `DownloadAllLanguagesButton`
 - `DeleteAllLanguagesButton`
 
-Below the table: `diskUsage` total and a "X of Y languages" counter.
+Below the table: `diskUsage` total (calculated asynchronously — a `…` placeholder is shown until it resolves) and a "X of Y languages" counter.
 
 ### Language buttons
 - **`DownloadLanguageButton`** (`ConsumerStatefulWidget`): icon with internal `_isLoading` flag — swaps to `CircularProgressIndicator` during `LanguageController.download()`. Optional `highlight` flag wraps it in a tinted rounded box (used during onboarding).

docs/routing.md

@@ -31,23 +31,40 @@ StartupPage.init():
   if SharedPreferences['appLanguage'] is null:
       return '/onboarding/1'                 # first time
 
-  for each available language:
-      ref.read(languageProvider(code).notifier).init()   # load disk state
+  # step 1: which languages are on the device? one stat() each, in parallel
+  await Future.wait(languageProvider(code).notifier.lazyInit() for all codes)
 
   if app language is not yet downloaded:
       return '/onboarding/2'                 # resume onboarding
 
   # (commented out for v0.9: third onboarding step on missing checkFrequency)
 
-  ref.read(backgroundSchedulerProvider.notifier).schedule()
-
   if SharedPreferences['recentPage'] && 'recentLang' && language is downloaded:
-      return '/view/<recentPage>/<recentLang>'   # resume last worksheet
-  return '/home'
+      navigateTo = '/view/<recentPage>/<recentLang>'   # resume last worksheet
+  else:
+      navigateTo = '/home'
+
+  # step 2: fully load only what the first screen renders
+  await Future.wait(languageProvider(code).notifier.init()
+                    for code in {appLanguage, recentLang?})
+
+  # step 3: the remaining downloaded languages, unawaited, 3 at a time
+  unawaited(_loadRemainingLanguages(...))
+  unawaited(backgroundSchedulerProvider.notifier.schedule())
+
+  return navigateTo
 ```
 
 The first `await` in `init()` is what makes the loading spinner appear; once `init()` resolves, `Navigator.pushReplacementNamed` jumps to the chosen route, so the user never sees the home screen flash.
 
+### Why the loading is staged
+
+Fully loading all 34 languages before the first frame is what made cold start feel broken on slow Android devices (see [in_progress_notes/investigation_cold_start.md](in_progress_notes/investigation_cold_start.md)): the cost grew linearly with the number of downloaded languages while the spinner sat frozen.
+
+Only the app language (for the menu) and the language of the resumed worksheet are needed before navigating; `lazyInit()` gives the *downloaded* flag for all the others, which is all the routing decision needs. Everything else is loaded afterwards — the widgets that use it (language selection menu, the drawer's translate icons) are driven by `languageProvider` and rebuild by themselves as languages arrive.
+
+Step 3 gets the `LanguageController`s handed to it rather than the `WidgetRef`: `StartupPage` is disposed by `pushReplacementNamed` while that work is still running, and a disposed `WidgetRef` must not be used.
+
 ## Navigation primitives
 
 - **`Navigator.pushNamed`** for normal in-app navigation.

docs/state-management.md

@@ -47,21 +47,24 @@ This page is the index of every provider in the app — what it holds, what it d
 | Provider | Type | Purpose |
 | --- | --- | --- |
 | `fileSystemProvider` | `Provider<FileSystem>` | `LocalFileSystem()` by default; overridden with `MemoryFileSystem` in tests |
-| `imageContentProvider` | `Provider.family<String, Resource>` | Returns base64-encoded PNG bytes for a given image. Used by `pageContentProvider` |
+| `imageContentProvider` | `FutureProvider.family<String, Resource>` | Returns base64-encoded PNG bytes for a given image. Used by `pageContentProvider` |
 | `pageContentProvider` | `FutureProvider.family<String, Resource>` | The HTML body of a worksheet, with images inlined as base64. Throws `LanguageNotDownloadedException` / `PageNotFoundException` / `LanguageCorruptedException`. `retry: null` |
-| `languageProvider` | `NotifierProvider.family<LanguageController, Language, String>` | Per-language state: pages, images, PDFs, disk path, size, download timestamp |
+| `languageProvider` | `NotifierProvider.family<LanguageController, Language, String>` | Per-language state: pages, images, PDFs, disk path, download timestamp |
 | `countDownloadedLanguagesProvider` | `Provider<int>` | Derived count for the settings page |
-| `diskUsageProvider` | `Provider<int>` | Sum of all `Language.sizeInKB` |
+| `languageSizeProvider` | `FutureProvider.family<int, String>` | Disk usage of one language in kB, walked on demand |
+| `diskUsageProvider` | `FutureProvider<int>` | Sum of all `languageSizeProvider`s |
+
+Disk usage is deliberately **not** part of `Language`: computing it means listing a language directory recursively and statting every file, which is far too expensive to do for every language at every cold start. Only the settings page asks for it, so it is its own provider and the `LanguagesTable` shows a placeholder while it resolves.
 
 `LanguageController` is the heart of the app. Methods:
-- `init()`: idempotent load from disk. Call once per language at startup.
-- `lazyInit()`: like `init()` but cheap — only sets `downloaded` + `path` + timestamp without parsing JSON. Used in the background isolate.
+- `init()`: idempotent load from disk. Everything it does is asynchronous — it runs on the UI isolate, once per language.
+- `lazyInit()`: like `init()` but cheap — a single `stat()` that sets `downloaded` + `path` + timestamp without parsing JSON. Used by the background isolate and by `StartupPage` for every language before the first frame (see [routing.md](routing.md)).
 - `download({force=false})`: clears (if forcing), downloads HTML+PDF zips, parses structure.
 - `deleteResources()`: clears assets dir, resets state.
 - `_load()`: the parser. Reads `structure/contents.json`, scans `pdf-<lang>-main/` for PDF files, registers images in `files/`. Catches all errors and clears the assets dir on failure.
 
 `Language` (immutable data class):
-- `languageCode`, `pages: Map<String,Page>`, `pageIndex: List<String>` (menu order), `images`, `path`, `sizeInKB`, `downloadTimestamp` (always UTC).
+- `languageCode`, `pages: Map<String,Page>`, `pageIndex: List<String>` (menu order), `images`, `path`, `downloadTimestamp` (always UTC).
 - `downloaded` getter is `languageCode != ''`.
 - `getPageTitles()` returns the menu in order: English-name → translated-title.