Skip to content

[PERFORMANCE] Migrate NPS Database to SQLite for Instant Startup and Reduced RAM #6

Description

@ted-devs

Description:
The current method of loading the NPS library involves downloading a large TSV file and parsing every row into memory as Dart objects on every app launch. This is computationally expensive, uses significant RAM, and causes a noticeable delay before the library is usable.

The Goal:
Move from a "Memory-First" approach to a "Database-First" approach using SQLite.

Proposed Changes:

  1. Local Database: Implement an SQLite database (via sqflite) to store the thousands of NPS game records.
  2. Initial/Manual Sync:
    • On the first launch, the app will perform a one-time "Bulk Import" of the TSV data into SQLite.
    • Add a "Refresh Library" button in Settings to manually trigger a fresh download and import.
  3. Lazy Loading (Pagination):
    • Update the HomeScreen to query the database in chunks (e.g., 20–50 items at a time) using LIMIT and OFFSET.
    • This ensures the UI is responsive even if the database grows to 10,000+ items.
  4. Optimized Search:
    • Utilize SQL LIKE queries for instant searching across the entire database without needing to filter in-memory lists.

Performance Impact:

  • Startup: Reduced from several seconds to < 200ms (instant).
  • RAM: Drastic reduction in memory footprint as only the visible records are held in objects.
  • Battery: Less CPU work required on every app launch.

Implementation Notes:

  • Bulk insertion should be performed in a separate Isolate using Database.batch() to prevent UI freezes during the initial sync.

Category: Performance / Architecture
Priority: High

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions