Skip to content

feature: deterministic Process exitCode wait in LlamaEngine.dispose() for iOS spawnFromProcess #5

Description

@sebasbad

Summary

When calling LlamaEngine.spawnFromProcess on iOS, calling await engine.dispose() sends a termination IPC signal to the underlying child process, but resolves before the OS kernel finishes unmapping the process's Metal GPU memory pages (mmap).

If a caller immediately spawns a new LlamaEngine.spawnFromProcess instance right after dispose(), the native llama_model_load_from_file call in the new child process can fail with failed to load model due to VRAM unmapping race conditions.


Proposed Solution

In LlamaEngine.dispose(), deterministically wait for the child process's exit signal before resolving the Future:

/// Inside LlamaEngine.dispose():
Future<void> dispose() async {
  if (_disposed) return;
  _sendPort?.send(DisposeCommand());
  
  // Deterministically wait for the underlying child process to terminate and release VRAM
  if (_process != null) {
    await _process!.exitCode;
  }
  
  _disposed = true;
}

Benefits

  1. Eliminates process timing race conditions during rapid GGUF model switching on iOS.
  2. Callers no longer need arbitrary Future.delayed backoffs after disposing an engine.
  3. Fully deterministic VRAM release behavior on mobile platforms.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions