Summary
When calling LlamaEngine.spawnFromProcess on iOS, calling await engine.dispose() sends a termination IPC signal to the underlying child process, but resolves before the OS kernel finishes unmapping the process's Metal GPU memory pages (mmap).
If a caller immediately spawns a new LlamaEngine.spawnFromProcess instance right after dispose(), the native llama_model_load_from_file call in the new child process can fail with failed to load model due to VRAM unmapping race conditions.
Proposed Solution
In LlamaEngine.dispose(), deterministically wait for the child process's exit signal before resolving the Future:
/// Inside LlamaEngine.dispose():
Future<void> dispose() async {
if (_disposed) return;
_sendPort?.send(DisposeCommand());
// Deterministically wait for the underlying child process to terminate and release VRAM
if (_process != null) {
await _process!.exitCode;
}
_disposed = true;
}
Benefits
- Eliminates process timing race conditions during rapid GGUF model switching on iOS.
- Callers no longer need arbitrary
Future.delayed backoffs after disposing an engine.
- Fully deterministic VRAM release behavior on mobile platforms.
Summary
When calling
LlamaEngine.spawnFromProcesson iOS, callingawait engine.dispose()sends a termination IPC signal to the underlying child process, but resolves before the OS kernel finishes unmapping the process's Metal GPU memory pages (mmap).If a caller immediately spawns a new
LlamaEngine.spawnFromProcessinstance right afterdispose(), the nativellama_model_load_from_filecall in the new child process can fail withfailed to load modeldue to VRAM unmapping race conditions.Proposed Solution
In
LlamaEngine.dispose(), deterministically wait for the child process's exit signal before resolving theFuture:Benefits
Future.delayedbackoffs after disposing an engine.