Register devices when the ONNX runtime is injected via Symbol.for('onnxruntime')

[astefanutti]

Problem

backends/onnx.js documents an injection point:

if (ORT_SYMBOL in globalThis) {
    // If the JS runtime exposes their own ONNX runtime, use it
    ONNX = globalThis[ORT_SYMBOL];
} else if (apis.IS_NODE_ENV) {
    ...

That branch sets ONNX and then falls past both of the branches that populate supportedDevices and
defaultDevices. Every other path sets the module and the device list together.

The result is that an embedder using the documented seam gets an empty device list, and every device
option is rejected before anything loads:

Error: Unsupported device: "webgpu". Should be one of:

— with nothing after the colon.

Reproduction

In a browser, with any alternative onnxruntime-web build:

globalThis[Symbol.for('onnxruntime')] = await import('onnxruntime-web/jspi');
const { pipeline } = await import('@huggingface/transformers');
await pipeline('text-generation', model_id, { device: 'webgpu', dtype: 'q4f16' });

Fix

Let the injected runtime decide which device list applies, by the package it is — not by the process it runs in, and not by assuming it is an official package at all.

Every official package stamps its env: onnxruntime-web sets env.versions.web, onnxruntime-node sets
env.versions.node (Env.versions in onnxruntime-common). A custom runtime that only implements the API surface (Tensor, InferenceSession, …) sets neither. So the stamp tells the three cases apart:

const injectedONNX = ORT_SYMBOL in globalThis ? globalThis[ORT_SYMBOL] : undefined;

const runtime = injectedONNX?.env?.versions?.node
    ? 'node'
    : injectedONNX?.env?.versions?.web
      ? 'web'
      : injectedONNX
        ? 'custom'
        : apis.IS_NODE_ENV
          ? 'node'
          : 'web';

if (runtime === 'custom') {
    ONNX = injectedONNX;                 // no device list assumed — the runtime applies its own defaults, as before
} else if (runtime === 'node') {
    ONNX = injectedONNX ?? ONNX_NODE;    // ...unchanged node device list
} else {
    ONNX = injectedONNX ?? ONNX_WEB;     // ...unchanged browser device list
}

• An injected onnxruntime-web gets the browser list wherever it runs — including processes where
  IS_NODE_ENV is true, which is exactly where embedders inject it (Electron renderers, Bun, Node fallbacks).
• An injected onnxruntime-node gets the platform list.
• A custom runtime keeps today's behaviour: no list, no executionProviders forced on it.
• Without an injected runtime nothing changes: the environment decides, as before.

Validation

tests/injected_runtime.test.js evaluates the module afresh under each of the three shapes - a module
stamped versions.web, one stamped versions.node, and a bare { Tensor, InferenceSession, env: {} } and checks the device list each resolves to.

With this branch applied and built, the same call above loads a real 4.9 GB model (onnx-community/gemma-4-E4B-it-ONNX, q4f16) on the GPU under an injected JSPI runtime:

Ready on WebGPU, q4f16 weights, loaded in 11.5s · 24 KV layers kept between turns.

[nico-martin]

Hi @astefanutti, thank you so much for looking into this! This looks like the right fix and keeps the normal Node/browser paths unchanged. I don't see a blocker. A focused injected-runtime test and provider-contract clarification would be useful follow-ups.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[xenova]

Hi @astefanutti 👋 thanks for the PR!

Currently, the PR has an issue which basically assumes that an onnxruntime symbol is an official onnxruntime-web or onnxruntime-node package. This may not be the case, because -- as some users have done in the past -- they create their own onnxruntime-like API with Tensor, InferenceSession, etc. and then inject it in this way.

You do, however, correctly point out that the current approach doesn't allow for these official onnxruntime versions to be used... which is definitely an issue! Perhaps a solution is to inspect the ONNX module that is attached to globalThis[Symbol.for('onnxruntime')] and detect some official ONNX Runtime-specific value, maybe a version, or something... and then if that is the case, we do it in the way you suggested. Otherwise, we can fall back to the old logic.

Let me know what you think! :) I think we could design an API-level "devices" getter which can solve this, and can tie in nicely with #1753 (comment)

[astefanutti]

Hi @xenova, thanks a lot for the review, that makes perfect sense.

That's right custom runtimes like Supabase's edge runtime injects a shim: { Tensor, InferenceSession: { create }, env: {} } and with this PR it would receive the browser device list, and every session a ['wasm'] provider list it never asked for.

Also Injectors in the wild do the opposite of what I assumed: an Electron app injects onnxruntime-web precisely because IS_NODE_ENV is true in its renderer, and Bun/Node fallbacks (#1676, and others) inject the web build too. With "the environment decides", they would get the Node list (coreml/cuda/dml, …) for a web runtime.

Your suggestion fixes both, because every official package stamps itself: onnxruntime-common types env.versions as { common, web?, node?, 'react-native'? }, the web builds set versions.web and onnxruntime-node sets versions.node, while a shim like Supabase's has neither. So the stamp both tells official from custom and says which list applies:

const injected = ORT_SYMBOL in globalThis ? globalThis[ORT_SYMBOL] : undefined;
const versions = injected?.env?.versions;
const flavour = versions?.node ? 'node'
    : versions?.web ? 'web'
    : injected ? 'custom'
    : apis.IS_NODE_ENV ? 'node' : 'web';
ONNX = injected ?? (flavour === 'node' ? ONNX_NODE : ONNX_WEB);
// 'node' → the platform switch as today; 'web' → the WebNN/WebGPU checks as today;
// 'custom' → no device list, exactly the current behaviour for an injected runtime.

The bundled paths are untouched, and an unstamped injected module keeps today's semantics.

I've updated the PR with that and added the focused test @nico-martin suggested: inject a stub stamped versions.web, one stamped versions.node, and a bare shim, and assert the resulting device list for each.

I'd keep the API-level part as a follow-up so this PR stays a fix: (a) let a custom runtime declare its devices via an optional supportedDevices on the injected module, and (b) expose the resolved list as env.backends.onnx.getSupportedDevices(), which is the shape you proposed in #1753 and would fit next to setLogLevel.

One thing (a) has to carry with it: the default device would need to come from the backend rather than from IS_NODE_ENV, otherwise a runtime declaring ['cpu'] under Deno still gets wasm as its default which is the #976 problem, still present today for any custom runtime that doesn't pass device: 'auto'.