[ECMA-262 layering] Refactor import-related host hooks

[nicolo-ribaudo]

Hi!

I plan to propose a refactor to the import-related host hooks of ECMA-262, and this PR shows how it would affect HTML.

Currently, ECMA-262 has two host hooks:

• HostResolveImportedModule(referrer, specifier), which synchronously returns the Module Record corresponding to the (referrer, specifier) pair;
• HostImportModuleDynamically(referrer, specifier), used for dynamic import(), which asynchronously loads the Module Record corresponding to (referrer, specifier) and all its dependencies, calls .Link()/.Evaluate() on it and then "returns" to ECMA-262 by calling FinishDynamicImport.

We are working on three ECMA-262 proposals related to modules, and each one of them has needs that are not satisfied by the existing layering:

• Module Blocks allows declaring a module inline. It needs a way to load the dependencies of an already available module record.
• Import Reflection, at least when it comes to JS modules, allows loading a module without loading/linking/evaluating its dependencies. It needs a way to load a module without doing a "full import", and it needs a way to later load/link/evaluate its dependencies.
• "Layer 0" of Compartments allows implementing custom module loaders, and thus needs to specify how the graph loading algorithm works. This is currently only handled by hosts, for example HTML defines it in fetch a single module script and fetch the descendants of a module script.

Instead of adding multiple new host hooks and duplicating the graph fetching algorithm, we can:

• expose a single host hook, HostLoadImportedModule(referrer, specifier), that loads a single Module Record and can copmlete asynchronously: it's an async version of HostResolveImportedModule.
• move the graph traversal logic for the load phase from hosts to ECMA-262: it delegates the actual fetching to hosts with HostLoadImportedModule, which is called recursively for each dependency.

Module Records will expose a third method, other than .Link() and .Evaluate(): .LoadRequestedModules(), which is the one responsible of calling HostLoadImportedModule.

The host algorithm to load/execute a top-level module (for example, using a <script type="module"> tag) will now look like this.

1. (HOST) Fetch the top-level Module Record
2. (HOST) Call module.LoadRequestedModules():
   • (ECMA-262) For each dependency, call HostLoadImportedModule(module, specifier):
     • (HOST) Fetch the corresponding Module Record and return it.
3. (HOST) When it finishes, call module.Link()
   • ... internal steps in ECMA-262 ...
4. (HOST) Call module.Evaluate()
   • ... internal steps in ECMA-262 ...

The algorithm for dynamic import will look like this.

1. (ECMA-262) Call HostLoadImportedModule(referrer, specifier):
   • (HOST) Fetch the corresponding Module Record and return it.
2. (ECMA-262) Call module.LoadRequestedModules():
   • (ECMA-262) For each dependency, call HostLoadImportedModule(module, specifier):
     • (HOST) Fetch the corresponding Module Record and return it.
3. (ECMA-262) When it finishes, call module.Link()
4. (ECMA-262) Call module.Evaluate()

For the Module Blocks proposal, the algorithm to import a module block will look like the above dynamic import algorithm, except that we skip step 1.

For the Import Reflection proposal, the algorithm to load an uninstantiated JS module will just call HostLoadImportedModule without then calling the various Module Record methods.

For the Compartments proposal, we will be able to use the new graph loading algorithm by just replacing calls to HostLoadImportedModule with calls to an user-provided function.

---

Some more precise remarks about these HTML changes

1. The only observable change (observable from HTML, rather than users) is that when you import the same specifier twice from the same file, only the first one causes a call to HostLoadImportedModule (unless the first one fails):

   await import("./dep.js"); // Calls HostLoadImportedModule
   await import("./dep.js"); // Returns the already loaded module

   In practice this doesn't change much, because HostImportModuleDynamically/HostResolveImportedModule were already required to return the same Module Record both times.

   We still call the host hook multiple times if the first dynamic import fails, because there was an explicit goal of allowing to retry failed dynamic imports (Normative: change idempotency for HostImportModuleDynamically tc39/ecma262#1645).
   EDIT: For now I reverted this change, since even if it's not observable by HTML it's observable in other places.

2. The first commit is a small refactor only within HTML (it doesn't affect layering) that makes it easier to move the graph traversal to ECMA-262. I believe that it should not have observable effects (I wrote why in the commit description), but if it does I'll find an alternative solution.

3. I had to add an hostDefined parameter to LoadRequestedModules, which ECMA-262 passes as-is to HostLoadImportedModule, so that HTML can propagate the correct request destination. I couldn't just store the destination on the referrer Module Record's [[HostDefined]] slot because it should only be propagated through static imports and not dynamic ones.

4. On the ECMA-262 side, this refactor doesn't take into account import assertions, because they have not been merged to the main spec yet. I wrote these HTML changes assuming that HostLoadImportedModule will receive a { [[Specifier]], [[Assertions]] } record instead of just the specifier, similarly to how the import assertions proposal updated HostResolveImportedModule to receive { [[Specifier]], [[Assertions]] }.

---

Links

• ECMA-262 PR: Layering: Add HostLoadImportedModule hook tc39/ecma262#2905
• ECMA-262 preview: https://ci.tc39.es/preview/tc39/ecma262/pull/2905
• Slides presented in the TC39 meeting: https://docs.google.com/presentation/d/1RVUE-MENQT8dj2wxvMLMDxg_VoMOwiwNQQged39QIEU/edit?usp=sharing

---

/index.html ( diff )
/infrastructure.html ( diff )
/webappapis.html ( diff )