Propose tools and APIs for lowering components to core modules

[dicej]

Rendered RFC

[avrabe]

Great to see this. Happy to share notes on canonical ABI edge cases from Meld if useful.

[cfallin]

This is really important work and I'm happy to see it being developed -- thanks!

[cfallin]

This is a really good point and I think it's very important to solve "right", and not just reject components that instantiate a module more than once: that's a fundamental capability of the component model that core Wasm (without metadata/wrapper) doesn't have, and we don't want to bifurcate the ecosystem into components that fit this restriction and those that don't.

Function duplication (your second option) seems conceptually appealing because it hides the complexity, but in practice I suspect a large majority of functions will be duplicated, because almost everything will access memory...

Maybe the best option here is to actually define a "just the module linking, please" subset of the component model semantics that gives (i) a flat index space of core modules, (ii) a wiring diagram instantiating them and connecting imports and exports? The host already has to do some work to provide some intrinsics so this proposal is not "free" in any case; so ingesting such a format should not be too much of an additional sell (though there is certainly a step-function increase from "one core module" to "graph of core modules"). It's also conceptually the cleanest IMHO: this really is a thing that the component semantics can describe that a core Wasm can't, but most core Wasm runtimes should have host APIs to instantiate a thing more than once, so we should just "pass it through".

---

Just to note it down, though I don't like it: I guess there could be a fourth option here, which is (at a high level) something like "reify the vmctx as actual Wasm state". That seems to be the most "honest" w.r.t. the lowering paradigm.

The idea is that one would reify data structures that look like Wasmtime's instance state as Wasm GC values. A Wasm memory could be an arrayref to an array-of-i8; a Wasm table could be an arrayref to an array-of-whatever. Given those, one could define a vmctx Wasm struct that contains memory refs and table refs as our native vmctx does today, as well as any globals, inlined; then the lowered functions take this vmctx struct ref as an implicit first arg.

This clearly would have nontrivial runtime overhead as well, since in essence we'd have two levels of indirection for any state access.

[dicej]

Maybe the best option here is to actually define a "just the module linking, please" subset of the component model semantics that gives (i) a flat index space of core modules, (ii) a wiring diagram instantiating them and connecting imports and exports?

Yeah, I expect this is what it would have to look like. One thought that crossed my mind would be to literally output a real component, but one that only uses the absolute minimum set of features needed to embed, instantiate, and link modules. Hosts would need to be able to parse and instantiate these "simple components" but not need to support the entire component model.

[jellevandenhooff]

Bit of a drive-by thought: My guess is that if the lowering tooling is performant enough, any wasm host would want to adopt it, and then the component-model spec splits in two: these new slim host bindings and the component-model guest bindings as today. Do you think you would end up committing to API stability on the host bindings part? Standardize them? I suspect wasm runtimes would want that.

[dicej]

Do you think you would end up committing to API stability on the host bindings part? Standardize them?

Yeah, I think for this to work the API would need to at least be "officially" documented in the same way the dylink.0 convention is documented. Ideally, though, the "slim host bindings" API/ABI would just be a subset of the Component Model ABI (e.g. some or all of the thread.* and context.* canonical built-ins) and therefore not need to be documented or standardized separately. I think that should work, in which case the TODO item in the Host C API for Lowered Components section will just be to describe the relevant CM ABI built-ins as C function declarations.