|
| 1 | +# Component Linking |
| 2 | + |
| 3 | +The Component Model enables multiple complementary forms of linking which allow |
| 4 | +producer toolchains to control which Core WebAssembly modules do or don't share |
| 5 | +low-level memory. At a high-level, there are two primary forms of linking: |
| 6 | +**shared-everything linking** and **shared-nothing linking**: |
| 7 | + |
| 8 | +When two modules are linked together to share Core WebAssembly `memory` and |
| 9 | +`table` instances, it is called **shared-everything linking**. In this case, |
| 10 | +the linked modules must have been compiled to agree on an implicit toolchain- |
| 11 | +or language-defined [ABI]. As an example, two modules compiled against the |
| 12 | +[WebAssembly/tool-conventions] C/C++ ABI could be shared-everything-linked |
| 13 | +together. |
| 14 | + |
| 15 | +When two modules that have been packaged as components are linked together, it |
| 16 | +is not possible for them to share the same `memory` or `table` instances and so |
| 17 | +this form of linking is called **shared-nothing linking**. In this case, the |
| 18 | +modules need to agree on the component-level types that stand between them, |
| 19 | +with each module being allowed to have a *different* ABI for producing and |
| 20 | +consuming component-level values of the common component-level types. |
| 21 | + |
| 22 | +A further sub-classification between **dynamic** and **static** is useful when |
| 23 | +describing shared-everything linking: |
| 24 | + |
| 25 | +In **shared-everything dynamic linking**, the producer toolchain keeps the Core |
| 26 | +WebAssembly modules handed to the runtime separate, thereby allowing the |
| 27 | +runtime to more-easily share the compiled machine code of common modules (such |
| 28 | +as libc, libpython or libjpeg). Importantly, while this linking is "dynamic" |
| 29 | +from the perspective of the producer of the individual modules, the set of |
| 30 | +dynamically-linked modules is still *statically* declared to the runtime before |
| 31 | +execution, allowing the runtime to perform traditional [AOT compilation] of |
| 32 | +each module (separately). (For |
| 33 | +[fully-runtime dynamic linking](#fully-runtime-dynamic-linking), see below.) |
| 34 | + |
| 35 | +In **shared-everything static linking**, the producer toolchain eagerly fuses |
| 36 | +intermediate units of WebAssembly code together to produce a *single* module |
| 37 | +that is handed to the runtime. Since this form of linking is handled by the |
| 38 | +producer toolchain, it's completely invisible to the Component Model and the |
| 39 | +runtime and thus mostly only relevant when talking about entire end-to-end |
| 40 | +workflows (like we'll do next). |
| 41 | + |
| 42 | +Given this terminology, the following diagram shows how the different forms of |
| 43 | +linking can be used together in the context of C/C++: |
| 44 | +<p align="center"><img src="examples/images/combined-linking.svg" width="800"></p> |
| 45 | + |
| 46 | +Digging into the steps of this diagram in more detail: |
| 47 | + |
| 48 | +The process starts by using a tool like [`wit-bindgen`] to generate C headers |
| 49 | +that expose core function signatures derived from the [Canonical ABI]. WIT type |
| 50 | +information that is needed later to build a component can be stored in a |
| 51 | +[custom section] that will be opaquely propagated to the component-specific |
| 52 | +tooling by the intervening Core WebAssembly build steps. |
| 53 | + |
| 54 | +Next, each C/C++ translation unit is compiled to a [WebAssembly Object File] |
| 55 | +using `clang`, optionally archived together using `ar`, and finally |
| 56 | +**shared-everything statically-linked** using [`wasm-ld`], all without any of |
| 57 | +these tools knowing about the Component Model. |
| 58 | + |
| 59 | +A single Core WebAssembly module can be trivially wrapped into a component |
| 60 | +using a tool like the [`wasm-tools`] `component new` command. Multiple Core |
| 61 | +WebAssembly modules can be **shared-everything dynamically-linked** together |
| 62 | +and loaded imperatively at runtime via emulated `dlopen()` using the `component |
| 63 | +link` command. (In the future, more-eager modes of native dynamic linking could |
| 64 | +be added.) For a low-level sketch of how dynamic linking works at the WAT |
| 65 | +level, see [this example](examples/SharedEverythingDynamicLinking.md). |
| 66 | + |
| 67 | +Lastly, multiple components can be **shared-nothing-linked** together using |
| 68 | +language-agnostic composition tools like [`wasm-compose`] or [`wac`]. Since the |
| 69 | +output of composition is itself a component, composite components can |
| 70 | +themselves be further composed with other components. For a low-level sketch of |
| 71 | +how shared-nothing linking works at the WAT level, see |
| 72 | +[this example](examples/LinkTimeVirtualization.md). |
| 73 | + |
| 74 | + |
| 75 | +## Fully-runtime dynamic linking |
| 76 | + |
| 77 | +While many use cases for dynamic linking are covered by what is described |
| 78 | +above, there are still some use cases that require "fully-runtime" dynamic |
| 79 | +linking where code is dynamically loaded that was not known (or may not have |
| 80 | +even existed) when execution started. |
| 81 | + |
| 82 | +One use case for fully-runtime dynamic linking is JIT compilation (where the |
| 83 | +running WebAssembly code generates the bytecode to be linked). This is possible |
| 84 | +in browsers today by having WebAssembly call into JS and using the [JS API]. |
| 85 | +Doing so from pure WebAssembly has been included in Core WebAssembly's list of |
| 86 | +[future features][JIT Future Feature] since the beginning of WebAssembly. |
| 87 | +This is a nuanced feature for many reasons including the fact that many |
| 88 | +WebAssembly execution environments don't provide the raw OS primitives (viz., |
| 89 | +making writable pages executable) to enable a WebAssembly runtime to perform |
| 90 | +the native JIT compilation necessary for performance. In any case, addressing |
| 91 | +this use case is ideally outside the scope of the Component Model. |
| 92 | + |
| 93 | +Another major use case for fully-runtime dynamic linking is implementing |
| 94 | +plugins that can be dynamically selected by WebAssembly code from a large |
| 95 | +and/or dynamically-populated store or registry. Such plugin models are |
| 96 | +sufficiently diverse (in how plugins are secured, discovered, transported, and |
| 97 | +compiled) that it's difficult to design a generic Component Model feature to |
| 98 | +support them all well. Based on this, it seems that the right place to |
| 99 | +address this use case *above* the Component Model, using an interface defined |
| 100 | +in [WIT] and allowing different platforms and applications to tailor the |
| 101 | +interface to their needs. |
| 102 | + |
| 103 | +For example, using the Preview 2 feature set of Component Model, a simple |
| 104 | +dynamic plugin interface might look like the following: |
| 105 | +```wit |
| 106 | +interface plugin-loader { |
| 107 | + load: func(name: string) -> plugin; |
| 108 | + resource plugin { |
| 109 | + handle-event: func(event: string, args: list<string>) -> string; |
| 110 | + } |
| 111 | +} |
| 112 | +``` |
| 113 | +The expectation here is that, if plugins are implemented by components, the |
| 114 | +`plugin` handle returned by `load` points to a component instance created by |
| 115 | +the host and the method calls to `handle-event` call exports of that component |
| 116 | +instance. |
| 117 | + |
| 118 | +While `plugin-loader` uses generic `string` types in the signature of |
| 119 | +`handle-event`, a particular application's plugin interface would naturally be |
| 120 | +customized to use whatever WIT types were appropriate, including handles to |
| 121 | +application-defined `resource` types. Because the signature of calls into the |
| 122 | +plugin are specified statically, a host can separately AOT-compile component |
| 123 | +plugins (e.g., on upload to the store or registry) into a native shared object |
| 124 | +or DLL that can be efficiently loaded at runtime. |
| 125 | + |
| 126 | +(There are a number of ways to improve upon this basic design with additional |
| 127 | +post-Preview 2 features of WIT and the Component Model.) |
| 128 | + |
| 129 | + |
| 130 | +[ABI]: https://en.wikipedia.org/wiki/Application_binary_interface |
| 131 | +[AOT compilation]: https://en.wikipedia.org/wiki/Ahead-of-time_compilation |
| 132 | +[WebAssembly/tool-conventions]: https://github.com/WebAssembly/tool-conventions |
| 133 | +[WebAssembly Object File]: https://github.com/WebAssembly/tool-conventions/blob/main/Linking.md |
| 134 | +[`wit-bindgen`]: https://github.com/bytecodealliance/wit-bindgen |
| 135 | +[Canonical ABI]: CanonicalABI.md |
| 136 | +[Custom Section]: https://webassembly.github.io/spec/core/binary/modules.html#custom-section |
| 137 | +[`wasm-ld`]: https://lld.llvm.org/WebAssembly.html |
| 138 | +[`wasm-tools`]: https://github.com/bytecodealliance/wasm-tools#tools-included |
| 139 | +[`wasm-compose`]: https://github.com/bytecodealliance/wasm-tools/tree/main/crates/wasm-compose |
| 140 | +[`wac`]: https://github.com/bytecodealliance/wac |
| 141 | +[JS API]: https://webassembly.github.io/spec/js-api/index.html |
| 142 | +[JIT Future Feature]: https://github.com/WebAssembly/design/blob/main/FutureFeatures.md#platform-independent-just-in-time-jit-compilation |
| 143 | +[WIT]: WIT.md |
0 commit comments