Skip to content

Commit f32b229

Browse files
committed
Add Linking.md to provide a better high-level picture of linking
Resolves #239
1 parent 736660c commit f32b229

File tree

3 files changed

+149
-3
lines changed

3 files changed

+149
-3
lines changed

design/mvp/Explainer.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -205,9 +205,11 @@ defined). Thus, the following two components are equivalent:
205205
Whereas modules and components represent immutable *code*, instances associate
206206
code with potentially-mutable *state* (e.g., linear memory) and thus are
207207
necessary to create before being able to *run* the code. Instance definitions
208-
create module or component instances by selecting a module or component and
209-
then supplying a set of named *arguments* which satisfy all the named *imports*
210-
of the selected module or component.
208+
create module or component instances by selecting a module or component to
209+
**instantiate** and then supplying a set of named *arguments* which satisfy all
210+
the named *imports* of the selected module or component. This low-level
211+
instantiation mechanism allows the Component Model to simultaneously support
212+
multiple different styles of traditional [linking](Linking.md).
211213

212214
The syntax for defining a core module instance is:
213215
```ebnf

design/mvp/Linking.md

Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
# Component Linking
2+
3+
The Component Model enables multiple complementary forms of linking which allow
4+
producer toolchains to control which Core WebAssembly modules do or don't share
5+
low-level memory. At a high-level, there are two primary forms of linking:
6+
**shared-everything linking** and **shared-nothing linking**:
7+
8+
When two modules are linked together to share Core WebAssembly `memory` and
9+
`table` instances, it is called **shared-everything linking**. In this case,
10+
the linked modules must have been compiled to agree on an implicit toolchain-
11+
or language-defined [ABI]. As an example, two modules compiled against the
12+
[WebAssembly/tool-conventions] C/C++ ABI could be shared-everything-linked
13+
together.
14+
15+
When two modules that have been packaged as components are linked together, it
16+
is not possible for them to share the same `memory` or `table` instances and so
17+
this form of linking is called **shared-nothing linking**. In this case, the
18+
modules need to agree on the component-level types that stand between them,
19+
with each module being allowed to have a *different* ABI for producing and
20+
consuming component-level values of the common component-level types.
21+
22+
A further sub-classification between **dynamic** and **static** is useful when
23+
describing shared-everything linking:
24+
25+
In **shared-everything dynamic linking**, the producer toolchain keeps the Core
26+
WebAssembly modules handed to the runtime separate, thereby allowing the
27+
runtime to more-easily share the compiled machine code of common modules (such
28+
as libc, libpython or libjpeg). Importantly, while this linking is "dynamic"
29+
from the perspective of the producer of the individual modules, the set of
30+
dynamically-linked modules is still *statically* declared to the runtime before
31+
execution, allowing the runtime to perform traditional [AOT compilation] of
32+
each module (separately). (For
33+
[fully-runtime dynamic linking](#fully-runtime-dynamic-linking), see below.)
34+
35+
In **shared-everything static linking**, the producer toolchain eagerly fuses
36+
intermediate units of WebAssembly code together to produce a *single* module
37+
that is handed to the runtime. Since this form of linking is handled by the
38+
producer toolchain, it's completely invisible to the Component Model and the
39+
runtime and thus mostly only relevant when talking about entire end-to-end
40+
workflows (like we'll do next).
41+
42+
Given this terminology, the following diagram shows how the different forms of
43+
linking can be used together in the context of C/C++:
44+
<p align="center"><img src="examples/images/combined-linking.svg" width="800"></p>
45+
46+
Digging into the steps of this diagram in more detail:
47+
48+
The process starts by using a tool like [`wit-bindgen`] to generate C headers
49+
that expose core function signatures derived from the [Canonical ABI]. WIT type
50+
information that is needed later to build a component can be stored in a
51+
[custom section] that will be opaquely propagated to the component-specific
52+
tooling by the intervening Core WebAssembly build steps.
53+
54+
Next, each C/C++ translation unit is compiled to a [WebAssembly Object File]
55+
using `clang`, optionally archived together using `ar`, and finally
56+
**shared-everything statically-linked** using [`wasm-ld`], all without any of
57+
these tools knowing about the Component Model.
58+
59+
A single Core WebAssembly module can be trivially wrapped into a component
60+
using a tool like the [`wasm-tools`] `component new` command. Multiple Core
61+
WebAssembly modules can be **shared-everything dynamically-linked** together
62+
and loaded imperatively at runtime via emulated `dlopen()` using the `component
63+
link` command. (In the future, more-eager modes of native dynamic linking could
64+
be added.) For a low-level sketch of how dynamic linking works at the WAT
65+
level, see [this example](examples/SharedEverythingDynamicLinking.md).
66+
67+
Lastly, multiple components can be **shared-nothing-linked** together using
68+
language-agnostic composition tools like [`wasm-compose`] or [`wac`]. Since the
69+
output of composition is itself a component, composite components can
70+
themselves be further composed with other components. For a low-level sketch of
71+
how shared-nothing linking works at the WAT level, see
72+
[this example](examples/LinkTimeVirtualization.md).
73+
74+
75+
## Fully-runtime dynamic linking
76+
77+
While many use cases for dynamic linking are covered by what is described
78+
above, there are still some use cases that require "fully-runtime" dynamic
79+
linking where code is dynamically loaded that was not known (or may not have
80+
even existed) when execution started.
81+
82+
One use case for fully-runtime dynamic linking is JIT compilation (where the
83+
running WebAssembly code generates the bytecode to be linked). This is possible
84+
in browsers today by having WebAssembly call into JS and using the [JS API].
85+
Doing so from pure WebAssembly has been included in Core WebAssembly's list of
86+
[future features][JIT Future Feature] since the beginning of WebAssembly.
87+
This is a nuanced feature for many reasons including the fact that many
88+
WebAssembly execution environments don't provide the raw OS primitives (viz.,
89+
making writable pages executable) to enable a WebAssembly runtime to perform
90+
the native JIT compilation necessary for performance. In any case, addressing
91+
this use case is ideally outside the scope of the Component Model.
92+
93+
Another major use case for fully-runtime dynamic linking is implementing
94+
plugins that can be dynamically selected by WebAssembly code from a large
95+
and/or dynamically-populated store or registry. Such plugin models are
96+
sufficiently diverse (in how plugins are secured, discovered, transported, and
97+
compiled) that it's difficult to design a generic Component Model feature to
98+
support them all well. Based on this, it seems that the right place to
99+
address this use case *above* the Component Model, using an interface defined
100+
in [WIT] and allowing different platforms and applications to tailor the
101+
interface to their needs.
102+
103+
For example, using the Preview 2 feature set of Component Model, a simple
104+
dynamic plugin interface might look like the following:
105+
```wit
106+
interface plugin-loader {
107+
load: func(name: string) -> plugin;
108+
resource plugin {
109+
handle-event: func(event: string, args: list<string>) -> string;
110+
}
111+
}
112+
```
113+
The expectation here is that, if plugins are implemented by components, the
114+
`plugin` handle returned by `load` points to a component instance created by
115+
the host and the method calls to `handle-event` call exports of that component
116+
instance.
117+
118+
While `plugin-loader` uses generic `string` types in the signature of
119+
`handle-event`, a particular application's plugin interface would naturally be
120+
customized to use whatever WIT types were appropriate, including handles to
121+
application-defined `resource` types. Because the signature of calls into the
122+
plugin are specified statically, a host can separately AOT-compile component
123+
plugins (e.g., on upload to the store or registry) into a native shared object
124+
or DLL that can be efficiently loaded at runtime.
125+
126+
(There are a number of ways to improve upon this basic design with additional
127+
post-Preview 2 features of WIT and the Component Model.)
128+
129+
130+
[ABI]: https://en.wikipedia.org/wiki/Application_binary_interface
131+
[AOT compilation]: https://en.wikipedia.org/wiki/Ahead-of-time_compilation
132+
[WebAssembly/tool-conventions]: https://github.com/WebAssembly/tool-conventions
133+
[WebAssembly Object File]: https://github.com/WebAssembly/tool-conventions/blob/main/Linking.md
134+
[`wit-bindgen`]: https://github.com/bytecodealliance/wit-bindgen
135+
[Canonical ABI]: CanonicalABI.md
136+
[Custom Section]: https://webassembly.github.io/spec/core/binary/modules.html#custom-section
137+
[`wasm-ld`]: https://lld.llvm.org/WebAssembly.html
138+
[`wasm-tools`]: https://github.com/bytecodealliance/wasm-tools#tools-included
139+
[`wasm-compose`]: https://github.com/bytecodealliance/wasm-tools/tree/main/crates/wasm-compose
140+
[`wac`]: https://github.com/bytecodealliance/wac
141+
[JS API]: https://webassembly.github.io/spec/js-api/index.html
142+
[JIT Future Feature]: https://github.com/WebAssembly/design/blob/main/FutureFeatures.md#platform-independent-just-in-time-jit-compilation
143+
[WIT]: WIT.md

design/mvp/examples/images/combined-linking.svg

Lines changed: 1 addition & 0 deletions
Loading

0 commit comments

Comments
 (0)