|
| 1 | +# pipelining Meeting Notes |
| 2 | + |
| 3 | +## Video 2019-04-05 |
| 4 | + |
| 5 | +[@alexcrichton] and [@nnethercote] met on video for ~30m and talked about various |
| 6 | +aspects of implementing pipelining in the compiler. |
| 7 | + |
| 8 | +[@alexcrichton]: https://github.com/alexcrichton |
| 9 | +[@nnethercote]: https://github.com/nnethercote |
| 10 | + |
| 11 | +#### What are metadata/rlibs? |
| 12 | + |
| 13 | +First we talked a bit about what are rlibs/metadata files and how is this all |
| 14 | +going to be put together. The recap is: |
| 15 | + |
| 16 | +* Rustc can produce metadata files (`--emit metadata`). These metadata files are |
| 17 | + like header files for Rust crates. They're internally a compiler-specific |
| 18 | + binary format and cannot be inspected. |
| 19 | + |
| 20 | +* Rustc can also produce rlibs (`--emit link --crate-type lib`). An rlib is an |
| 21 | + archive (a `*.a` file) which contains three things: |
| 22 | + |
| 23 | + * Object code (`*.o`) |
| 24 | + * Compressed bytecode (`*.bc.z`) |
| 25 | + * Metadata (`metadata.bin`) |
| 26 | + |
| 27 | + The metadata included here is the same as `--emit metadata` |
| 28 | + |
| 29 | +When you type `cargo build`, Cargo is likely building an `rlib` for almost all |
| 30 | +library crates in use. When you type `cargo check` Cargo produces metadata files |
| 31 | +for all crates. |
| 32 | + |
| 33 | +#### How are we going to pipeline? |
| 34 | + |
| 35 | +The goal of pipelining is similar to CPU instruction pipelining, which is to |
| 36 | +fill up available hardware with as much work as possible. This can increase |
| 37 | +overall throughput without actually speeding up the intermediate operations. For |
| 38 | +example let's say your compile looks like this: |
| 39 | + |
| 40 | + |
| 41 | +``` |
| 42 | + meta meta |
| 43 | +[-libA----|--------][-libB----|--------][-binary-----------] |
| 44 | +0s 5s 10s 15s 20s 30s |
| 45 | +``` |
| 46 | + |
| 47 | +Here we're have a `binary` which depends on `libB` which depends on `libA`. The |
| 48 | +whole compile currently takes 30s, but as noted here the metadata files for |
| 49 | +libraries are available before the compilation is finished. |
| 50 | + |
| 51 | +Fundamentally all rustc needs to produce an rlib is the `metadata.bin` file from |
| 52 | +upstream crates. In other words, to compile `libB`, all we need is the metadata |
| 53 | +from `libA`, not the entire rlib. We can theoretically restructure the |
| 54 | +compilation like so: |
| 55 | + |
| 56 | +``` |
| 57 | +[-libA----|--------] |
| 58 | + [-libB----|--------] |
| 59 | + [-binary-----------] |
| 60 | +0s 5s 10s 15s 20s |
| 61 | +``` |
| 62 | + |
| 63 | +By starting subsequent compilations as soon as metadata is available, we shaved |
| 64 | +10 seconds off this compilation. We also did that for free! Furthermore |
| 65 | +we're able to use 2 parallel rustc processes at times instead of having |
| 66 | +everything be serial. |
| 67 | + |
| 68 | + |
| 69 | +#### Compromise: linking is hard |
| 70 | + |
| 71 | +Although we can shave 10s off compilation as shown above, it's likely going to |
| 72 | +be very difficult to get the full wins there. There's a caveat when compiling |
| 73 | +`binary` that we do actually need the `*.rlib` files to link. We don't need them |
| 74 | +to typecheck and such, but the linking phase needs them. |
| 75 | + |
| 76 | +Now linking is the final stage of the compiler, so it's only very late that we |
| 77 | +end up needing all of the dependencies. This would require some degree of |
| 78 | +synchronization still, though, where rustc needs to know it cannot proceed until |
| 79 | +Cargo instructs it to. |
| 80 | + |
| 81 | +As a result, the current thinking is to compromise here and simply ignore |
| 82 | +pipelining for "linkable" crates. Crates that produce binaries, dylibs, |
| 83 | +proc-macros, etc, will all wait for all their dependencies to finish before |
| 84 | +proceeding, even if they could get some work done ahead of time. As a result the |
| 85 | +target compilation timeline for our example above looks like: |
| 86 | + |
| 87 | +``` |
| 88 | +[-libA----|--------] |
| 89 | + [-libB----|--------] |
| 90 | + [-binary-----------] |
| 91 | +0s 5s 10s 15s 25s |
| 92 | +``` |
| 93 | + |
| 94 | +but we're still saving time! Typically a dependency graph in Rust is far deeper |
| 95 | +than three crates, so the compile time wins are expected to be much larger. |
| 96 | + |
| 97 | +#### Step 1: What architecture is used to pipeline rustc? |
| 98 | + |
| 99 | +The first thing we then talked about was how rustc was going to be invoked in a |
| 100 | +pipelined fashion. There were two primary candidates we figured could be |
| 101 | +implemented: |
| 102 | + |
| 103 | +##### (a) Run rustc twice |
| 104 | + |
| 105 | +One option is to literally run `rustc --emit metadata foo.rs` and then |
| 106 | +subsequently execute `rustc --emit link foo.rs`. The second command is in theory |
| 107 | +accelerated by incremental compilation artifacts produced by the first command. |
| 108 | + |
| 109 | +**Pros**: |
| 110 | + |
| 111 | +* Feels "pure" from a build system perspective as it keeps rustc in line with |
| 112 | + basically all other build tools, you run it to completion and don't care about |
| 113 | + what happens in the middle. |
| 114 | + |
| 115 | +**Cons**: |
| 116 | + |
| 117 | +* We're unlikely to reap full benefits from this strategy. The second `rustc` |
| 118 | + command has to redo quite a bit of work to get back to the point the first |
| 119 | + command was at, and it's not an instantatenous piece of work even with |
| 120 | + incremental. As a result this may run a risk of slowing down compiles because |
| 121 | + the second command takes so long to start up. |
| 122 | + |
| 123 | +##### (b) Signal Cargo when metadata is ready |
| 124 | + |
| 125 | +The second option is for rustc to continue in-process after it produces metadata |
| 126 | +and go on to produce the final rlib. The compiler would, however, send a signal |
| 127 | +to Cargo (somehow) that metadata is ready to go. |
| 128 | + |
| 129 | +**Pros**: |
| 130 | + |
| 131 | +* This should get us the full speed of pipelined compilation. There's no |
| 132 | + "startup time" for the work involved in producing the rlib since it's already |
| 133 | + all in-process in rustc. |
| 134 | + |
| 135 | +**Cons**: |
| 136 | + |
| 137 | +* This is going to be significantly more difficult for other build systems to |
| 138 | + get integrated (those that aren't Cargo). |
| 139 | + |
| 140 | +Overall we decided that this option was the route to pursue due to the speed |
| 141 | +wins likely to be gained. |
| 142 | + |
| 143 | +#### Step 2: work with only metadata as input |
| 144 | + |
| 145 | +@alexcrichton claimed that rustc cannot produce an rlib today with only |
| 146 | +`*.rmeta` files as input. After some testing, it was found that this was a false |
| 147 | +claim. Invocations like so can produce working rlibs: |
| 148 | + |
| 149 | + |
| 150 | +``` |
| 151 | +$ rustc libA.rs --emit metadata,link --crate-type lib |
| 152 | +$ rm libA.rlib |
| 153 | +$ rustc libB.rs --emit metadata,link --crate-type lib --extern libA=liblibA.rmeta |
| 154 | +``` |
| 155 | + |
| 156 | +So that means this step is already done! The compiler is already capable of |
| 157 | +implementing the pipelining showed above where it can be invoked in parallel by |
| 158 | +Cargo. |
| 159 | + |
| 160 | +#### Step 3: telling Cargo when metadata is ready |
| 161 | + |
| 162 | +The next (and final) piece of implementation needed in rustc is that the |
| 163 | +compiler has to somehow tell Cargo when metadata is available on the filesystem. |
| 164 | +Cargo needs some mechanism to know when to start spawning more rustc processes |
| 165 | +(if possible), and it currently has none without watching the filesystem. |
| 166 | + |
| 167 | +There are two primary ways we could implement this: |
| 168 | + |
| 169 | +##### (a) Use a TCP server |
| 170 | + |
| 171 | +A simple option would be for Cargo to start a small TCP server locally whenever |
| 172 | +it builds. The compiler would then connect to this server whenever a metadata |
| 173 | +file is ready to go and tell Cargo that it can proceed. |
| 174 | + |
| 175 | +**Pros**: |
| 176 | + |
| 177 | +* Relatively simple to implement in Cargo and rustc |
| 178 | +* Should work on all platforms |
| 179 | + |
| 180 | +**Cons**: |
| 181 | + |
| 182 | +* The compiler has to somehow tell Cargo which compiler it is (disambiguating |
| 183 | + from other parallel invocations) |
| 184 | +* This is a weird interface without really much precedent. It's unclear how |
| 185 | + other build systems would take advantage of it easily. It just "feels wrong" |
| 186 | + and "icky". |
| 187 | + |
| 188 | +##### (b) Print a JSON message when metadata is ready |
| 189 | + |
| 190 | +An alternative solution proposed by @ehuss is that the compiler could print a |
| 191 | +message on stdout/stderr to Cargo whenever a file has been produced. Cargo |
| 192 | +already does this, for example, when invoked with `--message-format=json`. The |
| 193 | +compiler already emits errors as JSON blobs with `--error-format=json`, although |
| 194 | +the compiler doesn't emit other information via JSON right now. |
| 195 | + |
| 196 | +**Pros**: |
| 197 | + |
| 198 | +* Feels like a clean solution. No need for Cargo to figure out what rustc is |
| 199 | + printing what (it knows that from which process printed). |
| 200 | +* Pretty easy to implement in rustc, just another JSON message somewhere. |
| 201 | +* Should be somewhat usable by other build systems as it's pretty standard to |
| 202 | + listen to stderr/stdout from spawned processes. |
| 203 | + |
| 204 | +**Cons**: |
| 205 | + |
| 206 | +* Cargo would have to always invoke the compiler with `--error-format=json`. |
| 207 | + Cargo does not currently do this to ensure that compiler error diagnostics are |
| 208 | + rendered to the screen correctly (aka are colorized and formatted correctly). |
| 209 | + A [recent PR to rustc](https://github.com/rust-lang/rust/pull/59128) shows |
| 210 | + hope for Cargo to be able to do this, although it may take time to implement |
| 211 | + and stabilize that. This would become a required blocker to enabling pipelined |
| 212 | + compilation. |
| 213 | +* A JSON message format for rustc would need to be designed. There's no |
| 214 | + precedent to draw from in rustc yet to emit arbitrary JSON messages about |
| 215 | + progress so far. There's likely some desire to do so though! |
| 216 | + |
| 217 | +We decided this is the route to go as it seems the most viable for |
| 218 | +stabilization. |
0 commit comments