rustc-dev does not contain enough to link against rustc libraries

[RalfJung]

Ever since #72000, it is not sufficient to install rustc-dev to obtain a toolchain where one can link against private rustc libraries (for tools like Miri or Clippy). One has to also install llvm-dev. This subverts the purpose of rustc-dev; from what I can see, that package on its own is basically useless now.

[RalfJung]

This is a duplicate of #72564 which was closed as "expected breakage". IMO it makes sense to keep this open as there's clearly a bug here -- most of llvm-tools is useless for me, all I need is that one library to link against. And conversely, installing rustc-dev should be sufficient.

Maybe it would make sense to move that library to rustc-dev? If not, is there an issue tracking dependency support for rustup?

Cc @Mark-Simulacrum

[Mark-Simulacrum]

The problem is that (AFAIK) we can't duplicate the library without preventing installation of both. Currently llvm-tools-preview is much smaller than rustc-dev, and more likely for end users to install.

I see a couple of options:

• Move the llvm file required to a core component, even though it's not actually needed
• Move the llvm file to it's own component, asking both llvm-tools-preview and rustc-dev users to install it
• Add dependencies to rustup (I'm told this is Hard) and split into separate tarball

I am open to suggestions on other ways of fixing this or thoughts on which option is best.

Cc @kinnison as this is related to rustup

[kinnison]

Adding component interdependencies will be very difficult at this time. Not impossible; but if we want to do it we'll need to organise quite a bit. How large is this particular LLVM file? If it's not more than a few megabytes (i.e. less than 10) it may make sense to move it to a more core component such as rustc for minimum pain right now. However if it's large then that is less good.

Another option is to have it present in two names, one used by llvm-tools and one by rustc-dev - in the unusual case that someone has both installed, they're likely doing work which requires a lot of disk space anyway, so the duplication is less painful than it otherwise could be.

We could select a stop-gap measure until we can define and implement inter-component dependencies in the manifests, however if this is the only use-case for dependencies then it's debateable if we should bother.

If you want me to start defining component dependency support, let me know and I can give it a go.

[RalfJung]

The current solution of installing llvm-tools alongside rustc-dev manually is not horrible for Miri; we have a script to automate that anyway. So resolving this is not urgent.

But I think this is still worth tracking, i.e., I disagree with the old issue being closed. Hence me opening a new one.

[cuviper]

How large is this particular LLVM file? If it's not more than a few megabytes (i.e. less than 10) ...

In 2020-08-08 nightly-x86_64-unknown-linux-gnu, libLLVM-10-rust-1.47.0-nightly.so is 85M.

The rest of llvm-tools is only 4MB.

[Mark-Simulacrum]

The LLVM library in question is 85 MB, but it's also duplicated with an identical file in the lib/ directory. tarball size likely wouldn't be affected by putting it in the rustc component (which has the lib/ LLVM), but unpacked size on disk would go up by those 85MB. Maybe we should just eat that cost, not sure.

cbdbeff74332b29e368d3f3e7f3e99dbe9477e6a  ./lib/rustlib/x86_64-unknown-linux-gnu/lib/libLLVM-10-rust-1.47.0-nightly.so
cbdbeff74332b29e368d3f3e7f3e99dbe9477e6a  ./lib/libLLVM-10-rust-1.47.0-nightly.so

[rbtcollins]

Component dependencies is not that hard, yes we'll need to put it in the channel data, but I've written dependency interpreters more times than I care to think about now.

Lets put a heuristic in now to unbreak this and work on declarative dependencies in channel data for the long term.