Accelerate the rust compilation

[Kailai-Wang]

Currently it takes ~20min to do a round of rust compilation(e.g. cargo clippy or docker build), this could bring about long waiting time if the self-hosted github runner is severely occupied (image there're multiple ongoing PRs, the compilation will be repeated for all the merging to/from dev and syncing).

Maybe there're ways to accelerate the process:

• for plain rust compilation, there is sccache
• for rust builds in docker, check the following discussions:
  • https://stackoverflow.com/questions/58473606/cache-rust-dependencies-with-docker-build
  • https://blog.mgattozzi.dev/caching-rust-docker-builds/
  • https://stackoverflow.com/questions/39661138/optimising-cargo-build-times-in-docker

Low prio.

[Kailai-Wang]

Elevate the priority because we are going to change the build profile later in #326 which will significantly increase the build time

[Kailai-Wang]

sccache looks promising when building the binary natively, see https://github.com/litentry/litentry-parachain/actions/runs/1991565305

It took ~7min to run the cargo runtime test. Without sccache it would take ~20min+.

The problem remains with the docker build: it takes ~1hr on a github runner, see #393
Caching with docker/build-push-action doesn't do much, mostly because the cargo build is an "atomic" command and cannot break into sub-layers.

Looking for some other options like

• cargo-chef
• transferring the target/ directory around
• to build dependency first?

[Kailai-Wang]

Experiments with cargo-chef and buildx GHA cache on github-runner: it seems to work.

When running on a github-runner, I see a decrease from 80min to 30min with the docker layer cached, however, 30min is still longer than the self-hosted runner without cache.

Self-hosted runner seems to have problems with buildx GHA cache + cargo-chef, however it looks like more like a GHA issue, see here.

[Kailai-Wang]

cargo-chef + local buildx cache works on a self-hosted runner, see the comparison before and after: 25min vs 14min

The difference is not significant as on github-runner though.