Expose bootstrapping options with fvm

[bluesign]

Problem Definition

Currently there is no way to influence bootstrapping, I am trying to implement some options for emulator ( something like simpleContracts, to skip deploying often not used contracts) onflow/flow-emulator#170

Proposed Solution

Adding fvm options to enable and disable deployments for the below parts would be amazing.

• DKG
• QC
• Staking (I think also includes epoch etc here)
• LockedTokens
• Contract Audits

cc: @janezpodhostnik

[janezpodhostnik]

LockedTokens and Contract Audits could be removed without any problems, but I'm not sure if everything would still function without DKG, QC and Staking.

Is the main purpose of this to make bootstrapping faster? If so maybe we can make a measurement first and see what the expensive bits are.

[bluesign]

My main objective is to bootstrap emulator without those bits. onflow/flow-emulator#170

[devbugging]

@janezpodhostnik I think this would mostly apply and be useful in the case of running the emulator. We would also have a use for that on Playground where we just want a clean state without any pre-deployed contracts as none are required. Could the bootstrap procedure be configurable?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[joshuahannan]

Is this something that we think could be possible? We'd like the NFTStorefront and USDC contracts to be deployed to the emulator for bootstrapping, but can't do it without this

[janezpodhostnik]

The main problem with changing the bootstrapping procedure are actually our unit and integration tests.

A lot of the tests silently rely on certain things being on certain locations. When you change what gets bootstrapped where, some tests start failing and you spend quite some time fixing them all.

I would love to have a more modular approach to bootstrapping where each test (/application/usage) declares what it needs. And the bootstrapping procedure would figure out dependencies and magically compose everything.

Maybe the proper way to approach this is to build the framework for modular bootstrapping and start pulling contracts out of the current procedure in to the new one one by one.

This is not a short task, but we will definitely need it more and more as we move additional logic on chain.

[bluesign]

Maybe the proper way to approach this is to build the framework for modular bootstrapping and start pulling contracts out of the current procedure in to the new one one by one.

Currently we have N pre-created accounts ( if we keep this N constant and not increase) so some extra bootstrap option to deploy to some account index some contract can be pretty helpful. ( if we change N it would eventually break some 3rd party tests )

So instead of creating account, deploying contract and moving to next; we can create N pre-created accounts, and deploy by index somehow core contracts and additional user provided ones.

Then maybe we can have some default options ( replicating current system ), this would solve disabling some contracts etc on purpose like in the initial proposal without effecting legacy tests etc.

[joshuahannan]

If I understand correctly, we are uncomfortable updating the flow-go bootstrapping, but shouldn't there be code in the emulator repo somewhere that we can add more bootstrapping code to and just leave the flow-go bootstrapping the same?

We also just found out that we need to update the emulator bootstrapping to deploy and setup the VM bridge, so I figured there is probably a way we could just do that in the emulator without having to change any of the flow-go bootstrapping.

If that is possible, could y'all link me to the emulator code where that would be able to happen? I want to learn about it and try to figure out how to do it myself. We could include the NFTStorefront and USDC deployments there too

[joshuahannan]

@janezpodhostnik Would you be able to take a look at my previous comment? It would help unblock me from figuring out how we want to do the emulator bootstrapping for the bridge. Thank you!

[janezpodhostnik]

So instead of creating account, deploying contract and moving to next; we can create N pre-created accounts, and deploy by index somehow core contracts and additional user provided ones.

This sounds like a good idea, but how would we decide what is a good N. To small and we might need to change it in the future. To large and we are taking up unnecessary resources.

I suppose we could go with something like 10. We are currently using 5 I think and at least some of those could actually just go on the service account.

[bluesign]

Thinking a bit more on this I think N as a bootstrap option with default value 5 covers all cases.

Also this covers frequently requested emulator feature where people want to pre create N accounts on emulator launch with a flag.

[joshuahannan]

I'd really like to help with this but I'm not sure where to start. Like I said before, we also need to update emulator bootstrapping for the bridge, so I want to understand how everything works so I can start figuring out how to do it.

[j1010001]

Another reason we should prioritize this is bootstrapping of EVM bridge contract: #7053

In general - bootstrapping is fragile and hard to edit - people should not need to modify FVM to bootstrap new contract, it should be modular enough to add a new contract easily, without the need to modify flow-go.

[bluesign]

I made small PR to emulator for this, @janezpodhostnik can you have a look ?

onflow/flow-emulator#808

[janezpodhostnik]

That looks like a really general/low-level solution, which we also need and we can add right away! Nice!

Let me try to explain what I had in mind with this specific issue.

Imagine that we have some repos with a bunch of cadence contracts. As an example we can take the EVM bridge repository.

And now lets say that I'm trying to build on top of that and I would like my emulator (or in general my local FVM instance) to start up bootstrapped with all that is needed.

I think the EVM bridge repository should provide a bootstrap module that I can somehow import into bootstrapping and everything happens automatically.

I was thinking of an api like this:

bootstrap := fvm.NewBootstrapBuilder()

bootstrap.SetSomeSetting(value)

bootstrap.AddModule(fvm.EVMBootstrapModule(
// with some EVM related setting
)) 
bootstrap.AddModule(bridge.BridgeBootstrapModule())
bootstrap.AddModule(MyModule())

procedure, info, err := builder.Build()

_, _, _ := vm.Run(ctx, procedure, snapshotTree)

println(info.FirstEmptyAccount())

Modules can:

• edit bootstrap settings
• create accounts
• deploy contracts on new or existing accounts
• send setup transactions

A wish-list of mine would be:

• the ordering of how you add modules should not matter
• modules could import other modules (so if you are importing MyModule, MyModule could already define that it needs EVMBootstrapModule and it does not need to be imported explicitly)

With something like this the core bootstrapping we currently have could also be broken into modules. Some that come to mind:

• NFT Base
• NFT Storefront
• Fee Setup
• Epoch related stuff
• EVM
• Bridge
• Migration

I think the overall goals could be:

• core bootstrapping code is easy to read/maintain/extend
• developers can create custom bootstrapping modules that can be imported by others and multiple modules should be importable at one time without a major hassle
• unit/integration tests should be able to list their bootstrapping dependencies more clearly and explicitly
• try to reduce the downstream impact of changes in the bootstrapping procedure

[bluesign]

@janezpodhostnik agreed this would be a perfect solution.

• the ordering of how you add modules should not matter

this one is tricky, I think in normal cases also you would want [a, b, c] should be [a, b] + [c] ( like I would not want all my import addresses change if I bootstrap additional module especially with persistence ( in emulator scope ) )