Dealing with big states

[mhemrg]

Lotion stores the state in memory. So how can we deal with big states?

[arsen3d]

Example:

+initialState [];
+app.use(function (state, tx) {
+   state.push(tx);
+})

[mappum]

In my opinion, it's OK to keep the state in memory since it doesn't have to be huge. For example, after 9 years of the Bitcoin blockchain's existence, the UTXO set is still less than 2 GB which is very doable for even low-end hardware. I prefer being smart about not dumping unnecessary data into the state.

However, long-term I'm sure there will be use-cases for large states, so eventually mutating/accessing the state object could be async, and it could all just come from disk. Of course, performance for disk-state will be much slower than if it was kept in memory.

[arsen3d]

@mappum, Is there an example of how to keep the state lean, while persisting and validating the data as it comes in to

app.use(function (state, tx) {

Is it a good idea to store the data via rest API on IPFS for example or are the tx already stored in tendermint anyway.

[mappum]

The txs will already be stored in Tendermint so you don't have to worry about that, but if for instance your blockchain was dealing with large data files then it would make sense to only store the hash/identifier of the data in your state, and store the data itself on IPFS.