Merge pull request #2308 from dfinity/update-resource-limits

Update Resource Limits & Add new doc on Canister Storage

docs/developer-docs/backend/motoko/upgrading.md

@@ -5,7 +5,8 @@ Upgrading a canister allows you to preserve the existing state of a deployed can
 
 Motoko provides high-level support for preserving a canister's state using the Internet Computer's stable memory through a feature known as stable storage. This feature is designed to accommodate changes to both the Motoko compiler and the application data. 
 
-Stable storage is a Motoko-specific feature that uses ICP's stable memory data persistence feature. Stable memory is used to store data that persists across canister upgrades. The maximum data storage size of stable memory is 96GiB if the subnet can accomodate it. In comparison, heap storage refers to the regular Wasm data storage for a canister. Heap storage is not persisted across canister upgrades and is limited to 4GiB. 
+Stable storage is a Motoko-specific feature that uses ICP's stable memory data persistence feature. Stable memory is used to store data that persists across canister upgrades. The maximum data storage size of stable memory is 96GiB if the subnet can accommodate it. In comparison, heap memory (also referred to as "main memory") refers to the regular Wasm data storage for a canister. Heap memory is not persisted across canister upgrades and is limited to 4GiB. 
+
 
 Consider the following example: you have a dapp that manages professional profiles and social connections. To add a new feature to the dapp, you need to be able to update the canister code without losing any of the previously stored data. A canister upgrade enables you to update existing canister identifiers with program changes without losing the program state.
 

docs/developer-docs/backend/rust/7-upgrading.md

@@ -18,7 +18,7 @@ When a canister needs to be upgraded, the following workflow is used:
 
 Stable memory is a data persistence feature on ICP. Stable memory is used to store data that persists across canister upgrades. The maximum data storage size of stable memory is 96GiB if the subnet can accommodate it. 
 
-In comparison, heap storage refers to the regular Wasm data storage for a canister. Heap storage is not persisted across canister upgrades and is limited to 4GiB. 
+In comparison, heap memory (also referred to as "main memory") refers to the regular Wasm data storage for a canister. Heap memory is not persisted across canister upgrades and is limited to 4GiB. 
 
 Stable memory can be viewed as the communication channel between old and new versions of a canister. As good practice, communication protocols should be versioned. In some cases, developers may want to radically change something such as the serialization format or the stable data layout of their canister. In radical changes like these, the stable memory decoding mechanism may need to guess the data's format, which can become messy and complicated. To make this process easier, stable memory versioning should be planned for. It can be as simple as declaring that the first byte of the canister's stable memory will be used to represent the version number. 
 

docs/developer-docs/production/resource-limits.md

@@ -9,26 +9,41 @@ The limits depend on the message type as shown in the following table.
 
 ## Resource constraints and limits
 
-| Resource                                                                             | Constraint  |
+| Message resource limits                                                              | Constraint  |
 | ------------------------------------------------------------------------------------ | ----------- |
 | Message queue limit, messages per canister                                           | 500         |
 | Maximum ingress message payload                                                      | 2MB         |
 | Maximum cross-net inter-canister message payload                                     | 2MB         |
 | Maximum same-subnet inter-canister message payload (may be deprecated at some point) | 10MB        |
 | Maximum response size                                                                | 2MB         |
+
+| Instruction resource limits                                                          | Constraint  |
+| ------------------------------------------------------------------------------------ | ----------- |
 | Instruction limit, instructions per update call/heartbeat/timer                      | 20 Billion  |
 | Instruction limit, instructions per query calls                                      | 5 Billion   |
 | Instruction limit, instructions per canister install/upgrade                         | 200 Billion |
 | Instruction limit, instructions per inspect_message                                  | 200 Million |
+
+| Subnet limits                                                                        | Constraint  |
+| ------------------------------------------------------------------------------------ | ----------- |
 | Subnet capacity (total memory available per subnet)                                  | 700GiB      |
-| Wasm heap size, per canister                                                         | 4GiB        |
+
+| Memory resource limits                                                               | Constraint  |
+| ------------------------------------------------------------------------------------ | ----------- |
+| Wasm heap memory, per canister                                                       | 4GiB        |
 | Wasm stable memory, per canister                                                     | 96GiB       |
 | Wasm custom sections, per subnet                                                     | 2GiB        |
 | Wasm custom sections, per canister                                                   | 1MiB        |
 | Wasm custom sections, sections per canister                                          | 16          |
 | Wasm code section, per canister                                                      | 10MiB       |
+
+| Query call resource limits                                                           | Constraint  |
+| ------------------------------------------------------------------------------------ | ----------- |
 | Query calls execution threads, per replica node                                      | 4           |
 | Query calls execution threads, per canister                                          | 2           |
+
+| Update call resource limits                                                          | Constraint  |
+| ------------------------------------------------------------------------------------ | ----------- |
 | Update calls execution threads, per subnet                                           | 4           |
 | Update calls execution threads, per canister                                         | 1           |
 

docs/developer-docs/production/storage.md

@@ -0,0 +1,112 @@
+# Canister storage
+
+## Overview
+
+When developing projects on ICP, there are two primary forms of data storage that your canisters can utilize. The first type of storage is the canister's **heap memory**, sometimes referred to as the canister's *main memory*. Heap memory is **temporary**, and any data stored in a canister's heap memory is cleared whenever the canister is stopped or upgraded. 
+
+The second type of storage available to a canister is **stable memory**. Stable memory is a unique feature of ICP that  defines a separate data store aside from a canister's regular Wasm heap memory data storage. It is a secondary storage type that can be used to store data long-term, since all data stored in stable memory will persist across all canister processes.
+
+## Heap memory
+
+Heap memory refers to a canister's regular Wasm memory. It is not persisted, and does not store data long-term. A canister's heap memory is cleared whenever a canister is stopped or upgraded. Heap memory is used by default for storing things such as variable values, the result of an executed function, or other arguments passed to your canister. 
+
+Heap memory is limited to a maximum of 4GiB of data.
+
+## Stable memory
+
+Stable memory is a feature unique to the Internet Computer Protocol that provides a long-term, persistent data storage option separate from a canister's heap memory. When a canister is stopped or upgraded, the data stored in stable memory is not cleared or removed. The stable memory is preserved throughout the process while any other WebAssembly state is discarded. 
+
+To use stable memory, you must anticipate which portions of the canister's data that you want to persist across upgrades by indicating within the canister code. More information about this can be found in the section below, [Storage handling in different languages](#storage-handling-in-different-languages).
+
+By default, a canister's stable memory is empty. The maximum storage limit for stable memory is 96GiB if the subnet the canister is deployed on can accommodate it. If a canister uses more than 96GiB of stable memory, the canister will trap and become unrecoverable. 
+
+## Storage cost
+
+Storage cost is calculated on the GB of storage used by a canister per second, costing `127_000` cycles on a 13-node subnet and `127_000 / 13 * 34` cycles on a subnet with 34 nodes. In USD, this works out to about $0.431 and $1.127 respectively for storing 1 GB of data for a 30-day month. The cost is the same whether the canister is using heap memory, stable memory, or both. 
+
+You can learn more about storage costs [here](/docs/current/developer-docs/gas-cost).
+
+## Storage handling in different languages
+
+### Motoko storage handling
+
+In Motoko canisters, stable memory can be utilized through the Motoko **stable storage** and **stable variable** features. Stable storage is a Motoko-specific term used to refer Motoko's implementation of stable memory used to persist data across canister upgrades. Stable variables are Motoko defined variables that use the `stable` modifying keyword to indicate that the variable's value should persist across canister upgrades, such as:
+
+```motoko
+actor Counter {
+
+  stable var value = 0;
+
+  public func inc() : async Nat {
+    value += 1;
+    return value;
+  };
+}
+```
+
+To utilize stable memory when upgrading a Motoko canister, the following workflow can be used:
+
+- The canister must be stopped.
+- A `pre_upgrade` hook can be executed if one is defined. `pre_upgrade` hooks can be used for functions such as creating a backup of the canister's data. 
+- The canister's heap memory is discarded and a new version of the canister's Wasm module is initialized. Data stored in stable memory is made available to the new Wasm module.
+- The canister's new code is installed using the `--mode upgrade` flag.
+- The canister is started and now runs the newly upgraded code.
+- Stable variables are re-loaded as part of a `post_upgrade` hook. Stable memory bytes that stored those variables are overwritten with zeroes to minimize stable memory costs for the canister.
+
+#### Garbage collection
+
+A garbage collection process is an automatic utility that is used to manage the amount of memory used. For heap memory, garbage collection is used to remove unreferenced or dead objects in order to free up otherwise allocated heap memory. 
+
+In Motoko, the default garbage collection process uses a copying approach, which depends on the size of the amount of heap memory currently used. In contrast, an additional garbage collector that uses a marking approach can be selected, which is based on the amount of free heap memory. These garbage collection methods are triggered when there are enough changes made to heap memory since the last round of garbage collection. Garbage collection can be forced to run after every message using the `--force-gc` flag in the project's `dfx.json` file:
+
+```json
+  "defaults": {
+    "build": {
+      "packtool": "",
+      "args": "--force-gc"
+    }
+  },
+```
+
+Both of these garbage collectors, however, are unable to collect the entire heap memory pool due to the instruction limit per message on ICP. Since these garbage collectors cannot collect the entire heap memory pool, canisters do not benefit from fully utilizing the entire 4GiB memory pool, as there must be some heap memory space left for the garbage collector to operate.
+
+A beta incremental garbage collection process is available, which uses incremental messages that distributes the garbage collection work across multiple messages when needed. This form of garbage collection allows for the allocation of up to 3x more heap space after it has been run, while consuming less cycles on average. Using this garbage collection service, canisters can benefit from using the entire 4GiB of heap memory, as it can garbage collect the entire heap memory pool. 
+
+The incremental garbage collector can be used by specifying the `--incremental-gc` compiler flag in a project's `dfx.json` file, such as:
+
+```json
+{
+  "canisters": {
+    “my_dapp”: {
+       "main": "src/my-dapp.mo",
+       "type": "motoko",
+       "args" : "--incremental-gc"
+    },
+  },
+}
+```
+
+:::info
+This garbage collector is still in beta testing and should be used with caution.
+:::
+
+You can learn more about the incremental garbage collector [here](https://github.com/dfinity/motoko/pull/3837). 
+
+
+### Rust storage handling
+
+To utilize stable memory for canisters written in Rust, the crates [ic-stable-memory](https://github.com/seniorjoinu/ic-stable-memory) and [ic-stable-structures](https://github.com/dfinity/stable-structures) can be used. It is recommended to review the documentation for these crates to learn more, or review the tutorial on the stable structures crate [here](https://mmapped.blog/posts/14-stable-structures.html).
+
+As good practice, stable memory should be versioned since the stable memory decoding mechanism may need to guess the data's format in situations where the serialization format or stable data layout of a canister drastically changes. To make this process easier, stable memory should be versioned. This can be as simple as declaring that the first byte of the canister's stable memory will be used to represent the version number.
+
+To utilize stable memory when upgrading a Rust canister, the following workflow can be used:
+
+- The canister must be stopped.
+- A `pre_upgrade` hook can be executed if one is defined. `pre_upgrade` hooks can be used for functions such as creating a backup of the canister's data. 
+- The canister's heap memory is discarded and a new version of the canister's Wasm module is initialized. Data stored in stable memory is made available to the new Wasm module.
+- The canister's new code is installed using the `--mode upgrade` flag.
+- The canister is started and now runs the newly upgraded code.
+- A `post_upgrade` hook is called on the newly created instance if one is defined. The init function is not executed.
+
+
+

docs/samples/nft.md

@@ -33,7 +33,7 @@ The NFT canister example contains all those cases and shows how it can be done.
 Since the basic functions required in [DIP-721](https://github.com/Psychedelic/DIP721) are very straightforward to implement, this section only discusses how the above ideas are handled and implemented.
 
 ### Stable storage for canister upgrades
-During canister code upgrades, any data stored in a canister's heap storage is not persisted between different canister calls. Only memory in stable memory is carried over.
+During canister code upgrades, any data stored in a canister's heap memory is not persisted between different canister calls. Only memory in stable memory is carried over.
 Because of that it is necessary to write all data to stable memory before the upgrade happens. After the upgrade, it is normal to load data from stable memory into memory
 during the `post_upgrade` function. The `post_upgrade` function is called by the system after the upgrade happened.
 In case an error occurs during any part of the upgrade (including `post_upgdrade`), the entire upgrade is reverted.

docs/samples/persistent-storage.md

@@ -1,4 +1,4 @@
-# Persistent storage
+# Example: Dapp using persistent storage
 
 ## Overview
 The example dapp shows how to build a simple dapp in Motoko, which will have persistent storage. The dapp is a simple counter, which will increment a counter, retrieve the counter value and reset the counter value by calling backend functions. The functions are exposed through a Candid interface.

docs/tutorials/developer-journey/level-2/2.1-storage-persistence.md

@@ -2,7 +2,7 @@
 
 ## Overview
 
-The Internet Computer handles persistent data storage using a feature known as **stable memory**. **Stable memory** is a unique feature of the Internet Computer that defines a data store separate from the canister's regular Wasm memory data store, which is known as **heap storage**. A canister's heap storage is not persistent storage and does not persist across canister upgrades; canister data and state stored in heap storage is removed when a canister is upgraded or reinstalled. For immutable canisters that use less than the heap storage limit of 4GiB, heap storage can be used. For larger canisters, and especially those that intend to be upgraded and changed over time, stable memory is important since it persists across canister upgrades, has a much larger storage capacity than heap storage, and is very beneficial for vertically scaling a dapp.
+The Internet Computer handles persistent data storage using a feature known as **stable memory**. **Stable memory** is a unique feature of the Internet Computer that defines a data store separate from the canister's regular Wasm memory data store, which is known as **heap memory**. A canister's heap memory is not persistent storage and does not persist across canister upgrades; canister data and state stored in heap memory is removed when a canister is upgraded or reinstalled. For immutable canisters that use less than the heap memory limit of 4GiB, heap memory can be used. For larger canisters, and especially those that intend to be upgraded and changed over time, stable memory is important since it persists across canister upgrades, has a much larger storage capacity than heap memory, and is very beneficial for vertically scaling a dapp.
 
 To use stable memory requires anticipating and indicating which canister data you want to be retained after a canister upgrade. For some canisters, this data might be all of the canister's data, while for others it may be only certain parts or none. By default, stable memory starts empty, and can hold up to 96GiB as long as the subnet the canister is running on has the available space. If a canister uses more stable memory than 96GiB, the canister will trap and become unrecoverable.
 
@@ -12,7 +12,7 @@ There are several terms associated with memory and storage on ICP. To avoid conf
 
 - **Stable memory:** Stable memory refers to the Internet Computer's long-term data storage feature. Stable memory is not language specific, and can be utilized by canisters written in Motoko, Rust, or any other language. Stable memory can hold up to 96GiB if the subnet can accommodate it.
 
-- **Heap storage:** Heap storage refers to the regular Wasm memory data store for each canister. This storage is temporary and is not persisted across canister upgrades. Data stored in heap storage is removed when the canister is upgraded or reinstalled. Heap storage is limited to 4GiB.
+- **Heap memory:** Heap memory refers to the regular Wasm memory data store for each canister. This storage is temporary and is not persisted across canister upgrades. Data stored in heap memory is removed when the canister is upgraded or reinstalled. Heap memory is limited to 4GiB.
 
 - **Stable storage:** Stable storage is a Motoko-specific term that refers to the Motoko stable storage feature. Stable storage uses ICP's stable memory to persist data across canister upgrades.
 
@@ -46,7 +46,7 @@ For canisters written in Rust, stable memory can be utilized through two Rust cr
 - Then, the system discards the canister's heap memory and initializes a new version of the canister's Wasm module. Stable memory is preserved and made available to the new Wasm module.
 - Next, the new canister code is installed using the `--mode upgrade` flag.
 - Then, the canister is started, now running the newly upgraded code.
-- A `post_upgrade` hook is called on the newly created instance if one is defined it. The `init` function is not executed.
+- A `post_upgrade` hook is called on the newly created instance if one is defined. The `init` function is not executed.
 
 ## Stable storage and stable variables