Merge pull request cockroachdb#3193 from cockroachdb/marc/draft_encryption_at_rest

Draft: encryption at rest

Author: rmloveland

_includes/sidebar-data-v2.1.json

@@ -330,6 +330,12 @@
                 "urls": [
                   "/${VERSION}/roles.html"
                 ]
+              },
+              {
+                "title": "Encryption At Rest",
+                "urls": [
+                  "/${VERSION}/encryption.html"
+                ]
               }
             ]
           },

v2.1/encryption.md

@@ -0,0 +1,182 @@
+---
+title: Encryption At Rest
+summary: Encryption At Rest encrypts node data on local disk transparently.
+toc: false
+---
+
+<span class="version-tag">New in v2.1:</span>
+Encryption At Rest provides transparent encryption for node data on local disk.
+
+{% include experimental-warning.md %}
+
+<div id="toc"></div>
+
+## Overview
+
+Encryption at rest allows encryption of all files on disk using AES in counter mode, with all key
+sizes allowed.
+
+Encryption is performed in the [storage layer](architecture/storage-layer.html) and configured per store.
+All files used by the store, regardless of contents, are encrypted with the desired algorithm.
+
+To allow arbitrary rotation schedules and ensure security of the keys, we use two layers of keys:
+
+| Level | Description |
+|-|-|
+| Store keys | Provided by the user in a file. They are used to encrypt the list of data keys (see below).<br><br>This is known as a **key encryption key**: it's only purpose is to encrypt other keys.<br><br>Store keys are never persisted by CockroachDB.<br><br>Since very little data is encrypted using this key, it can have a very long lifetime without risk of reuse. |
+| Data keys | Automatically generated by CockroachDB. They are used to encrypt all files on disk.<br><br> This is known as a **data encryption key**.<br><br>Data keys are persisted in a key registry file, encrypted using the store key.<br><br>The key has a short lifetime to avoid reuse. |
+
+Store keys are specified by passing a path to a locally readable file. The file must contain 32 bytes (the key ID)
+followed by the key (16, 24, or 32 bytes). The size of the key dictates the version of AES to use (AES-128, AES-192, or AES-256).
+
+At startup, CockroachDB uses a data key with the same length as the store key. If encryption has just been enabled,
+the key size has changed, or the data key is too old (default lifetime is one week), CockroachDB generates a new data key.
+
+Any new file created by the store uses the currently-active data key. All data keys (both active and previous) are stored in a key registry file and encrypted with the active store key.
+
+CockroachDB does not currently force re-encryption of older files but instead relies on normal RocksDB churn to slowly rewrite all data with the desired encryption.
+
+## Rotating Keys
+
+Key rotation is necessary for encryption at rest for multiple reasons:
+
+* prevent key reuse with the same encryption parameters (after many files)
+* reduce the risk of key exposure
+
+Store keys are specified by the user and must be rotated by specifying different keys.
+This is done by setting the `key` parameter of the `--enterprise-encryption` flag to the path to the new key,
+and `old-key` to the previously-used key.
+
+Data keys will automatically be rotated at startup if any of the following conditions are met:
+
+* the active store key changed
+* the encryption type changed (different key size, or plaintext to/from encryption)
+* the current data key is `rotation-period` old or more.
+
+Once rotated, an old key cannot be made the active key again.
+
+Upon store key rotation the data keys registry is decrypted using the old key and encrypted with the new
+key. The new data key is used to encrypt all new data from this point on.
+
+## Changing encryption type
+
+The user can change from plaintext to encryption, between encryption algorithms (various key sizes),
+or from encryption to plaintext.
+
+When changing the encryption type to plaintext, the data key registry is no longer encrypted and all previous
+data keys are readable by anyone. All data on the store is effectively readable.
+
+When changing from plaintext to encryption, it will take some time for all data to eventually be re-written
+and encrypted.
+
+## Recommendations
+
+There are a number of considerations to keep in mind when running with encryption.
+
+Key management is the most dangerous aspect of encryption. The following rules should be kept in mind:
+
+* make sure only the unix user running the `cockroach` process has access to the keys
+* do not store the keys on the same partition/drive as the cockroach data. It is best to load keys at run time from a separate system (eg: keywhiz, vault)
+* rotate store keys frequently (every few weeks to months)
+* keep the data key rotation period low (default is one week)
+
+A few other recommendations apply for best security practices:
+
+* do not switch from encrypted to plaintext, this leaks data keys. Once transitioned to plaintext, all data must be considered reachable.
+* do not copy the encrypted files as the data keys are not easily available.
+* if encryption is desired, start a node with it enabled from first run without ever running in plaintext.
+
+Note that backups taken with the [`BACKUP`](backup.html) statement are not encrypted (whether you are using this feature or not). If you want encrypted backups, you will need to encrypt your backup files using your preferred encryption method.
+
+## Example
+
+### Generating key files
+
+Cockroach determines which encryption algorithm to use based on the size of the key file.
+The key file must contain random data making up the key ID (32 bytes) and the actual key (16, 24, or 32
+bytes depending on the encryption algorithm).
+
+| Algorithm | Key size | Key file size |
+|-|-|-|
+| AES-128 | 128 bits (16 bytes) | 48 bytes |
+| AES-192 | 192 bits (24 bytes) | 56 bytes |
+| AES-256 | 256 bits (32 bytes) | 64 bytes |
+
+Generating a key file can be done using the `cockroach` CLI:
+
+{% include copy-clipboard.html %}
+~~~ shell
+$ cockroach gen encryption-key -s 128 /path/to/my/aes-128.key
+~~~
+
+Or the equivalent [openssl](https://www.openssl.org/docs/man1.0.2/apps/openssl.html) CLI command:
+
+{% include copy-clipboard.html %}
+~~~ shell
+$ openssl rand -out /path/to/my/aes-128.key 48
+~~~
+
+### Starting a node with encryption
+
+Encryption is configured at node start time using the `--enterprise-encryption` command line flag.
+The flag specifies the encryption options for one of the stores on the node. If multiple stores exist,
+the flag must be specified for each store.
+
+The flag takes the form: `--enterprise-encryption=path=<store path>,key=<key file>,old-key=<old key file>,rotation-period=<period>`.
+
+The allowed components in the flag are:
+
+| Component | Requirement | Description |
+|-|-|-|
+| path            | required | Path of the store to apply encryption to |
+| key             | required | Path to the key file to encrypt data with, or `plain` for plaintext |
+| old-key         | required | Path to key file the data is encrypted with, or `plain` for plaintext |
+| rotation-period | optional | How often data keys should be automatically rotated |
+
+The `key` and `old-key` components must **always** be specified. They allow for transitions between
+encryption algorithms, and between plaintext and encrypted.
+
+Starting a node for the first time using AES-128 encryption can be done using:
+{% include copy-clipboard.html %}
+~~~ shell
+$ cockroach start --store=cockroach-data --enterprise-encryption=path=cockroach-data,key=/path/to/my/aes-128.key,old-key=plain
+~~~
+
+{{site.data.alerts.callout_danger}}
+Once specified for a given store, the `--enterprise-encryption` flag must always be present.
+{{site.data.alerts.end}}
+
+### Checking encryption status
+
+Encryption status can be see on the node's stores report, reachable through: `http(s)://nodeaddress:8080/#/reports/stores/local`.
+
+The report shows the currently-active store key as well as the currently-active data key. The `encryption_type` field
+reflects the algorithm currently in use.
+
+### Changing encryption algorithm or keys
+
+Encryption type and keys can be changed at any time by restarting the node.
+To change keys or encryption type, the `key` component of the `--enterprise-encryption` flag is set to the new key,
+while the key previously used must be specified in the `old-key` component.
+
+For example, we can switch from AES-128 to AES-256 using:
+
+{% include copy-clipboard.html %}
+~~~ shell
+$ cockroach start --store=cockroach-data --enterprise-encryption=path=cockroach-data,key=/path/to/my/aes-256.key,old-key=/path/to/my/aes-128.key
+~~~
+
+Upon starting, the node will read the existing data keys using the old encryption key (`aes-128.key`), then rewrite
+the data keys using the new key (`aes-256.key`). A new data key will be generated to match the desired `AES-256` algorithm.
+
+The new key can be seen as active in the admin UI under the stores report page.
+
+To disable encryption, specify `key=plain`. The data keys will be stored in plaintext and new data will not be encrypted.
+
+To rotate keys, specify `key=/path/to/my/new-aes-128.key` and `old-key=/path/to/my/old-aes-128.key`. The data keys
+will be decrypted using the old key then encrypted using the new key. A new data key will also be generated.
+
+## See Also
+
++ [Enterprise Licensing](enterprise-licensing.html)
++ [`BACKUP`](backup.html)