Skip to content

Commit b6c1305

Browse files
authored
Update architecture/index.md (#439)
Updated the Architecture topic with the following: - New intro detailing the long term tde goals in a paragraph - Updated the ## Typical setup scenarios topic with better writing and improved flow - Added note to WAL Encryption that it is not to be used in prod env - General small fixes to paragraphs, wrongly written words and such
1 parent dfcef9f commit b6c1305

File tree

1 file changed

+45
-68
lines changed
  • contrib/pg_tde/documentation/docs/architecture

1 file changed

+45
-68
lines changed

contrib/pg_tde/documentation/docs/architecture/index.md

Lines changed: 45 additions & 68 deletions
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,12 @@
22

33
`pg_tde` is a **customizable, complete, data at rest encryption extension** for PostgreSQL.
44

5-
Let's break down what it means.
5+
The following sections break down the key architectural components of this design.
66

77
**Customizable** means that `pg_tde` aims to support many different use cases:
88

99
* Encrypting either every table in every database or only some tables in some databases
10-
* Encryption keys can be stored on various external key storage servers including Hashicorp Vault, KMIP servers.
10+
* Encryption keys can be stored on various external key storage servers including HashiCorp Vault and KMIP servers.
1111
* Using one key for everything or different keys for different databases
1212
* Storing every key on the same key storage, or using different storages for different databases
1313
* Handling permissions: who can manage database specific or global permissions, who can create encrypted or not encrypted tables
@@ -20,7 +20,7 @@ Let's break down what it means.
2020
* Indexes
2121
* Sequences
2222
* Temporary tables
23-
* Write Ahead Log (WAL)
23+
* Write Ahead Log (WAL), still in beta. **Do not enable this feature in production environments**.
2424

2525
**Extension** means that `pg_tde` should be implemented only as an extension, possibly compatible with any PostgreSQL distribution, including the open source community version. This requires changes in the PostgreSQL core to make it more extensible. Therefore, `pg_tde` currently works only with the [Percona Server for PostgreSQL](https://docs.percona.com/postgresql/17/index.html) - a binary replacement of community PostgreSQL and included in Percona Distribution for PostgreSQL.
2626

@@ -82,13 +82,16 @@ Later decisions are made using a slightly modified Storage Manager (SMGR) API: w
8282

8383
### WAL encryption
8484

85+
!!! note
86+
The WAL encryption feature is currently in beta and is not effective unless explicitly enabled. It is not yet production ready. **Do not enable this feature in production environments**.
87+
8588
WAL encryption is controlled globally via a global GUC variable, `pg_tde.wal_encrypt`, that requires a server restart.
8689

8790
WAL keys also contain the [LSN](https://www.postgresql.org/docs/17/wal-internals.html) of the first WAL write after key creation. This allows `pg_tde` to know which WAL ranges are encrypted or not and with which key.
8891

89-
The setting only controls writes so that only WAL writes are encrypted when WAL encryption is enabled. This means that WAL files can contain both encrypted and unencrpyted data, depending on what the status of this variable was when writing the data.
92+
The setting only controls writes so that only WAL writes are encrypted when WAL encryption is enabled. This means that WAL files can contain both encrypted and unencrypted data, depending on what the status of this variable was when writing the data.
9093

91-
`pg_tde` keeps track of the encryption status of WAL records using internal keys. When the server is restarted it writes a new internal key if WAL encryption is enabled, or if it is disabled and was previously enabled it writes a dummy key signalling that WAL encryption ended.
94+
`pg_tde` keeps track of the encryption status of WAL records using internal keys. When the server is restarted it writes a new internal key if WAL encryption is enabled, or if it is disabled and was previously enabled it writes a dummy key signaling that WAL encryption ended.
9295

9396
With this information the WAL reader code can decide if a specific WAL record has to be decrypted or not and which key it should use to decrypt it.
9497

@@ -156,7 +159,7 @@ With these details `pg_tde` does the following based on user operations:
156159
* Uploads a new principal key to it after this key is created
157160
* Retrieves the principal key from the service when it is required for decryption
158161

159-
Retreival of the principal key is cached so it only happens when necessary.
162+
Retrieval of the principal key is cached so it only happens when necessary.
160163

161164
### Key provider management
162165

@@ -169,24 +172,23 @@ For such situations, `pg_tde` also provides [command line tools](../command-line
169172
### Sensitive key provider information
170173

171174
!!! important
172-
173175
Authentication details for key providers are sensitive and must be protected.
174176
Do not store these credentials in the `$PGDATA` directory alongside the database. Instead, ensure they are stored in a secure location with strict file system permissions to prevent unauthorized access.
175177

176178
## User interface
177179

178-
### Setting up pg_tde
180+
### Set up pg_tde
179181

180-
To use `pg_tde`, users are required to:
182+
To get started with `pg_tde`, follow these steps:
181183

182184
* Add `pg_tde` to the `shared_preload_libraries` in `postgresql.conf` as this is required for the SMGR extensions
183185
* Execute `CREATE EXTENSION pg_tde` in the databases where they want to use encryption
184186
* Optionally, enable `pg_tde.wal_encrypt` in `postgresql.conf`
185-
* Optionally, disable `pg_tde.inherit_global_providers` in `postgresql.conf` (enabled by default)
187+
* Optionally, disable `pg_tde.inherit_global_providers` in `postgresql.conf` (it is enabled by default)
186188

187-
### Adding providers
189+
### Add providers
188190

189-
Keyring providers can be added to either the global or to the database specific scope.
191+
You can add keyring providers to either the global or database specific scope.
190192

191193
If `pg_tde.inherit_global_providers` is `on`, global providers are visible for all databases, and can be used.
192194
If `pg_tde.inherit_global_providers` is `off`, global providers are only used for WAL encryption.
@@ -203,7 +205,7 @@ To add a database specific provider:
203205
pg_tde_add_database_key_provider_<TYPE>('provider_name', ... details ...)
204206
```
205207

206-
### Changing providers
208+
### Change providers
207209

208210
To change a value of a global provider:
209211

@@ -217,13 +219,9 @@ To change a value of a database specific provider:
217219
pg_tde_change_database_key_provider_<TYPE>('provider_name', ... details ...)
218220
```
219221

220-
These functions also allow changing the type of a provider.
221-
222-
The functions however do not migrate any data. They are expected to be used during infrastructure migration, for example when the address of a server changes.
222+
These functions also allow changing the type of a provider but **do not** migrate any data. They are expected to be used during infrastructure migration, for example when the address of a server changes.
223223

224-
Note that in these functions do not verify the parameters. For that, see `pg_tde_verify_key`.
225-
226-
### Changing providers from the command line
224+
### Change providers from the command line
227225

228226
To change a provider from a command line, `pg_tde` provides the `pg_tde_change_key_provider` command line tool.
229227

@@ -233,11 +231,12 @@ This tool work similarly to the above functions, with the following syntax:
233231
pg_tde_change_key_provider <dbOid> <providerType> ... details ...
234232
```
235233

236-
Note that since this tool is expected to be offline, it bypasses all permission checks!
234+
!!! note
235+
Since this tool is expected to be offline, it bypasses all permission checks. This is also the reason why it requires a `dbOid` instead of a name, as it has no way to process the catalog and look up names.
237236

238-
This is also the reason why it requires a `dbOid` instead of a name, as it has no way to process the catalog and look up names.
237+
This tool does not validate any parameters.
239238

240-
### Deleting providers
239+
### Delete providers
241240

242241
Providers can be deleted by using the following functions:
243242

@@ -250,7 +249,7 @@ For database specific providers, the function first checks if the provider is us
250249

251250
For global providers, the function checks if the provider is used anywhere, WAL or any specific database, and returns an error if it is.
252251

253-
### Listing/querying providers
252+
### List/query providers
254253

255254
`pg_tde` provides 2 functions to show providers:
256255

@@ -259,7 +258,18 @@ For global providers, the function checks if the provider is used anywhere, WAL
259258

260259
These functions return a list of provider names, type and configuration.
261260

262-
### Creating and rotating keys
261+
### Provider permissions
262+
263+
`pg_tde` implements access control based on execution rights on the administration functions.
264+
265+
For keys and providers administration, it provides two pair of functions:
266+
267+
```sql
268+
pg_tde_GRANT_database_key_management_TO_role
269+
pg_tde_REVOKE_database_key_management_FROM_role
270+
```
271+
272+
### Create and rotate keys
263273

264274
Principal keys can be created using the following functions:
265275

@@ -279,7 +289,7 @@ pg_tde_set_default_key_using_(global/database)_key_provider('key-name', 'provide
279289

280290
With `pg_tde.inherit_global_key_providers`, it is also possible to set up a default global principal key, which will be used by any database which has the `pg_tde` extension enabled, but doesn't have a database specific principal key configured using `pg_tde_set_key_using_(global/database)_key_provider`.
281291

282-
With this feature, it is possible for the entire database server to easily use the same principal key for all databases, completely disabling multi-tenency.
292+
With this feature, it is possible for the entire database server to easily use the same principal key for all databases, completely disabling multi-tenancy.
283293

284294
#### Manage a default key
285295

@@ -310,7 +320,13 @@ The `pg_tde_delete_key()` function unsets the principal key for the current data
310320

311321
`pg_tde_verify_key()` checks that the key provider is accessible, that the current principal key can be downloaded from it, and that it is the same as the current key stored in memory - if any of these fail, it reports an appropriate error.
312322

313-
### Creating encrypted tables
323+
### Key permissions
324+
325+
Users with management permissions to a specific database `(pg_tde_(grant/revoke)_(global/databse)_key_management_(to/from)_role)` can change the keys for the database, and use the current key functions. This includes creating keys using global providers, if `pg_tde.inherit_global_providers` is enabled.
326+
327+
Also the `pg_tde_(grant/revoke)_database_key_management_to_role` function deals with only the specific permission for the above function: it allows a user to change the key for the database, but not to modify the provider configuration.
328+
329+
### Create encrypted tables
314330

315331
To create an encrypted table or modify an existing table to be encrypted, use the following commands:
316332

@@ -319,47 +335,8 @@ CREATE TABLE t1(a INT) USING tde_heap;
319335
ALTER TABLE t1 SET ACCESS METHOD tde_heap;
320336
```
321337

322-
### Changing the `pg_tde.inherit_global_keys` setting
323-
324-
It is possible for users to use `pg_tde` with `inherit_global_keys = on`, refer to global keys / keyrings in databases, and then change this setting to `off`.
325-
326-
In this case existing references to global providers, or the global default principal key will remain working as before, but new references to the global scope can't be made.
327-
328-
## Typical setup scenarios
329-
330-
### Simple "one principal key" encryption
331-
332-
1. Passing the option from the postgres config file the extension: `shared_preload_libraries=‘pg_tde’`
333-
2. `CREATE EXTENSION pg_tde;` in `template1`
334-
3. Adding a global key provider
335-
4. Adding a default principal key using the same global provider
336-
5. Enable WAL encryption to use the default principal key using `ALTER SYSTEM SET pg_tde.wal_encrypt=‘ON’`
337-
6. Restart the server
338-
7. Optionally: setting the `default_table_access_method` to `tde_heap` so that tables are encrypted by default
339-
340-
Database users don't need permissions to any of the encryption functions:
341-
encryption is managed by the admins, normal users only have to create tables with encryption, which requires no specific permissions.
342-
343-
### One key storage, but different keys per database
344-
345-
1. Installing the extension: `shared_preload_libraries` + `pg_tde.wal_encrypt`
346-
2. `CREATE EXTENSION pg_tde;` in `template1`
347-
3. Adding a global key provider
348-
4. Changing the WAL encryption to use the proper global key provider
349-
5. Giving users that are expected to manage database keys permissions for database specific key management, but not database specific key provider management:
350-
specific databases HAVE to use the global key provider
351-
352-
Note: setting the `default_table_access_method` to `tde_heap` is possible, but instead of `ALTER SYSTEM` only per database using `ALTER DATABASE`, after a principal key is configured for that specific database.
353-
354-
Alternatively `ALTER SYSTEM` is possible, but table creation in the database will fail if there's no principal key for the database, that has to be created first.
355-
356-
### Complete multi tenancy
357-
358-
1. Installing the extension: `shared_preload_libraries` + `pg_tde.wal_encrypt` (that's not multi tenant currently)
359-
2. `CREATE EXTENSION pg_tde;` in any database
360-
3. Adding a global key provider for WAL
361-
4. Changing the WAL encryption to use the proper global key provider
338+
### Change the pg_tde.inherit_global_keys setting
362339

363-
No default configuration: key providers / principal keys are configured as a per database level, permissions are managed per database
340+
It is possible to use `pg_tde` with `inherit_global_keys = on`, refer to the global keys or keyrings in databases, and then change this setting to `off`.
364341

365-
Same note about `default_table_access_method` as above - but in a multi tenant setup, `ALTER SYSTEM` doesn't make much sense.
342+
In this case, existing references to global providers or the global default principal key keep working as before, but new references to the global scope cannot be made.

0 commit comments

Comments
 (0)