You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Updated the Architecture topic with the following:
- New intro detailing the long term tde goals in a paragraph
- Updated the ## Typical setup scenarios topic with better writing and
improved flow
- Added note to WAL Encryption that it is not to be used in prod env
- General small fixes to paragraphs, wrongly written words and such
Copy file name to clipboardExpand all lines: contrib/pg_tde/documentation/docs/architecture/index.md
+45-68Lines changed: 45 additions & 68 deletions
Original file line number
Diff line number
Diff line change
@@ -2,12 +2,12 @@
2
2
3
3
`pg_tde` is a **customizable, complete, data at rest encryption extension** for PostgreSQL.
4
4
5
-
Let's break down what it means.
5
+
The following sections break down the key architectural components of this design.
6
6
7
7
**Customizable** means that `pg_tde` aims to support many different use cases:
8
8
9
9
* Encrypting either every table in every database or only some tables in some databases
10
-
* Encryption keys can be stored on various external key storage servers including Hashicorp Vault, KMIP servers.
10
+
* Encryption keys can be stored on various external key storage servers including HashiCorp Vault and KMIP servers.
11
11
* Using one key for everything or different keys for different databases
12
12
* Storing every key on the same key storage, or using different storages for different databases
13
13
* Handling permissions: who can manage database specific or global permissions, who can create encrypted or not encrypted tables
@@ -20,7 +20,7 @@ Let's break down what it means.
20
20
* Indexes
21
21
* Sequences
22
22
* Temporary tables
23
-
* Write Ahead Log (WAL)
23
+
* Write Ahead Log (WAL), still in beta. **Do not enable this feature in production environments**.
24
24
25
25
**Extension** means that `pg_tde` should be implemented only as an extension, possibly compatible with any PostgreSQL distribution, including the open source community version. This requires changes in the PostgreSQL core to make it more extensible. Therefore, `pg_tde` currently works only with the [Percona Server for PostgreSQL](https://docs.percona.com/postgresql/17/index.html) - a binary replacement of community PostgreSQL and included in Percona Distribution for PostgreSQL.
26
26
@@ -82,13 +82,16 @@ Later decisions are made using a slightly modified Storage Manager (SMGR) API: w
82
82
83
83
### WAL encryption
84
84
85
+
!!! note
86
+
The WAL encryption feature is currently in beta and is not effective unless explicitly enabled. It is not yet production ready. **Do not enable this feature in production environments**.
87
+
85
88
WAL encryption is controlled globally via a global GUC variable, `pg_tde.wal_encrypt`, that requires a server restart.
86
89
87
90
WAL keys also contain the [LSN](https://www.postgresql.org/docs/17/wal-internals.html) of the first WAL write after key creation. This allows `pg_tde` to know which WAL ranges are encrypted or not and with which key.
88
91
89
-
The setting only controls writes so that only WAL writes are encrypted when WAL encryption is enabled. This means that WAL files can contain both encrypted and unencrpyted data, depending on what the status of this variable was when writing the data.
92
+
The setting only controls writes so that only WAL writes are encrypted when WAL encryption is enabled. This means that WAL files can contain both encrypted and unencrypted data, depending on what the status of this variable was when writing the data.
90
93
91
-
`pg_tde` keeps track of the encryption status of WAL records using internal keys. When the server is restarted it writes a new internal key if WAL encryption is enabled, or if it is disabled and was previously enabled it writes a dummy key signalling that WAL encryption ended.
94
+
`pg_tde` keeps track of the encryption status of WAL records using internal keys. When the server is restarted it writes a new internal key if WAL encryption is enabled, or if it is disabled and was previously enabled it writes a dummy key signaling that WAL encryption ended.
92
95
93
96
With this information the WAL reader code can decide if a specific WAL record has to be decrypted or not and which key it should use to decrypt it.
94
97
@@ -156,7 +159,7 @@ With these details `pg_tde` does the following based on user operations:
156
159
* Uploads a new principal key to it after this key is created
157
160
* Retrieves the principal key from the service when it is required for decryption
158
161
159
-
Retreival of the principal key is cached so it only happens when necessary.
162
+
Retrieval of the principal key is cached so it only happens when necessary.
160
163
161
164
### Key provider management
162
165
@@ -169,24 +172,23 @@ For such situations, `pg_tde` also provides [command line tools](../command-line
169
172
### Sensitive key provider information
170
173
171
174
!!! important
172
-
173
175
Authentication details for key providers are sensitive and must be protected.
174
176
Do not store these credentials in the `$PGDATA` directory alongside the database. Instead, ensure they are stored in a secure location with strict file system permissions to prevent unauthorized access.
175
177
176
178
## User interface
177
179
178
-
### Setting up pg_tde
180
+
### Set up pg_tde
179
181
180
-
To use `pg_tde`, users are required to:
182
+
To get started with `pg_tde`, follow these steps:
181
183
182
184
* Add `pg_tde` to the `shared_preload_libraries` in `postgresql.conf` as this is required for the SMGR extensions
183
185
* Execute `CREATE EXTENSION pg_tde` in the databases where they want to use encryption
184
186
* Optionally, enable `pg_tde.wal_encrypt` in `postgresql.conf`
185
-
* Optionally, disable `pg_tde.inherit_global_providers` in `postgresql.conf` (enabled by default)
187
+
* Optionally, disable `pg_tde.inherit_global_providers` in `postgresql.conf` (it is enabled by default)
186
188
187
-
### Adding providers
189
+
### Add providers
188
190
189
-
Keyring providers can be added to either the global or to the database specific scope.
191
+
You can add keyring providers to either the global or database specific scope.
190
192
191
193
If `pg_tde.inherit_global_providers` is `on`, global providers are visible for all databases, and can be used.
192
194
If `pg_tde.inherit_global_providers` is `off`, global providers are only used for WAL encryption.
@@ -203,7 +205,7 @@ To add a database specific provider:
These functions also allow changing the type of a provider.
221
-
222
-
The functions however do not migrate any data. They are expected to be used during infrastructure migration, for example when the address of a server changes.
222
+
These functions also allow changing the type of a provider but **do not** migrate any data. They are expected to be used during infrastructure migration, for example when the address of a server changes.
223
223
224
-
Note that in these functions do not verify the parameters. For that, see `pg_tde_verify_key`.
225
-
226
-
### Changing providers from the command line
224
+
### Change providers from the command line
227
225
228
226
To change a provider from a command line, `pg_tde` provides the `pg_tde_change_key_provider` command line tool.
229
227
@@ -233,11 +231,12 @@ This tool work similarly to the above functions, with the following syntax:
Note that since this tool is expected to be offline, it bypasses all permission checks!
234
+
!!! note
235
+
Since this tool is expected to be offline, it bypasses all permission checks. This is also the reason why it requires a `dbOid` instead of a name, as it has no way to process the catalog and look up names.
237
236
238
-
This is also the reason why it requires a `dbOid` instead of a name, as it has no way to process the catalog and look up names.
237
+
This tool does not validate any parameters.
239
238
240
-
### Deleting providers
239
+
### Delete providers
241
240
242
241
Providers can be deleted by using the following functions:
243
242
@@ -250,7 +249,7 @@ For database specific providers, the function first checks if the provider is us
250
249
251
250
For global providers, the function checks if the provider is used anywhere, WAL or any specific database, and returns an error if it is.
252
251
253
-
### Listing/querying providers
252
+
### List/query providers
254
253
255
254
`pg_tde` provides 2 functions to show providers:
256
255
@@ -259,7 +258,18 @@ For global providers, the function checks if the provider is used anywhere, WAL
259
258
260
259
These functions return a list of provider names, type and configuration.
261
260
262
-
### Creating and rotating keys
261
+
### Provider permissions
262
+
263
+
`pg_tde` implements access control based on execution rights on the administration functions.
264
+
265
+
For keys and providers administration, it provides two pair of functions:
266
+
267
+
```sql
268
+
pg_tde_GRANT_database_key_management_TO_role
269
+
pg_tde_REVOKE_database_key_management_FROM_role
270
+
```
271
+
272
+
### Create and rotate keys
263
273
264
274
Principal keys can be created using the following functions:
With `pg_tde.inherit_global_key_providers`, it is also possible to set up a default global principal key, which will be used by any database which has the `pg_tde` extension enabled, but doesn't have a database specific principal key configured using `pg_tde_set_key_using_(global/database)_key_provider`.
281
291
282
-
With this feature, it is possible for the entire database server to easily use the same principal key for all databases, completely disabling multi-tenency.
292
+
With this feature, it is possible for the entire database server to easily use the same principal key for all databases, completely disabling multi-tenancy.
283
293
284
294
#### Manage a default key
285
295
@@ -310,7 +320,13 @@ The `pg_tde_delete_key()` function unsets the principal key for the current data
310
320
311
321
`pg_tde_verify_key()` checks that the key provider is accessible, that the current principal key can be downloaded from it, and that it is the same as the current key stored in memory - if any of these fail, it reports an appropriate error.
312
322
313
-
### Creating encrypted tables
323
+
### Key permissions
324
+
325
+
Users with management permissions to a specific database `(pg_tde_(grant/revoke)_(global/databse)_key_management_(to/from)_role)` can change the keys for the database, and use the current key functions. This includes creating keys using global providers, if `pg_tde.inherit_global_providers` is enabled.
326
+
327
+
Also the `pg_tde_(grant/revoke)_database_key_management_to_role` function deals with only the specific permission for the above function: it allows a user to change the key for the database, but not to modify the provider configuration.
328
+
329
+
### Create encrypted tables
314
330
315
331
To create an encrypted table or modify an existing table to be encrypted, use the following commands:
316
332
@@ -319,47 +335,8 @@ CREATE TABLE t1(a INT) USING tde_heap;
319
335
ALTERTABLE t1 SET ACCESS METHOD tde_heap;
320
336
```
321
337
322
-
### Changing the `pg_tde.inherit_global_keys` setting
323
-
324
-
It is possible for users to use `pg_tde` with `inherit_global_keys = on`, refer to global keys / keyrings in databases, and then change this setting to `off`.
325
-
326
-
In this case existing references to global providers, or the global default principal key will remain working as before, but new references to the global scope can't be made.
327
-
328
-
## Typical setup scenarios
329
-
330
-
### Simple "one principal key" encryption
331
-
332
-
1. Passing the option from the postgres config file the extension: `shared_preload_libraries=‘pg_tde’`
333
-
2.`CREATE EXTENSION pg_tde;` in `template1`
334
-
3. Adding a global key provider
335
-
4. Adding a default principal key using the same global provider
336
-
5. Enable WAL encryption to use the default principal key using `ALTER SYSTEM SET pg_tde.wal_encrypt=‘ON’`
337
-
6. Restart the server
338
-
7. Optionally: setting the `default_table_access_method` to `tde_heap` so that tables are encrypted by default
339
-
340
-
Database users don't need permissions to any of the encryption functions:
341
-
encryption is managed by the admins, normal users only have to create tables with encryption, which requires no specific permissions.
342
-
343
-
### One key storage, but different keys per database
344
-
345
-
1. Installing the extension: `shared_preload_libraries` + `pg_tde.wal_encrypt`
346
-
2.`CREATE EXTENSION pg_tde;` in `template1`
347
-
3. Adding a global key provider
348
-
4. Changing the WAL encryption to use the proper global key provider
349
-
5. Giving users that are expected to manage database keys permissions for database specific key management, but not database specific key provider management:
350
-
specific databases HAVE to use the global key provider
351
-
352
-
Note: setting the `default_table_access_method` to `tde_heap` is possible, but instead of `ALTER SYSTEM` only per database using `ALTER DATABASE`, after a principal key is configured for that specific database.
353
-
354
-
Alternatively `ALTER SYSTEM` is possible, but table creation in the database will fail if there's no principal key for the database, that has to be created first.
355
-
356
-
### Complete multi tenancy
357
-
358
-
1. Installing the extension: `shared_preload_libraries` + `pg_tde.wal_encrypt` (that's not multi tenant currently)
359
-
2.`CREATE EXTENSION pg_tde;` in any database
360
-
3. Adding a global key provider for WAL
361
-
4. Changing the WAL encryption to use the proper global key provider
338
+
### Change the pg_tde.inherit_global_keys setting
362
339
363
-
No default configuration: key providers / principal keys are configured as a per database level, permissions are managed per database
340
+
It is possible to use `pg_tde` with `inherit_global_keys = on`, refer to the global keys or keyrings in databases, and then change this setting to `off`.
364
341
365
-
Same note about `default_table_access_method` as above - but in a multi tenant setup, `ALTER SYSTEM` doesn't make much sense.
342
+
In this case, existing references to global providers or the global default principal key keep working as before, but new references to the global scope cannot be made.
0 commit comments