Add documentation for failure stores. #1368
Conversation
TBD on recipes. Most links are not complete and need updating from "???".
Co-authored-by: Lee Hinman <[email protected]>
Co-authored-by: Lee Hinman <[email protected]>
Co-authored-by: Lee Hinman <[email protected]>
Co-authored-by: David Kilfoyle <[email protected]>
|
Adding @slobodanadamovic as reviewer for the new roles in the reference docs. Is there anywhere else that we should expand with more failure store info for security beyond that reference? |
| :::{warning} | ||
| Documents redirected to the failure store in the event of a failed ingest pipeline will be stored in their original, unprocessed form. If an ingest pipeline normally redacts sensitive information from a document, then failed documents in their original, unprocessed form may contain sensitive information. | ||
|
|
||
| Furthermore, failed documents are likely to be structured differently than normal data in a data stream, and thus are not supported by [document level security](../../../deploy-manage/users-roles/cluster-or-deployment-auth/controlling-access-at-document-field-level.md#document-level-security) or [field level security](../../../deploy-manage/users-roles/cluster-or-deployment-auth/controlling-access-at-document-field-level.md#field-level-security). |
There was a problem hiding this comment.
This is actually not true. We decided to support FLS and DLS against failure store. The main reason is because it would not be possible to prevent using FLS/DLS when users define implicit read access to backing .fs* indices. It felt wrong to prevent using FLS/DLS when users defined explicit read_failure_store access to data streams with FLS/DLS restrictions. Our biggest concern was that users would be expecting the DLS/FLS to stop certain docs/fields from being visible and it wouldn't.
Right now, if users include FLS/DLS when granting access to failure store then we'll try to honour it. We should just make sure to highlight it (which you already did) that these documents are structured differently, and because they may contain sensitive information, the users should take extra care when defining access to them.
| @@ -381,6 +384,8 @@ To learn how to assign privileges to a role, refer to [](/deploy-manage/users-ro | |||
|
|
|||
| This privilege is not available in {{serverless-full}}. | |||
|
|
|||
| `read_failure_store` | |||
| : Read-only access to actions performed on a data stream's failure store. Required for access to failure store data (count, explain, get, mget, get indexed scripts, more like this, multi percolate/search/termvector, percolate, scroll, clear_scroll, search, suggest, tv). | |||
There was a problem hiding this comment.
The manage_failure_store and read_failure_store privileges are special in terms that they only grant access to failure store when accessed through data stream names using ::failures selector. We should try and highlight somehow that these privileges cannot be used to grant direct read/manage access to failure store backing indices (.fs*) or any other regular indices. Hence, they should only be granted to data streams that have failure store enabled.
No, that's the only place we should document them. I left some comments. I think we should try to point out that the new privileges are only granting access to failure store when accessed using |
Adds a new section to the documentation to explain new failure store functionality.
Preview:
https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/1368/manage-data/data-store/data-streams/failure-store
This PR relies on links to updates in: