Event forwarding #1371
Event forwarding #1371
Conversation
phil-snowplow
added
the
do not merge
✅ Deploy Preview for snowplow-docs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
re Should we break out Error handling and troubleshooting into its own page? yeah I find it a bit odd to have this section so prominently on the main landing page. A separate page would be good - and consistent with the GTM SS section that has a Testing and debugging page |
| - **Failure timestamp**: when the error occurred | ||
| - **Transformation state**: data state at the point of failure | ||
|
|
||
| ### Querying failed event logs |
There was a problem hiding this comment.
is this section AWS only?
|
|
||
| ## Failure types and retry logic | ||
|
|
||
| Forwarders uses the same retry logic and failure handling as the underlying [Snowbridge failure model](/docs/api-reference/snowbridge/concepts/failure-model/index.md). Snowplow handles event forwarding failures differently depending on the type: |
There was a problem hiding this comment.
Said on call but commenting for reference: I would remove the dependency on api docs and just describe the failure behaviours as below, which are already quite good!
Won't die on this hill but think it's the wiser move
|
|
||
| Forwarders uses the same retry logic and failure handling as the underlying [Snowbridge failure model](/docs/api-reference/snowbridge/concepts/failure-model/index.md). Snowplow handles event forwarding failures differently depending on the type: | ||
|
|
||
| - **Invalid data failures**: Snowplow treats events that fail transformation or violate destination API requirements as unrecoverable. Event forwarding creates [event forwarding error failed events](https://iglucentral.com/?q=event_forwarding_error) and logs them in your cloud storage bucket without retry. |
There was a problem hiding this comment.
Would remove. Everything here should be covered in the points below.
"Invalid" concept is an implementation detail. Some API failures are treated the same but that's best dealt with in the destination bit.
| Forwarders uses the same retry logic and failure handling as the underlying [Snowbridge failure model](/docs/api-reference/snowbridge/concepts/failure-model/index.md). Snowplow handles event forwarding failures differently depending on the type: | ||
|
|
||
| - **Invalid data failures**: Snowplow treats events that fail transformation or violate destination API requirements as unrecoverable. Event forwarding creates [event forwarding error failed events](https://iglucentral.com/?q=event_forwarding_error) and logs them in your cloud storage bucket without retry. | ||
| - **Transformation failures**: Snowplow treats JavaScript transformation errors as invalid data and logs them to your cloud storage bucket without retry. |
There was a problem hiding this comment.
| - **Transformation failures**: Snowplow treats JavaScript transformation errors as invalid data and logs them to your cloud storage bucket without retry. | |
| - **Transformation failures**: Where the mapping transformation encounters an error, event forwarding creates [event forwarding error failed events](https://iglucentral.com/?q=event_forwarding_error) and logs them in your cloud storage bucket without retry. |
Possibly add something like: "There are safeguards against deploying with such errors, but for example custom transformation may sometimes hit errors at runtime."
|
|
||
| - **Invalid data failures**: Snowplow treats events that fail transformation or violate destination API requirements as unrecoverable. Event forwarding creates [event forwarding error failed events](https://iglucentral.com/?q=event_forwarding_error) and logs them in your cloud storage bucket without retry. | ||
| - **Transformation failures**: Snowplow treats JavaScript transformation errors as invalid data and logs them to your cloud storage bucket without retry. | ||
| - **Destination failures**: when API requests fail (e.g., HTTP 4xx/5xx responses), Snowplow retries based on the destination-specific retry policy. See the list of [available destinations](/docs/destinations/forwarding-events/integrations/index.md) for destination-specific details. |
There was a problem hiding this comment.
Actually, this one is good! We could specify that they're either:
- Setup (in which case the customer gets alerted to fix - eg. wrong API key)
- unretryable (in which case we get a bad row, like transformation errors - eg. missing required data)
- retryable (in which case they get retried until they succeed)
| This page is an overview of how to monitor event forwarder performance and diagnose delivery issues. Snowplow provides both summary metrics and detailed failed event logs, to help you understand failure patterns and troubleshoot specific problems. | ||
|
|
||
| ## Failure types and retry logic | ||
|
|
There was a problem hiding this comment.
If suggestions below aren't clean, we could consider reversing the layout.
"Failures can result in one of n behaviours":
- retries {detail}
- Event Forwarding failed events {this happens for transformation errors (eg runtime error), and unrecoverable API errors (eg. missing required data)
- Alerts to fix a problem {this happens for API errors that indicate an incorrect setup, eg. incorrect api key or permissions}
- Oversized failed events {as below}
Draft PR for new Event Forwarding documentation.
Overview of changes
New event forwarding documentation structure:
New component:
Open questions
Still to do
/docs/fundamentals/destinations/section to reflect the changes abovedeprecateddocs/api-reference/failed-events/index.mdwith new event forwarding failure type